Error: undefined method `[]' for nil:NilClass when applying manifest

[aponert]

Describe the Bug

When an inventory variable, which is an array, is used inside a plan, the following error occurs:
undefined method []' for nil:NilClass`

When a scalar value is used, the plan finishes successfully.

Expected Behavior

Both scalar values and arrays should be working inside a plan which applies a manifest block

Steps to Reproduce

I added a plan and two inventory files below in the context section. One inventory file contains a variable which is an array. Running the plan with this inventory should reproduce the failure. The other inventory with the scalar variable is a working example. I just added this to emphasize, that the array var is somehow the reason why it fails.

Environment

• Version 2.14.0 and onwards (earlier versions work)
• Platform macOS 10.15.4 and onwards (not tested with earlier versions)

Additional Context

This bug may be somehow related to #1836 because this bug also occured when I used array inventory vars. This bug was fixed and until 2.14.0 my plan worked flawlessly.

Plan which can the bug be reproduced with

plan reproduce::error (
  TargetSpec $targets,
) {
    $hosts = get_targets($targets)
    $hosts.each |$target| {
    $vars = $target.vars()
    $roles = $vars['roles']
    apply_prep($target)
    apply($target) {
      notify { 'foo':
        message => 'bar'
      }
    }
  }
}

Inventory file which causes problem

---
version: 2
targets:
  - alias: web
    vars:
      roles:
        - array
        - values
    uri: example.org

Inventory file with works

---
version: 2
targets:
  - alias: web
    vars:
      roles: single value
    uri: example.org

[beechtom]

This seems to be a bug introduced by #1798 , specifically these lines: https://github.com/puppetlabs/bolt/pull/1798/files#diff-2ba6f352b8c1a13c88bd62f87af4aec3R76-R82 .

Previously, plan variables were deserialized as part of PAL's catalog compiler ( https://github.com/puppetlabs/bolt/pull/1798/files#diff-2ba6f352b8c1a13c88bd62f87af4aec3L76-L107 ) before collisions with facts and target variables were detected. The change in this PR instead filters out variable collisions before creating the compiler, and sets them as variables when creating the temp environment. Once the catalog compiler is created, Puppet attempts to deserialize the variables and raises an error.

The reason this raises an error seems to be due to the order of the variables in the variables hash. Prior to this change, that hash looks like this:

{"roles"=>{"__ptype"=>"LocalRef", "__pvalue"=>"$['hosts'][0]['vars']['roles']"},
 "targets"=>"example.org",
 "hosts"=>
  [{"__ptype"=>"Target",
    "uri"=>"example.org",
    "name"=>"example.org",
    "safe_name"=>"example.org",
    "target_alias"=>["web"],
    "config"=>{},
    "facts"=>{},
    "vars"=>["array","values"]
  ...

And after this change, the hash looks like this:

{"targets"=>"example.org",
 "hosts"=>
  [{"__ptype"=>"Target",
    "uri"=>"example.org",
    "name"=>"example.org",
    "safe_name"=>"example.org",
    "target_alias"=>["web"],
    "config"=>{},
    "facts"=>{},
    "vars"=>["array","values"]
  ...
 "roles"=>{"__ptype"=>"LocalRef", "__pvalue"=>"$['hosts'][0]['vars']['roles']"}}

[nicklewis]

I don't think it's due to hash order but rather the fact that one key references another key which we remove before deserialization.

The value of the roles variable is deduplicated to be a reference to 'roles' in the target's vars hash. However, the conflict detection notices that vars['roles'] will cause a conflict with roles and so removes it from the hash we pass to populate the scope. Since that happens before the reference is resolved for roles, that resolution later fails.

[aponert]

After reading your comment I tried to rename the $roles variable used in my plan to something different. now that the plan variable's name is different from the key name in vars['roles'] my plan is working as it should. So it seems your assumption is right.