NiPyAPI upsert_parameter_to_context timeout issue

[j7smith]

Nipyapi version: 0.19.1
NiFi version: 1.16.0
NiFi-Registry version:
Python version: 3.7
Operating System: Windows/Linux
I am attempting to automate the update/modification of several Parameter Context parameters using the supplied NiPyAPI calls.

I setup a list of ParameterEntity (parameter) then do the following:

# interate the list of parameters
for parameter in parameter_list:
   # get the object to be updated
   parameter_context_object = nipyapi.parameters.get_parameter_context(Name_Of_Parameter_Context, identifier_type='name')

   # update the Parameter Context object
   nipyapi.parameters.upsert_parameter_to_context(parameter_context_object, parameter)

When I execute the code I get the following exception on the initial call of the loop: Timed Out waiting for _update_complete to complete

It seems to (partially) work as it does update the first parameter. I searched for alternatives and documentation but thus far haven't found anything.

At this point I'm blocked. Any suggestion or alternatives?

Note: I've also looked into using the nipyapi.utils.wait_to_complete api call but it made no difference.

[Chaffelson]

A couple of questions:

1. Are you preparing the ParameterEntity using the preparation call? And are you appending it to the ParameterContext or replacing an existing parameter?
   There's examples here in the tests if you are not sure.
2. wait_to_complete etc. won't help here, as it sounds like something isn't being handled causing the update to not return. You may want to turn up verbosity in logging to see what call it gets to and hangs on, if that is indeed what is happening.

If you can share those results here, we can investigate further.

[j7smith]

In regards to question number 2, I contact the admin about the logging. As for the wait_to_complete it makes sense it does not help in this case. Thanks.

As for question number 1. It's a mixed bag scenario. There is a combination of existing parameters to update and new parameters. These are coming from another/existing parameter context.

It starting to make some sense now. I will try using the prepare_parameter and let you know the results

Thanks

[j7smith]

Hi Chaffelson. I did some additional testing using the prepare_parameter API call but still ran into issues. So, I simplified the scenario down to a single case. Adding a new parameter to the Parameter Context. Here is the code:

parameter_context_to_update = nipyapi.parameters.get_parameter_context('BAZ1', identifier_type='name', greedy=False)

prepared_parameter = nipyapi.parameters.prepare_parameter(
   'TEST_PARAMETER_NAME_1', 
   'TEST_PARAMETER_VALUE_1', 
   description='This is a test', 
   sensitive=False
)

nipyapi.parameters.upsert_parameter_to_context(parameter_context_to_update, prepared_parameter)

As before it seems to (partially) work as I do see the new parameter in the Parameter Context but I'm still getting the error message: Timed Out waiting for _update_complete to complete.

[j7smith]

Hi. Any comments or updates on this issue?

[Chaffelson]

Sorry, maintaining this library is a side project for the community and I can't always respond as quickly as I'd like.

It sounds like it is applying the Parameter to the Context, which is good. It also sounds like it's taking a long time for the Canvas to finish updating with the new Parameter and return.

Two things to look at:

1. It looks like I foolishly set a hard-coded timeout of 10s here, this should be replaced with a variable which defaults to the timeout in the main config file. It also looks like there's an unused (and unnecessary) refresh switch on the function which could be deprecated.
2. You should examine the NiFi log to see what is causing such a long delay in the Parameter Context updating. It is likely to be a misbehaving Processor or Controller which is failing to update in a timely manner. It may be useful to determine if you get the timeout when adding an unused Parameter to the Context vs updating an existing one.