deploy_rsconnect causes kernel to die unexpectedly

[FMKerckhof]

Describe the bug

Dear, I am trying to deploy an sklearn random forest model to Posit connect from within an ipython notebook in jupyterlab on Posit workbench.
I have followed the python instructions from the Get started-guide as well as the Posit youtube video on MLOps and all works well until I am trying to deploy the model using vetiver.deploy_rsconnect where I get the following error: RSConnectException: -n/--name cannot be specified in conjunction with options -s/--server (full error trace below)

Full error trace

/opt/python/3.10.6/lib/python3.10/site-packages/vetiver/write_fastapi.py:21: UserWarning: Pinned vetiver model has no active version and no datetime on versions,
              Do you need to check your pinned model?
              Using version 4509
  warnings.warn(

---------------------------------------------------------------------------
RSConnectException                        Traceback (most recent call last)
Input In [10], in <cell line: 1>()
----> 1 vetiver.deploy_rsconnect(
      2     connect_server = connect_server,
      3     board = model_board,
      4     pin_name = "kytos_lab/cars_mpg",
      5 )

File /opt/python/3.10.6/lib/python3.10/site-packages/vetiver/rsconnect.py:104, in deploy_rsconnect(connect_server, board, pin_name, version, extra_files, new, app_id, title, python, conda_mode, force_generate, log_callback, image)
     94 tmp_app = temp + "/app.py"
     96 write_app(
     97     board=board,
     98     pin_name=pin_name,
   (...)
    101     overwrite=False,
    102 )
--> 104 deploy_python_fastapi(
    105     connect_server=connect_server,
    106     directory=temp,
    107     extra_files=extra_files,
    108     excludes=None,
    109     entry_point="app:api",
    110     new=new,
    111     app_id=app_id,
    112     title=title,
    113     python=python,
    114     conda_mode=conda_mode,
    115     force_generate=force_generate,
    116     log_callback=log_callback,
    117     image=image,
    118 )

File /opt/python/3.10.6/lib/python3.10/site-packages/rsconnect/actions.py:897, in deploy_python_fastapi(connect_server, directory, extra_files, excludes, entry_point, new, app_id, title, python, conda_mode, force_generate, log_callback, image)
    856 def deploy_python_fastapi(
    857     connect_server: api.RSConnectServer,
    858     directory: str,
   (...)
    869     image: str = None,
    870 ) -> typing.Tuple[str, typing.Union[list, None]]:
    871     """
    872     A function to deploy a Python ASGI API module to RStudio Connect.  Depending on the files involved
    873         and network latency, this may take a bit of time.
   (...)
    895         of log lines.  The log lines value will be None if a log callback was provided.
    896     """
--> 897     return _deploy_by_python_framework(
    898         connect_server,
    899         directory,
    900         extra_files,
    901         excludes,
    902         entry_point,
    903         gather_basic_deployment_info_for_fastapi,
    904         image,
    905         new,
    906         app_id,
    907         title,
    908         python,
    909         conda_mode,
    910         force_generate,
    911         log_callback,
    912     )

File /opt/python/3.10.6/lib/python3.10/site-packages/rsconnect/actions.py:1210, in _deploy_by_python_framework(connect_server, directory, extra_files, excludes, entry_point, gatherer, image, new, app_id, title, python, conda_mode, force_generate, log_callback)
   1201 _, environment = get_python_env_info(
   1202     directory,
   1203     python,
   1204     conda_mode=conda_mode,
   1205     force_generate=force_generate,
   1206 )
   1207 bundle = create_api_deployment_bundle(
   1208     directory, extra_files, excludes, entry_point, app_mode, environment, True, image
   1209 )
-> 1210 return _finalize_deploy(
   1211     connect_server,
   1212     app_store,
   1213     directory,
   1214     app_id,
   1215     app_mode,
   1216     deployment_name,
   1217     deployment_title,
   1218     default_title,
   1219     bundle,
   1220     log_callback,
   1221 )

File /opt/python/3.10.6/lib/python3.10/site-packages/rsconnect/actions.py:769, in _finalize_deploy(connect_server, app_store, file_name, app_id, app_mode, name, title, title_is_default, bundle, log_callback)
    737 def _finalize_deploy(
    738     connect_server: api.RSConnectServer,
    739     app_store: AppStore,
   (...)
    747     log_callback: typing.Callable,
    748 ) -> typing.Tuple[str, typing.Union[list, None]]:
    749     """
    750     A common function to finish up the deploy process once all the data (bundle
    751     included) has been resolved.
   (...)
    767     of log lines.  The log lines value will be None if a log callback was provided.
    768     """
--> 769     app = deploy_bundle(connect_server, app_id, name, title, title_is_default, bundle, None)
    770     app_url, log_lines, _ = spool_deployment_log(connect_server, app, log_callback)
    771     app_store.set(
    772         connect_server.url,
    773         abspath(file_name),
   (...)
    778         app_mode,
    779     )

File /opt/python/3.10.6/lib/python3.10/site-packages/rsconnect/actions.py:1793, in deploy_bundle(remote_server, app_id, name, title, title_is_default, bundle, env_vars)
   1771 def deploy_bundle(
   1772     remote_server: api.TargetableServer,
   1773     app_id: int,
   (...)
   1778     env_vars: typing.List[typing.Tuple[str, str]],
   1779 ) -> typing.Dict[str, typing.Any]:
   1780     """
   1781     Deploys the specified bundle.
   1782 
   (...)
   1791     task that may be queried for deployment progress.
   1792     """
-> 1793     ce = RSConnectExecutor(
   1794         server=remote_server,
   1795         app_id=app_id,
   1796         name=name,
   1797         title=title,
   1798         title_is_default=title_is_default,
   1799         bundle=bundle,
   1800         env_vars=env_vars,
   1801     )
   1802     ce.deploy_bundle()
   1803     return ce.state["deployed_info"]

File /opt/python/3.10.6/lib/python3.10/site-packages/rsconnect/api.py:341, in RSConnectExecutor.__init__(self, name, url, api_key, insecure, cacert, ca_data, cookies, account, token, secret, timeout, logger, **kwargs)
    339 self.reset()
    340 self._d = kwargs
--> 341 self.setup_remote_server(
    342     name=name,
    343     url=url or kwargs.get("server"),
    344     api_key=api_key,
    345     insecure=insecure,
    346     cacert=cacert,
    347     ca_data=ca_data,
    348     account_name=account,
    349     token=token,
    350     secret=secret,
    351 )
    352 self.setup_client(cookies, timeout)
    353 self.logger = logger

File /opt/python/3.10.6/lib/python3.10/site-packages/rsconnect/api.py:390, in RSConnectExecutor.setup_remote_server(self, name, url, api_key, insecure, cacert, ca_data, account_name, token, secret)
    378 def setup_remote_server(
    379     self,
    380     name: str = None,
   (...)
    388     secret: str = None,
    389 ):
--> 390     validation.validate_connection_options(
    391         url=url,
    392         api_key=api_key,
    393         insecure=insecure,
    394         cacert=cacert,
    395         account_name=account_name,
    396         token=token,
    397         secret=secret,
    398         name=name,
    399     )
    401     if cacert and not ca_data:
    402         ca_data = text_type(cacert.read())

File /opt/python/3.10.6/lib/python3.10/site-packages/rsconnect/validation.py:21, in validate_connection_options(url, api_key, insecure, cacert, account_name, token, secret, name)
     18 present_options_mutually_exclusive_with_name = _get_present_options(options_mutually_exclusive_with_name)
     20 if name and present_options_mutually_exclusive_with_name:
---> 21     raise RSConnectException(
     22         "-n/--name cannot be specified in conjunction with options {}".format(
     23             ", ".join(present_options_mutually_exclusive_with_name)
     24         )
     25     )
     26 if not name and not url and not shinyapps_options:
     27     raise RSConnectException(
     28         "You must specify one of -n/--name OR -s/--server OR -A/--account, -T/--token, -S/--secret."
     29     )

RSConnectException: -n/--name cannot be specified in conjunction with options -s/--server

I find the documentation generally a bit limited to figure out what is going on. I have been looking at the Vetiver python function reference, the documentation of Posit connect on deploying Vetiver and the Rsconnect-python documentation but I get sometimes contradictory information (for example should I use deploy_rsconnect or deploy_python_fastapi?).

Because my notebook kernel kept dying unexpectedly (after very long waits of doing nothing), I tried to also execute the deploy_python_fastapi in interactive python REPL from the terminal - this just keeps hanging for hours without any errors.

I did not yet try to directly use the rsconnect-python CLI outside of an interactive python environment, but I hoped that I could just run the example without any problems.

To Reproduce

I tried to publish the default example to my posit connect server and I observe the same error. To reproduce this example, you will need to have a Posit connect server to publish to.

from vetiver.data import mtcars
from vetiver import VetiverModel, write_app, vetiver_pin_write, VetiverAPI
from sklearn import tree
from pins import board_folder, board_rsconnect 
from rsconnect.actions import deploy_python_fastapi
from rsconnect.api import RSConnectServer

# below you should change the values or use environmental variables to load them
API_KEY = <get api key from env var>
SERVER = <get connect server address from env var>

connect_server = RSConnectServer(
    url=SERVER, # load from an .env file
    api_key=API_KEY # load from an .env file
)

model_board = board_rsconnect(server_url=SERVER,
                        api_key=API_KEY,
                        allow_pickle_read=True,
                        versioned=True
                       )
                       
car_mod = tree.DecisionTreeRegressor().fit(mtcars.drop(columns="mpg"), mtcars["mpg"])

# don't forget to replace user_name with the user associated with the API_KEY

v = VetiverModel(car_mod, model_name = "user_name/cars_mpg", 
                 prototype_data = mtcars.drop(columns="mpg"))

vetiver_pin_write(model_board, v)

app = VetiverAPI(v, check_prototype=True)

deploy_rsconnect(
    connect_server = connect_server,
    board = model_board,
    pin_name = "user_name/cars_mpg",
)

Expected behavior

I would expect that following the code above allows me to deploy my model to Posit connect

Desktop (please complete the following information):

• OS: Ubuntu 22.04 LTS
• Browser: Firefox 118.0.2 (64-bit)
• Version: Posit Connect v2023.09.0-0-g3448da6cf4 - Posit Workbench 2023.09.0+463.pro11, “Desert Sunflower” (6cc4ae5f)
• Python version: 3.10.6

[isabelizimm]

Hi there @FMKerckhof, thank you for opening up this issue! That is a little perplexing...what version of vetiver and rsconnect-python are you using? This error had previously popped up in rsconnect-python, and vetiver pinned rsconnect-python to avoid older versions and hopefully avoid this. If possible, I'd recommend running pip install --upgrade vetiver rsconnect-python to get the latest versions. Let me know if that helps!

As for documentation, we are in the process of updating all the information on the Connect docs to use vetiver.deploy_rsconnect, but you make a great point that some sort of trouble-shooting might be useful.

[FMKerckhof]

Hi @isabelizimm , thank you for the swift reply - I am using vetiver 0.2.1 and rsconnect-python 1.20.0. I will upgrade vetiver to latest (0.2.3) and get back to you.

[FMKerckhof]

I upgraded vetiver to the latest (0.2.3) using pip as instructed, and the issue still persists - I get the exact same error message.

[FMKerckhof]

@isabelizimm apologies that I did not catch this earlier - since I am working on Posit workbench, I have a python version installed in /opt/python/... (as per Posit docs) - as well as a system python. On both I also had a version of rsconnect-python installed. In the terminal (and the Posit Workbench terminal) rsconnect-python was 1.20.0, but for jupyterlab the rsconnect-python version was 1.10.0 - after updating deploy did start.

In case other users would land here because deploys fail: I had to add a very minimal requirements.txt to the "extra_files" argument of deploy_rsconnect - in my case, automatic resolution caused the deploy to crash because I had RPy2 imported in the environment (and it is very hard to publish something using RPy2 to connect apparently). If I did not specify this, pip freeze would create a very large requirements.txt.

Finally, @isabelizimm I would like to ask you something about a warning which I do not really understand (unless I should make a dedicated issue out of this?):

/opt/python/3.10.6/lib/python3.10/site-packages/vetiver/write_fastapi.py:21: UserWarning: Pinned vetiver model has no active version and no datetime on versions,
              Do you need to check your pinned model?
              Using version 4524

Could you tell me what this means, how should we "activate" a pinned vetiver model?

[isabelizimm]

This is a super helpful update! That's a good thing to note--in Posit Workbench, oftentimes people specify a Python environment to "deploy," that is different than the one they are working in.

The way to activate a pinned version would be to pass in a version number as an argument (eg, your function might look like deploy_rsconnect(connect_server, board, pin_name, version="4524", extra_files=['requirements.txt'])) . Otherwise, it will deploy the latest version of the pin. This warning primarily exists to nudge users to ensure the pin they expect is being used. I'm open to any feedback if there's a way to make this more clear for users (in this issue or a separate one)!

[FMKerckhof]

Thanks - I don't think it's necessary to raise an issue - more a matter of documentation. Is there a way I could contribute to the documentation?

I have documented R packages in the past, but have only very limited experience with python - nonetheless it seems like you are using the Makefile to autogenerate some documentation - for deploy_rsconnect I take it takes the info from here?

[isabelizimm]

Yes please! Any documentation contribution would be so welcome 😄 And you are correct, that's the correct spot in deploy_rsconnect! In the Python world, docs are usually generated from docstrings, which is what you linked, similar to roxygen comments in R. Vetiver uses a numpydoc style, and you'll likely want to use something like a "Notes" section to add extra context to a function.

[isabelizimm]

I'm going to close out this issue since the original question has been resolved, but please let me know if you have any follow up questions/comments!