Cyclic imports in cuda-bindings causing import errors

[gflegar]

Is this a duplicate?

• I confirmed there appear to be no duplicate issues for this bug and that I agree to the Code of Conduct

Type of Bug

Compile-time Error

Component

cuda.bindings

Describe the bug

We are using a custom Python environment, where cyclic imports in Python do not work. This is usually not a problem for the vast majority of Python packages, but unfortunately it does cause issues when trying to use cuda-bindings, since there are some cyclic imports between the various cython files.

One example is the cycle between the driver -> _lib.utils -> driver submodules.

Would it be possible to refactor this code to remove the cycle?

How to Reproduce

Run the following code with cuda-bindings installed, which attempts to approximate our import system:

orig_import = __builtins__.__import__

import_stack = []


def import_hook(name, globals=None, locals=None, fromlist=(), *args, **kwargs):
  """Approximate a custom import system that does not allow import cycles."""

  stack_entry = (tuple(fromlist) if fromlist is not None else None, name)
  if stack_entry in import_stack:
    raise ImportError(
        f"Import cycle detected: {stack_entry}, stack: {import_stack}"
    )
  import_stack.append(stack_entry)
  res = orig_import(name, globals, locals, fromlist, *args, **kwargs)
  import_stack.pop()
  return res


__builtins__.__import__ = import_hook


import cuda.bindings.nvvm   # works
import cuda.bindings.nvrtc  # fails due to driver -> _lib.utils -> driver cycle

I get the following error on an Ubuntu system with Python 3.12.9 when running the above code:

Traceback (most recent call last):
  File "import_hook_test.py", line 24, in <module>
    import cuda.bindings.nvrtc  # fails due to driver -> _lib.utils -> driver cycle
    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "import_hook_test.py", line 15, in import_hook
    res = orig_import(name, globals, locals, fromlist, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "cuda/bindings/nvrtc.pyx", line 1, in init cuda.bindings.nvrtc
  File "import_hook_test.py", line 15, in import_hook
    res = orig_import(name, globals, locals, fromlist, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "cuda/bindings/driver.pyx", line 1, in init cuda.bindings.driver
  File "import_hook_test.py", line 15, in import_hook
    res = orig_import(name, globals, locals, fromlist, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "cuda/bindings/_lib/utils.pyx", line 1, in init cuda.bindings._lib.utils
  File "import_hook_test.py", line 11, in import_hook
    raise ImportError(
ImportError: Import cycle detected: ((), 'cuda.bindings.driver'), stack: [(None, 'cuda.bindings.nvrtc'), ((), 'cuda.bindings.driver'), ((), 'cuda.bindings._lib.utils')]

Expected behavior

Imports with the custom hook without raising an exception.

Operating System

Custom linux-based system

nvidia-smi output

N/A - fails at import

[mdboom]

👋 Hi, I'm new to the project and working on this as a good ramp-up task.

Doing some analysis on this, these are the import cycles created when importing cuda.bindings.nvrtc:

I think I've found a way to remove the cycle involving cuda.bindings.utils._get_handle (by having modules self-register their types in the handle map, rather than pre-generating it). That gets us something like this (which also imports fewer things by default):

The cycle between cuda.bindings._lib.utils and cuda.bindings.driver is going to be a lot more challenging. The best I can think of would involve changing how Cython works to resolve cross-module symbols lazily after import time, but that's "non-trivial".

I will probably pursue a patch to remove the _get_handle loop in any event, since I think it has other advantages.

[gflegar]

👋

The cycle between cuda.bindings._lib.utils and cuda.bindings.driver is going to be a lot more challenging.

Yeah, I was also trying to see how easy it would be to move a few things around as a patch before posting this, and this is exactly where I got stuck and figured out there's a lot of interdependencies between the two files, so moving a few things around is not super easy. (And the error message I see internally is complaining about exactly these two files depending on one another.)

[leofang]

@gflegar while we're working on untangling the imports, we'd still like to understand better why this trips you, since the said circular import has been there since the beginning of this project, and it's been working fine over the years and across multiple Python versions. It seems that Cython handles the cimports properly for regular Python users, and this is different from normal Python imports.

@oleksandr-pavlyk and I have looked into if this is related to the multi-phase initialization (I tried rebuilding the package with DCYTHON_PEP489_MULTI_PHASE_INIT=0 but found nothing different). I'd like to know:

• Which Python interpreter do you use? (CPython, PyPy, ...)
• What is your Python version?
• Do you build cuda-bindings from source or pip install the binary wheel?
  • If the former, what is your Cython version?

I am looking for ways to reproduce your Python environment without the provided import hook, so as to understand this issue better.