Revert stubgen-related patches #157831

ftynse · 2025-09-10T10:57:04Z

Despite several hotfixes, things remain broken, in particular:

installation/distribution (ninja install / install-distribution);
downstream projects with bindings exposed.

See #157583 (comment) for more details.

Reverts #155741, #157583, #157697. Let's make sure things are fixed and re-land as a unit.

…157697)" This reverts commit 53bcfe2.

This reverts commit d02c7ae.

This reverts commit 22c2e15.

llvmbot · 2025-09-10T10:57:33Z

@llvm/pr-subscribers-mlir

Author: Oleksandr "Alex" Zinenko (ftynse)

Changes

Despite several hotfixes, things remain broken, in particular:

installation/distribution (ninja install / install-distribution);
downstream projects with bindings exposed.

See #157583 (comment) for more details.

Reverts #155741, #157583, #157697. Let's make sure things are fixed and re-land as a unit.

Patch is 117.87 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/157831.diff

13 Files Affected:

(modified) .ci/all_requirements.txt (+3-7)
(modified) mlir/cmake/modules/AddMLIRPython.cmake (+20-107)
(modified) mlir/examples/standalone/python/CMakeLists.txt (-1)
(modified) mlir/python/CMakeLists.txt (+12-46)
(removed) mlir/python/mlir/_mlir_libs/.gitignore (-2)
(added) mlir/python/mlir/_mlir_libs/_mlir/init.pyi (+12)
(added) mlir/python/mlir/_mlir_libs/_mlir/dialects/pdl.pyi (+63)
(added) mlir/python/mlir/_mlir_libs/_mlir/dialects/quant.pyi (+142)
(added) mlir/python/mlir/_mlir_libs/_mlir/dialects/transform/init.pyi (+25)
(added) mlir/python/mlir/_mlir_libs/_mlir/ir.pyi (+2846)
(added) mlir/python/mlir/_mlir_libs/_mlir/passmanager.pyi (+36)
(added) mlir/python/mlir/_mlir_libs/_mlirExecutionEngine.pyi (+24)
(modified) mlir/python/requirements.txt (+2-3)

diff --git a/.ci/all_requirements.txt b/.ci/all_requirements.txt
index ac9682a09bec1..dea9646f648ad 100644
--- a/.ci/all_requirements.txt
+++ b/.ci/all_requirements.txt
@@ -194,9 +194,9 @@ ml-dtypes==0.5.1 ; python_version < "3.13" \
     --hash=sha256:d13755f8e8445b3870114e5b6240facaa7cb0c3361e54beba3e07fa912a6e12b \
     --hash=sha256:fd918d4e6a4e0c110e2e05be7a7814d10dc1b95872accbf6512b80a109b71ae1
     # via -r mlir/python/requirements.txt
-nanobind==2.9.2 \
-    --hash=sha256:c37957ffd5eac7eda349cff3622ecd32e5ee1244ecc912c99b5bc8188bafd16e \
-    --hash=sha256:e7608472de99d375759814cab3e2c94aba3f9ec80e62cfef8ced495ca5c27d6e
+nanobind==2.7.0 \
+    --hash=sha256:73b12d0e751d140d6c1bf4b215e18818a8debfdb374f08dc3776ad208d808e74 \
+    --hash=sha256:f9f1b160580c50dcf37b6495a0fd5ec61dc0d95dae5f8004f87dd9ad7eb46b34
     # via -r mlir/python/requirements.txt
 numpy==2.0.2 \
     --hash=sha256:0123ffdaa88fa4ab64835dcbde75dcdf89c453c922f18dced6e27c90d1d0ec5a \
@@ -383,10 +383,6 @@ swig==4.3.1 \
     --hash=sha256:efec16327029f682f649a26da726bb0305be8800bd0f1fa3e81bf0769cf5b476 \
     --hash=sha256:fc496c0d600cf1bb2d91e28d3d6eae9c4301e5ea7a0dec5a4281b5efed4245a8
     # via -r lldb/test/requirements.txt
-typing-extensions==4.15.0 \
-    --hash=sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466 \
-    --hash=sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548
-    # via -r mlir/python/requirements.txt
 urllib3==2.5.0 \
     --hash=sha256:3fc47733c7e419d4bc3f6b3dc2b4f890bb743906a30d56ba4a5bfa4bbff92760 \
     --hash=sha256:e6b01673c0fa6a13e374b50871808eb3bf7046c4b125b216f6bf1cc604cff0dc
diff --git a/mlir/cmake/modules/AddMLIRPython.cmake b/mlir/cmake/modules/AddMLIRPython.cmake
index ee6c87a8b635e..2b883558d33c6 100644
--- a/mlir/cmake/modules/AddMLIRPython.cmake
+++ b/mlir/cmake/modules/AddMLIRPython.cmake
@@ -99,63 +99,6 @@ function(declare_mlir_python_sources name)
   endif()
 endfunction()
 
-# Function: generate_type_stubs
-# Turns on automatic type stub generation (via nanobind's stubgen) for extension modules.
-# Arguments:
-#   MODULE_NAME: The name of the extension module as specified in declare_mlir_python_extension.
-#   DEPENDS_TARGET: The dso target corresponding to the extension module
-#     (e.g., something like StandalonePythonModules.extension._standaloneDialectsNanobind.dso)
-#   MLIR_DEPENDS_TARGET: The dso target corresponding to the main/core extension module
-#     (e.g., something like StandalonePythonModules.extension._mlir.dso)
-#   OUTPUT_DIR: The root output directory to emit the type stubs into.
-# Outputs:
-#   NB_STUBGEN_CUSTOM_TARGET: The target corresponding to generation which other targets can depend on.
-function(generate_type_stubs MODULE_NAME DEPENDS_TARGET MLIR_DEPENDS_TARGET OUTPUT_DIR)
-  cmake_parse_arguments(ARG
-    ""
-    ""
-    "OUTPUTS"
-    ${ARGN})
-  if(EXISTS ${nanobind_DIR}/../src/stubgen.py)
-    set(NB_STUBGEN "${nanobind_DIR}/../src/stubgen.py")
-  elseif(EXISTS ${nanobind_DIR}/../stubgen.py)
-    set(NB_STUBGEN "${nanobind_DIR}/../stubgen.py")
-  else()
-    message(FATAL_ERROR "generate_type_stubs(): could not locate 'stubgen.py'!")
-  endif()
-  file(REAL_PATH "${NB_STUBGEN}" NB_STUBGEN)
-
-  set(_module "${MLIR_PYTHON_PACKAGE_PREFIX}._mlir_libs.${MODULE_NAME}")
-  file(REAL_PATH "${MLIR_BINARY_DIR}/${MLIR_BINDINGS_PYTHON_INSTALL_PREFIX}/.." _import_path)
-
-  set(NB_STUBGEN_CMD
-      "${Python_EXECUTABLE}"
-      "${NB_STUBGEN}"
-      --module
-      "${_module}"
-      -i
-      "${_import_path}"
-      --recursive
-      --include-private
-      --output-dir
-      "${OUTPUT_DIR}"
-      --quiet)
-
-  list(TRANSFORM ARG_OUTPUTS PREPEND "${OUTPUT_DIR}/" OUTPUT_VARIABLE _generated_type_stubs)
-  add_custom_command(
-    OUTPUT ${_generated_type_stubs}
-    COMMAND ${NB_STUBGEN_CMD}
-    WORKING_DIRECTORY "${CMAKE_CURRENT_FUNCTION_LIST_DIR}"
-    DEPENDS
-      "${MLIR_DEPENDS_TARGET}.extension._mlir.dso"
-      "${MLIR_DEPENDS_TARGET}.sources.MLIRPythonSources.Core.Python"
-      "${DEPENDS_TARGET}"
-  )
-  set(_name "MLIRPythonModuleStubs_${_module}")
-  add_custom_target("${_name}" ALL DEPENDS ${_generated_type_stubs})
-  set(NB_STUBGEN_CUSTOM_TARGET "${_name}" PARENT_SCOPE)
-endfunction()
-
 # Function: declare_mlir_python_extension
 # Declares a buildable python extension from C++ source files. The built
 # module is considered a python source file and included as everything else.
@@ -172,12 +115,11 @@ endfunction()
 #     on. These will be collected for all extensions and put into an
 #     aggregate dylib that is linked against.
 #   PYTHON_BINDINGS_LIBRARY: Either pybind11 or nanobind.
-#   GENERATE_TYPE_STUBS: List of generated type stubs expected from stubgen relative to _mlir_libs.
 function(declare_mlir_python_extension name)
   cmake_parse_arguments(ARG
     ""
     "ROOT_DIR;MODULE_NAME;ADD_TO_PARENT;PYTHON_BINDINGS_LIBRARY"
-    "SOURCES;PRIVATE_LINK_LIBS;EMBED_CAPI_LINK_LIBS;GENERATE_TYPE_STUBS"
+    "SOURCES;PRIVATE_LINK_LIBS;EMBED_CAPI_LINK_LIBS"
     ${ARGN})
 
   if(NOT ARG_ROOT_DIR)
@@ -193,13 +135,12 @@ function(declare_mlir_python_extension name)
   set_target_properties(${name} PROPERTIES
     # Yes: Leading-lowercase property names are load bearing and the recommended
     # way to do this: https://gitlab.kitware.com/cmake/cmake/-/issues/19261
-    EXPORT_PROPERTIES "mlir_python_SOURCES_TYPE;mlir_python_EXTENSION_MODULE_NAME;mlir_python_EMBED_CAPI_LINK_LIBS;mlir_python_DEPENDS;mlir_python_BINDINGS_LIBRARY;mlir_python_GENERATE_TYPE_STUBS"
+    EXPORT_PROPERTIES "mlir_python_SOURCES_TYPE;mlir_python_EXTENSION_MODULE_NAME;mlir_python_EMBED_CAPI_LINK_LIBS;mlir_python_DEPENDS;mlir_python_BINDINGS_LIBRARY"
     mlir_python_SOURCES_TYPE extension
     mlir_python_EXTENSION_MODULE_NAME "${ARG_MODULE_NAME}"
     mlir_python_EMBED_CAPI_LINK_LIBS "${ARG_EMBED_CAPI_LINK_LIBS}"
     mlir_python_DEPENDS ""
     mlir_python_BINDINGS_LIBRARY "${ARG_PYTHON_BINDINGS_LIBRARY}"
-    mlir_python_GENERATE_TYPE_STUBS "${ARG_GENERATE_TYPE_STUBS}"
   )
 
   # Set the interface source and link_libs properties of the target
@@ -302,32 +243,6 @@ function(add_mlir_python_modules name)
       )
       add_dependencies(${modules_target} ${_extension_target})
       mlir_python_setup_extension_rpath(${_extension_target})
-      get_target_property(_generate_type_stubs ${sources_target} mlir_python_GENERATE_TYPE_STUBS)
-      if(_generate_type_stubs)
-        generate_type_stubs(
-          ${_module_name}
-          ${_extension_target}
-          ${name}
-          "${CMAKE_CURRENT_BINARY_DIR}/_mlir_libs"
-          OUTPUTS "${_generate_type_stubs}"
-        )
-        add_dependencies("${modules_target}" "${NB_STUBGEN_CUSTOM_TARGET}")
-        set(_stubgen_target "${MLIR_PYTHON_PACKAGE_PREFIX}.${_module_name}_type_stub_gen")
-        declare_mlir_python_sources(
-          ${_stubgen_target}
-          ROOT_DIR "${CMAKE_CURRENT_BINARY_DIR}/_mlir_libs"
-          ADD_TO_PARENT "${sources_target}"
-          SOURCES "${_generate_type_stubs}"
-        )
-        set(_pure_sources_target "${modules_target}.sources.${sources_target}_type_stub_gen")
-        add_mlir_python_sources_target(${_pure_sources_target}
-          INSTALL_COMPONENT ${modules_target}
-          INSTALL_DIR "${ARG_INSTALL_PREFIX}/_mlir_libs"
-          OUTPUT_DIRECTORY "${ARG_ROOT_PREFIX}/_mlir_libs"
-          SOURCES_TARGETS ${_stubgen_target}
-        )
-        add_dependencies(${modules_target} ${_pure_sources_target})
-      endif()
     else()
       message(SEND_ERROR "Unrecognized source type '${_source_type}' for python source target ${sources_target}")
       return()
@@ -763,28 +678,26 @@ function(add_mlir_python_extension libname extname)
       # the super project handle compile options as it wishes.
       get_property(NB_LIBRARY_TARGET_NAME TARGET ${libname} PROPERTY LINK_LIBRARIES)
       target_compile_options(${NB_LIBRARY_TARGET_NAME}
-        PRIVATE
-          -Wall -Wextra -Wpedantic
-          -Wno-c++98-compat-extra-semi
-          -Wno-cast-qual
-          -Wno-covered-switch-default
-          -Wno-deprecated-literal-operator
-          -Wno-nested-anon-types
-          -Wno-unused-parameter
-          -Wno-zero-length-array
-          ${eh_rtti_enable})
+	PRIVATE
+	  -Wall -Wextra -Wpedantic
+	  -Wno-c++98-compat-extra-semi
+	  -Wno-cast-qual
+	  -Wno-covered-switch-default
+	  -Wno-nested-anon-types
+	  -Wno-unused-parameter
+	  -Wno-zero-length-array
+	  ${eh_rtti_enable})
 
       target_compile_options(${libname}
-        PRIVATE
-          -Wall -Wextra -Wpedantic
-          -Wno-c++98-compat-extra-semi
-          -Wno-cast-qual
-          -Wno-covered-switch-default
-          -Wno-deprecated-literal-operator
-          -Wno-nested-anon-types
-          -Wno-unused-parameter
-          -Wno-zero-length-array
-          ${eh_rtti_enable})
+	PRIVATE
+	  -Wall -Wextra -Wpedantic
+	  -Wno-c++98-compat-extra-semi
+	  -Wno-cast-qual
+	  -Wno-covered-switch-default
+	  -Wno-nested-anon-types
+	  -Wno-unused-parameter
+	  -Wno-zero-length-array
+	  ${eh_rtti_enable})
     endif()
 
     if(APPLE)
diff --git a/mlir/examples/standalone/python/CMakeLists.txt b/mlir/examples/standalone/python/CMakeLists.txt
index cb10518e94e33..a0eca9c095775 100644
--- a/mlir/examples/standalone/python/CMakeLists.txt
+++ b/mlir/examples/standalone/python/CMakeLists.txt
@@ -39,7 +39,6 @@ declare_mlir_python_extension(StandalonePythonSources.NanobindExtension
   EMBED_CAPI_LINK_LIBS
     StandaloneCAPI
   PYTHON_BINDINGS_LIBRARY nanobind
-  GENERATE_TYPE_STUBS
 )
 
 
diff --git a/mlir/python/CMakeLists.txt b/mlir/python/CMakeLists.txt
index 8e7949480f21e..c983914722ce1 100644
--- a/mlir/python/CMakeLists.txt
+++ b/mlir/python/CMakeLists.txt
@@ -1,9 +1,5 @@
 include(AddMLIRPython)
 
-# Specifies that all MLIR packages are co-located under the `mlir_standalone`
-# top level package (the API has been embedded in a relocatable way).
-add_compile_definitions("MLIR_PYTHON_PACKAGE_PREFIX=${MLIR_PYTHON_PACKAGE_PREFIX}.")
-
 ################################################################################
 # Structural groupings.
 ################################################################################
@@ -27,6 +23,11 @@ declare_mlir_python_sources(MLIRPythonSources.Core.Python
     passmanager.py
     rewrite.py
     dialects/_ods_common.py
+
+    # The main _mlir module has submodules: include stubs from each.
+    _mlir_libs/_mlir/__init__.pyi
+    _mlir_libs/_mlir/ir.pyi
+    _mlir_libs/_mlir/passmanager.pyi
 )
 
 declare_mlir_python_sources(MLIRPythonSources.Core.Python.Extras
@@ -42,6 +43,7 @@ declare_mlir_python_sources(MLIRPythonSources.ExecutionEngine
   ADD_TO_PARENT MLIRPythonSources
   SOURCES
     execution_engine.py
+    _mlir_libs/_mlirExecutionEngine.pyi
   SOURCES_GLOB
     runtime/*.py
 )
@@ -193,6 +195,7 @@ declare_mlir_dialect_python_bindings(
   TD_FILE dialects/TransformOps.td
   SOURCES
     dialects/transform/__init__.py
+    _mlir_libs/_mlir/dialects/transform/__init__.pyi
   DIALECT_NAME transform
   GEN_ENUM_BINDINGS_TD_FILE
     "../../include/mlir/Dialect/Transform/IR/TransformAttrs.td"
@@ -364,7 +367,8 @@ declare_mlir_python_sources(
   ROOT_DIR "${CMAKE_CURRENT_SOURCE_DIR}/mlir"
   GEN_ENUM_BINDINGS
   SOURCES
-    dialects/quant.py)
+    dialects/quant.py
+    _mlir_libs/_mlir/dialects/quant.pyi)
 
 declare_mlir_dialect_python_bindings(
   ADD_TO_PARENT MLIRPythonSources.Dialects
@@ -380,6 +384,7 @@ declare_mlir_dialect_python_bindings(
   TD_FILE dialects/PDLOps.td
   SOURCES
     dialects/pdl.py
+    _mlir_libs/_mlir/dialects/pdl.pyi
   DIALECT_NAME pdl)
 
 declare_mlir_dialect_python_bindings(
@@ -505,11 +510,6 @@ declare_mlir_python_extension(MLIRPythonExtension.Core
 
     # Dialects
     MLIRCAPIFunc
-  GENERATE_TYPE_STUBS
-    "_mlir/__init__.pyi"
-    "_mlir/ir.pyi"
-    "_mlir/passmanager.pyi"
-    "_mlir/rewrite.pyi"
 )
 
 # This extension exposes an API to register all dialects, extensions, and passes
@@ -531,8 +531,6 @@ declare_mlir_python_extension(MLIRPythonExtension.RegisterEverything
     MLIRCAPIConversion
     MLIRCAPITransforms
     MLIRCAPIRegisterEverything
-  GENERATE_TYPE_STUBS
-    "_mlirRegisterEverything.pyi"
 )
 
 declare_mlir_python_extension(MLIRPythonExtension.Dialects.Linalg.Pybind
@@ -547,8 +545,6 @@ declare_mlir_python_extension(MLIRPythonExtension.Dialects.Linalg.Pybind
   EMBED_CAPI_LINK_LIBS
     MLIRCAPIIR
     MLIRCAPILinalg
-  GENERATE_TYPE_STUBS
-    "_mlirDialectsLinalg.pyi"
 )
 
 declare_mlir_python_extension(MLIRPythonExtension.Dialects.GPU.Pybind
@@ -563,8 +559,6 @@ declare_mlir_python_extension(MLIRPythonExtension.Dialects.GPU.Pybind
   EMBED_CAPI_LINK_LIBS
     MLIRCAPIIR
     MLIRCAPIGPU
-  GENERATE_TYPE_STUBS
-    "_mlirDialectsGPU.pyi"
 )
 
 declare_mlir_python_extension(MLIRPythonExtension.Dialects.LLVM.Pybind
@@ -579,8 +573,6 @@ declare_mlir_python_extension(MLIRPythonExtension.Dialects.LLVM.Pybind
   EMBED_CAPI_LINK_LIBS
     MLIRCAPIIR
     MLIRCAPILLVM
-  GENERATE_TYPE_STUBS
-    "_mlirDialectsLLVM.pyi"
 )
 
 declare_mlir_python_extension(MLIRPythonExtension.Dialects.Quant.Pybind
@@ -595,8 +587,6 @@ declare_mlir_python_extension(MLIRPythonExtension.Dialects.Quant.Pybind
   EMBED_CAPI_LINK_LIBS
     MLIRCAPIIR
     MLIRCAPIQuant
-  GENERATE_TYPE_STUBS
-    "_mlirDialectsQuant.pyi"
 )
 
 declare_mlir_python_extension(MLIRPythonExtension.Dialects.NVGPU.Pybind
@@ -611,8 +601,6 @@ declare_mlir_python_extension(MLIRPythonExtension.Dialects.NVGPU.Pybind
   EMBED_CAPI_LINK_LIBS
     MLIRCAPIIR
     MLIRCAPINVGPU
-  GENERATE_TYPE_STUBS
-    "_mlirDialectsNVGPU.pyi"
 )
 
 declare_mlir_python_extension(MLIRPythonExtension.Dialects.PDL.Pybind
@@ -627,8 +615,6 @@ declare_mlir_python_extension(MLIRPythonExtension.Dialects.PDL.Pybind
   EMBED_CAPI_LINK_LIBS
     MLIRCAPIIR
     MLIRCAPIPDL
-  GENERATE_TYPE_STUBS
-    "_mlirDialectsPDL.pyi"
 )
 
 declare_mlir_python_extension(MLIRPythonExtension.Dialects.SparseTensor.Pybind
@@ -643,8 +629,6 @@ declare_mlir_python_extension(MLIRPythonExtension.Dialects.SparseTensor.Pybind
   EMBED_CAPI_LINK_LIBS
     MLIRCAPIIR
     MLIRCAPISparseTensor
-  GENERATE_TYPE_STUBS
-    "_mlirDialectsSparseTensor.pyi"
 )
 
 declare_mlir_python_extension(MLIRPythonExtension.Dialects.Transform.Pybind
@@ -659,8 +643,6 @@ declare_mlir_python_extension(MLIRPythonExtension.Dialects.Transform.Pybind
   EMBED_CAPI_LINK_LIBS
     MLIRCAPIIR
     MLIRCAPITransformDialect
-  GENERATE_TYPE_STUBS
-    "_mlirDialectsTransform.pyi"
 )
 
 declare_mlir_python_extension(MLIRPythonExtension.AsyncDialectPasses
@@ -674,8 +656,6 @@ declare_mlir_python_extension(MLIRPythonExtension.AsyncDialectPasses
     LLVMSupport
   EMBED_CAPI_LINK_LIBS
     MLIRCAPIAsync
-  GENERATE_TYPE_STUBS
-    "_mlirAsyncPasses.pyi"
 )
 
 if(MLIR_ENABLE_EXECUTION_ENGINE)
@@ -690,8 +670,6 @@ if(MLIR_ENABLE_EXECUTION_ENGINE)
       LLVMSupport
     EMBED_CAPI_LINK_LIBS
       MLIRCAPIExecutionEngine
-    GENERATE_TYPE_STUBS
-      "_mlirExecutionEngine.pyi"
   )
 endif()
 
@@ -706,8 +684,6 @@ declare_mlir_python_extension(MLIRPythonExtension.GPUDialectPasses
     LLVMSupport
   EMBED_CAPI_LINK_LIBS
     MLIRCAPIGPU
-  GENERATE_TYPE_STUBS
-    "_mlirGPUPasses.pyi"
 )
 
 declare_mlir_python_extension(MLIRPythonExtension.LinalgPasses
@@ -721,8 +697,6 @@ declare_mlir_python_extension(MLIRPythonExtension.LinalgPasses
     LLVMSupport
   EMBED_CAPI_LINK_LIBS
     MLIRCAPILinalg
-  GENERATE_TYPE_STUBS
-    "_mlirLinalgPasses.pyi"
 )
 
 declare_mlir_python_extension(MLIRPythonExtension.Dialects.SMT.Pybind
@@ -740,8 +714,6 @@ declare_mlir_python_extension(MLIRPythonExtension.Dialects.SMT.Pybind
     MLIRCAPIIR
     MLIRCAPISMT
     MLIRCAPIExportSMTLIB
-  GENERATE_TYPE_STUBS
-    "_mlirDialectsSMT.pyi"
 )
 
 declare_mlir_python_extension(MLIRPythonExtension.SparseTensorDialectPasses
@@ -755,8 +727,6 @@ declare_mlir_python_extension(MLIRPythonExtension.SparseTensorDialectPasses
     LLVMSupport
   EMBED_CAPI_LINK_LIBS
     MLIRCAPISparseTensor
-  GENERATE_TYPE_STUBS
-    "_mlirSparseTensorPasses.pyi"
 )
 
 declare_mlir_python_extension(MLIRPythonExtension.TransformInterpreter
@@ -770,8 +740,6 @@ declare_mlir_python_extension(MLIRPythonExtension.TransformInterpreter
     LLVMSupport
   EMBED_CAPI_LINK_LIBS
     MLIRCAPITransformDialectTransforms
-  GENERATE_TYPE_STUBS
-    "_mlirTransformInterpreter.pyi"
 )
 
 # TODO: Figure out how to put this in the test tree.
@@ -830,8 +798,6 @@ if(MLIR_INCLUDE_TESTS)
       LLVMSupport
     EMBED_CAPI_LINK_LIBS
       MLIRCAPIPythonTestDialect
-    GENERATE_TYPE_STUBS
-      "_mlirPythonTestNanobind.pyi"
   )
 endif()
 
@@ -851,7 +817,7 @@ endif()
 add_mlir_python_common_capi_library(MLIRPythonCAPI
   INSTALL_COMPONENT MLIRPythonModules
   INSTALL_DESTINATION "${MLIR_BINDINGS_PYTHON_INSTALL_PREFIX}/_mlir_libs"
-  OUTPUT_DIRECTORY "${MLIR_BINARY_DIR}/${MLIR_BINDINGS_PYTHON_INSTALL_PREFIX}/_mlir_libs"
+  OUTPUT_DIRECTORY "${MLIR_BINARY_DIR}/python_packages/mlir_core/mlir/_mlir_libs"
   RELATIVE_INSTALL_ROOT "../../../.."
   DECLARED_HEADERS
     MLIRPythonCAPI.HeaderSources
@@ -880,7 +846,7 @@ endif()
 ################################################################################
 
 add_mlir_python_modules(MLIRPythonModules
-  ROOT_PREFIX "${MLIR_BINARY_DIR}/${MLIR_BINDINGS_PYTHON_INSTALL_PREFIX}"
+  ROOT_PREFIX "${MLIR_BINARY_DIR}/python_packages/mlir_core/mlir"
   INSTALL_PREFIX "${MLIR_BINDINGS_PYTHON_INSTALL_PREFIX}"
   DECLARED_SOURCES
     MLIRPythonSources
diff --git a/mlir/python/mlir/_mlir_libs/.gitignore b/mlir/python/mlir/_mlir_libs/.gitignore
deleted file mode 100644
index 8f0c82ab0aac9..0000000000000
--- a/mlir/python/mlir/_mlir_libs/.gitignore
+++ /dev/null
@@ -1,2 +0,0 @@
-_mlir/**/*.pyi
-*.pyi
diff --git a/mlir/python/mlir/_mlir_libs/_mlir/__init__.pyi b/mlir/python/mlir/_mlir_libs/_mlir/__init__.pyi
new file mode 100644
index 0000000000000..03449b70b7fa3
--- /dev/null
+++ b/mlir/python/mlir/_mlir_libs/_mlir/__init__.pyi
@@ -0,0 +1,12 @@
+
+globals: "_Globals"
+
+class _Globals:
+    dialect_search_modules: list[str]
+    def _register_dialect_impl(self, dialect_namespace: str, dialect_class: type) -> None: ...
+    def _register_operation_impl(self, operation_name: str, operation_class: type) -> None: ...
+    def append_dialect_search_prefix(self, module_name: str) -> None: ...
+    def _check_dialect_module_loaded(self, dialect_namespace: str) -> bool: ...
+
+def register_dialect(dialect_class: type) -> type: ...
+def register_operation(dialect_class: type, *, replace: bool = ...) -> type: ...
diff --git a/mlir/python/mlir/_mlir_libs/_mlir/dialects/pdl.pyi b/mlir/python/mlir/_mlir_libs/_mlir/dialects/pdl.pyi
new file mode 100644
index 0000000000000..d12c6839deaba
--- /dev/null
+++ b/mlir/python/mlir/_mlir_libs/_mlir/dialects/pdl.pyi
@@ -0,0 +1,63 @@
+#  Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+#  See https://llvm.org/LICENSE.txt for license information.
+#  SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+
+from mlir.ir import Type, Context
+
+__all__ = [
+    'PDLType',
+    'AttributeType',
+    'OperationType',
+    'RangeType',
+    'TypeType',
+    'ValueType',
+]
+
+
+class PDLType(Type):
+  @staticmethod
+  def isinstance(type: Type) -> bool: ...
+
+
+class AttributeType(Type):
+  @staticmethod
+  def isinstance(type: Type) -> bool: ...
+
+  @staticmethod
+  def get(context: Context | None = None) -> AttributeType: ...
+
+
+class OperationType(Type):
+  @staticmethod
+  def isinstance(type: Type) -> bool: ...
+
+  @staticmethod
+  def get(context: Context | None = None) -> OperationType: ...
+
+
+class RangeType(Type):
+  @staticmethod
+  def isinstance(type: Type) -> bool: ...
+
+  @staticmethod
+  def get(element_type: Type) -> RangeType: ...
+
+  @property
+  def element_type(self) -> Type: ...
+
+
+class TypeType(Type):
+  @staticmethod
+  def isinstance(type: Type) -> bool: ...
+
+  @staticmethod
+  def get(context: Context | None = None) -> TypeType: ...
+
+
+class ValueType(Type):
+  @staticmethod
+  def isinstance(type: Type) -> bool: ...
+
+  @staticmethod
+  def get(context: Context | None = None) -> ValueType: ...
diff --git a/mlir/python/mlir/_mlir_libs/_mlir/dialects/quant.pyi b/mlir/python/mlir/_mlir_libs/_mlir/dialects/quant.pyi
new file mode 100644
index 0000000000000..3f5304584edef
--- /dev/null
+++ b/mlir/python/mlir/_mlir_libs/_mlir/dialects/quant.pyi
@@ -0,0 +1,1...
[truncated]

joker-eph

Makes sense to me. While the overall feature is likely desirable, this seems a bit unstable in-tree and probably deserves a bit more testing.

jpienaar

Builds broken, so good to rollback and fix off critical path.

…lvm#157173)" This reverts commit 46d8fdd. The whole set of commits got reverted in llvm#157831, reverting this one too.

…7173)" (#157995) This reverts commit 46d8fdd. The whole set of commits got reverted in #157831, reverting this one too.

…ration (#157173)" (#157995) This reverts commit 46d8fdd. The whole set of commits got reverted in llvm/llvm-project#157831, reverting this one too.

This a reland of #155741 which was reverted at #157831. This version is narrower in scope - it only turns on automatic stub generation for `MLIRPythonExtension.Core._mlir` and **does not do anything automatically**. Specifically, the only CMake code added to `AddMLIRPython.cmake` is the `mlir_generate_type_stubs` function which is then used only in a manual way. The API for `mlir_generate_type_stubs` is: ``` Arguments: MODULE_NAME: The fully-qualified name of the extension module (used for importing in python). DEPENDS_TARGETS: List of targets these type stubs depend on being built; usually corresponding to the specific extension module (e.g., something like StandalonePythonModules.extension._standaloneDialectsNanobind.dso) and the core bindings extension module (e.g., something like StandalonePythonModules.extension._mlir.dso). OUTPUT_DIR: The root output directory to emit the type stubs into. OUTPUTS: List of expected outputs. DEPENDS_TARGET_SRC_DEPS: List of cpp sources for extension library (for generating a DEPFILE). IMPORT_PATHS: List of paths to add to PYTHONPATH for stubgen. PATTERN_FILE: (Optional) Pattern file (see https://nanobind.readthedocs.io/en/latest/typing.html#pattern-files). Outputs: NB_STUBGEN_CUSTOM_TARGET: The target corresponding to generation which other targets can depend on. ``` Downstream users should use `mlir_generate_type_stubs` in coordination with `declare_mlir_python_sources` to turn on stub generation for their own downstream dialect extensions and upstream dialect extensions if they so choose. Standalone example shows an example. Note, downstream will also need to set `-DMLIR_PYTHON_PACKAGE_PREFIX=...` correctly for their bindings.

…7930) This a reland of llvm/llvm-project#155741 which was reverted at llvm/llvm-project#157831. This version is narrower in scope - it only turns on automatic stub generation for `MLIRPythonExtension.Core._mlir` and **does not do anything automatically**. Specifically, the only CMake code added to `AddMLIRPython.cmake` is the `mlir_generate_type_stubs` function which is then used only in a manual way. The API for `mlir_generate_type_stubs` is: ``` Arguments: MODULE_NAME: The fully-qualified name of the extension module (used for importing in python). DEPENDS_TARGETS: List of targets these type stubs depend on being built; usually corresponding to the specific extension module (e.g., something like StandalonePythonModules.extension._standaloneDialectsNanobind.dso) and the core bindings extension module (e.g., something like StandalonePythonModules.extension._mlir.dso). OUTPUT_DIR: The root output directory to emit the type stubs into. OUTPUTS: List of expected outputs. DEPENDS_TARGET_SRC_DEPS: List of cpp sources for extension library (for generating a DEPFILE). IMPORT_PATHS: List of paths to add to PYTHONPATH for stubgen. PATTERN_FILE: (Optional) Pattern file (see https://nanobind.readthedocs.io/en/latest/typing.html#pattern-files). Outputs: NB_STUBGEN_CUSTOM_TARGET: The target corresponding to generation which other targets can depend on. ``` Downstream users should use `mlir_generate_type_stubs` in coordination with `declare_mlir_python_sources` to turn on stub generation for their own downstream dialect extensions and upstream dialect extensions if they so choose. Standalone example shows an example. Note, downstream will also need to set `-DMLIR_PYTHON_PACKAGE_PREFIX=...` correctly for their bindings.

This a reland of llvm/llvm-project#155741 which was reverted at llvm/llvm-project#157831. This version is narrower in scope - it only turns on automatic stub generation for `MLIRPythonExtension.Core._mlir` and **does not do anything automatically**. Specifically, the only CMake code added to `AddMLIRPython.cmake` is the `mlir_generate_type_stubs` function which is then used only in a manual way. The API for `mlir_generate_type_stubs` is: ``` Arguments: MODULE_NAME: The fully-qualified name of the extension module (used for importing in python). DEPENDS_TARGETS: List of targets these type stubs depend on being built; usually corresponding to the specific extension module (e.g., something like StandalonePythonModules.extension._standaloneDialectsNanobind.dso) and the core bindings extension module (e.g., something like StandalonePythonModules.extension._mlir.dso). OUTPUT_DIR: The root output directory to emit the type stubs into. OUTPUTS: List of expected outputs. DEPENDS_TARGET_SRC_DEPS: List of cpp sources for extension library (for generating a DEPFILE). IMPORT_PATHS: List of paths to add to PYTHONPATH for stubgen. PATTERN_FILE: (Optional) Pattern file (see https://nanobind.readthedocs.io/en/latest/typing.html#pattern-files). Outputs: NB_STUBGEN_CUSTOM_TARGET: The target corresponding to generation which other targets can depend on. ``` Downstream users should use `mlir_generate_type_stubs` in coordination with `declare_mlir_python_sources` to turn on stub generation for their own downstream dialect extensions and upstream dialect extensions if they so choose. Standalone example shows an example. Note, downstream will also need to set `-DMLIR_PYTHON_PACKAGE_PREFIX=...` correctly for their bindings.

ftynse added 3 commits September 10, 2025 10:51

Revert "[MLIR][Python] move generated type stubs to build directory (#…

4e6f5e6

…157697)" This reverts commit 53bcfe2.

Revert "[MLIR][Python] fix stubgen (#157583)"

6b6fab0

This reverts commit d02c7ae.

Revert "[MLIR][Python] enable type stub auto-generation (#155741)"

91165fa

This reverts commit 22c2e15.

ftynse requested review from makslevental, rolfmorel and stellaraccident as code owners September 10, 2025 10:57

llvmbot added mlir:python MLIR Python bindings mlir labels Sep 10, 2025

ftynse requested review from joker-eph and jpienaar September 10, 2025 10:57

joker-eph approved these changes Sep 10, 2025

View reviewed changes

jpienaar approved these changes Sep 10, 2025

View reviewed changes

ftynse merged commit 0a83e96 into main Sep 10, 2025
13 checks passed

ftynse deleted the users/ftynse/revert-stubgen branch September 10, 2025 11:35

makslevental mentioned this pull request Sep 10, 2025

[MLIR][Python] reland (narrower) type stub generation #157930

Merged

Ahajha mentioned this pull request Sep 11, 2025

Revert "[bazel][mlir][python] Port #155741: stub auto-generation (#157173)" #157995

Merged

rupprecht pushed a commit that referenced this pull request Sep 12, 2025

Revert "[bazel][mlir][python] Port #155741: stub auto-generation (#15…

ed1f1b8

…7173)" (#157995) This reverts commit 46d8fdd. The whole set of commits got reverted in #157831, reverting this one too.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Revert stubgen-related patches #157831

Revert stubgen-related patches #157831

Uh oh!

ftynse commented Sep 10, 2025

Uh oh!

llvmbot commented Sep 10, 2025

Uh oh!

joker-eph left a comment

Uh oh!

jpienaar left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Revert stubgen-related patches #157831

Revert stubgen-related patches #157831

Uh oh!

Conversation

ftynse commented Sep 10, 2025

Uh oh!

llvmbot commented Sep 10, 2025

Uh oh!

joker-eph left a comment

Choose a reason for hiding this comment

Uh oh!

jpienaar left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants