Add py::module_local documentation

Author: jagerman

docs/advanced/cast/stl.rst

@@ -167,6 +167,20 @@ Python, and to define a set of available operations, e.g.:
         }, py::keep_alive<0, 1>()) /* Keep vector alive while iterator is used */
         // ....
 
+Please take a look at the :ref:`macro_notes` before using the
+``PYBIND11_MAKE_OPAQUE`` macro.
+
+.. seealso::
+
+    The file :file:`tests/test_opaque_types.cpp` contains a complete
+    example that demonstrates how to create and expose opaque types using
+    pybind11 in more detail.
+
+.. _stl_bind:
+
+Binding STL containers
+======================
+
 The ability to expose STL containers as native Python objects is a fairly
 common request, hence pybind11 also provides an optional header file named
 :file:`pybind11/stl_bind.h` that does exactly this. The mapped containers try
@@ -188,14 +202,34 @@ The following example showcases usage of :file:`pybind11/stl_bind.h`:
     py::bind_vector<std::vector<int>>(m, "VectorInt");
     py::bind_map<std::map<std::string, double>>(m, "MapStringDouble");
 
-Please take a look at the :ref:`macro_notes` before using the
-``PYBIND11_MAKE_OPAQUE`` macro.
+When binding STL containers pybind11 considers the types of the container's
+elements to decide whether the container should be confined to the local module
+(via the :ref:`module_local` feature).  If the container element types are
+anything other than already-bound custom types bound without
+``py::module_local()`` the container binding will have ``py::module_local()``
+applied.  This includes converting types such as numeric types, strings, Eigen
+types; and types that have not yet been bound at the time of the stl container
+binding.  This module-local binding is designed to avoid potential conflicts
+between module bindings (for example, from two separate modules each attempting
+to bind ``std::vector<int>`` as a python type).
+
+It is possible to override this behavior to force a definition to be either
+module-local or global.  To do so, you can pass the attributes
+``py::module_local()`` (to make the binding module-local) or
+``py::module_local(false)`` (to make the binding global) into the
+``py::bind_vector`` or ``py::bind_map`` arguments:
 
-.. seealso::
+.. code-block:: cpp
 
-    The file :file:`tests/test_opaque_types.cpp` contains a complete
-    example that demonstrates how to create and expose opaque types using
-    pybind11 in more detail.
+    py::bind_vector<std::vector<int>>(m, "VectorInt", py::module_local(false));
+
+Note, however, that such a global binding would make it impossible to load this
+module at the same time as any other pybind module that also attempts to bind
+the same container type (``std::vector<int>`` in the above example).
+
+See :ref:`module_local` for more details on module-local bindings.
+
+.. seealso::
 
     The file :file:`tests/test_stl_binders.cpp` shows how to use the
     convenience STL container wrappers.

docs/advanced/classes.rst

@@ -635,3 +635,139 @@ inheritance, which can lead to undefined behavior. In such cases, add the tag
 
 The tag is redundant and does not need to be specified when multiple base types
 are listed.
+
+.. _module_local:
+
+Module-local class bindings
+===========================
+
+When creating a binding for a class, pybind by default makes that binding
+"global" across modules.  What this means is that a type defined in one module
+can be passed to functions of other modules that expect the same C++ type.  For
+example, this allows the following:
+
+.. code-block:: cpp
+
+    // In the module1.cpp binding code for module1:
+    py::class_<Pet>(m, "Pet")
+        .def(py::init<std::string>());
+
+.. code-block:: cpp
+
+    // In the module2.cpp binding code for module2:
+    m.def("pet_name", [](Pet &p) { return p.name(); });
+
+.. code-block:: pycon
+
+    >>> from module1 import Pet
+    >>> from module2 import pet_name
+    >>> mypet = Pet("Kitty")
+    >>> pet_name(mypet)
+    'Kitty'
+
+When writing binding code for a library, this is usually desirable: this
+allows, for example, splitting up a complex library into multiple Python
+modules.
+
+In some cases, however, this can cause conflicts.  For example, suppose two
+unrelated modules make use of an external C++ library and each provide custom
+bindings for one of that library's classes.  This will result in an error when
+a Python program attempts to import both modules (directly or indirectly)
+because of conflicting definitions on the external type:
+
+.. code-block:: cpp
+
+    // dogs.cpp
+
+    // Binding for external library class:
+    py::class<pets::Pet>(m, "Pet")
+        .def("name", &pets::Pet::name);
+
+    // Binding for local extension class:
+    py::class<Dog, pets::Pet>(m, "Dog")
+        .def(py::init<std::string>());
+
+.. code-block:: cpp
+
+    // cats.cpp, in a completely separate project from the above dogs.cpp.
+
+    // Binding for external library class:
+    py::class<pets::Pet>(m, "Pet")
+        .def("get_name", &pets::Pet::name);
+
+    // Binding for local extending class:
+    py::class<Cat, pets::Pet>(m, "Cat")
+        .def(py::init<std::string>());
+
+.. code-block:: pycon
+
+    >>> import cats
+    >>> import dogs
+    Traceback (most recent call last):
+      File "<stdin>", line 1, in <module>
+    ImportError: generic_type: type "Pet" is already registered!
+
+To get around this, you can tell pybind11 to keep the external class binding
+localized to the module by passing the ``py::module_local()`` attribute into
+the ``py::class_`` constructor:
+
+.. code-block:: cpp
+
+    // Pet binding in dogs.cpp:
+    py::class<pets::Pet>(m, "Pet", py::module_local())
+        .def("name", &pets::Pet::name);
+
+.. code-block:: cpp
+
+    // Pet binding in cats.cpp:
+    py::class<pets::Pet>(m, "Pet", py::module_local())
+        .def("get_name", &pets::Pet::name);
+
+This makes the Python-side ``dogs.Pet`` and ``cats.Pet`` into distinct classes
+that can only be accepted as ``Pet`` arguments within those classes.  This
+avoids the conflict and allows both modules to be loaded.
+
+One limitation of this approach is that because ``py::module_local`` types are
+distinct on the Python side, it is not possible to pass such a module-local
+type as a C++ ``Pet``-taking function outside that module.  For example, if the
+above ``cats`` and ``dogs`` module are each extended with a function:
+
+.. code-block:: cpp
+
+    m.def("petname", [](pets::Pet &p) { return p.name(); });
+
+you will only be able to call the function with the local module's class:
+
+.. code-block:: pycon
+
+    >>> import cats, dogs  # No error because of the added py::module_local()
+    >>> mycat, mydog = cats.Cat("Fluffy"), dogs.Dog("Rover")
+    >>> (cats.petname(mycat), dogs.petname(mydog))
+    ('Fluffy', 'Rover')
+    >>> cats.petname(mydog)
+    Traceback (most recent call last):
+      File "<stdin>", line 1, in <module>
+    TypeError: petname(): incompatible function arguments. The following argument types are supported:
+        1. (arg0: cats.Pet) -> str
+
+    Invoked with: <dogs.Dog object at 0x123>
+
+.. note::
+
+    STL bindings (as provided via the optional :file:`pybind11/stl_bind.h`
+    header) apply ``py::module_local`` by default when the bound type might
+    conflict with other modules; see :ref:`stl_bind` for details.
+
+.. note::
+
+    The localization of the bound types is actually tied to the shared object
+    or binary generated by the compiler/linker.  For typical modules created
+    with ``PYBIND11_MODULE()``, this distinction is not significant.  It is
+    possible, however, when :ref:`embedding` to embed multiple modules in the
+    same binary (see :ref:`embedding_modules`).  In such a case, the
+    localization will apply across all embedded modules within the same binary.
+
+.. seealso::
+
+    The file :file:`tests/test_local_bindings.cpp` contains additional examples
+    that demonstrate how ``py::module_local()`` works.

docs/advanced/embedding.rst

@@ -1,3 +1,5 @@
+.. _embedding:
+
 Embedding the interpreter
 #########################
 
@@ -131,6 +133,7 @@ embedding the interpreter. This makes it easy to import local Python files:
     int n = result.cast<int>();
     assert(n == 3);
 
+.. _embedding_modules:
 
 Adding embedded modules
 =======================