Skip to content

Commit 97d83b5

Browse files
Chris Choschmalliso
authored andcommitted
DOCSP-11073: Add multiple collection example for Source Connector (#36)
add PR template
1 parent 7c4b3e8 commit 97d83b5

File tree

1 file changed

+20
-0
lines changed

1 file changed

+20
-0
lines changed

source/kafka-source.txt

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -143,6 +143,8 @@ an example source connector configuration file, see
143143

144144
.. seealso:: :ref:`Custom pipeline example <custom-pipeline-example>`.
145145

146+
.. seealso:: :ref:`Multiple source example <multi-source-example>`.
147+
146148
| **Default**: []
147149
| **Accepted Values**: A valid JSON array
148150

@@ -261,6 +263,24 @@ to filter or modify the change events output. In this example, we set the
261263
``_id`` field which MongoDB uses as the :manual:`resume token
262264
</changeStreams/#change-stream-resume-token/>`.
263265

266+
.. _multi-source-example:
267+
268+
Multiple Source Example
269+
-----------------------
270+
271+
You can configure the source connector to listen for events from multiple
272+
collections by using the ``pipeline`` configuration with a custom
273+
:manual:`aggregation pipeline </reference/operator/aggregation-pipeline/>`
274+
to match your collection names.
275+
276+
The following sample configuration shows how you can set your source
277+
connector to listen to the ``collection1`` and ``collection2``
278+
collections by matching with a regular expression on the collection names:
279+
280+
.. code-block:: properties
281+
282+
pipeline=[{"$match": {"ns.coll": {"$regex": /^(collection1|collection2)$/}}}]
283+
264284
.. _topic-naming-example:
265285

266286
Topic Naming Example

0 commit comments

Comments
 (0)