Skip to content

Commit 4580571

Browse files
DOCSP-10375 doc for convert expressions in filename (#28)
* DOCSP-10375 doc for convert expressions in filename
1 parent 4d57cec commit 4580571

File tree

1 file changed

+29
-4
lines changed

1 file changed

+29
-4
lines changed

source/reference/pipeline/out.txt

Lines changed: 29 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -267,10 +267,35 @@ is a combination of the following:
267267
Limitations
268268
-----------
269269

270-
{+dl+} interprets null values in partitions as empty strings (``""``). If you
271-
want {+dl+} to handle null values in filenames, you must wrap the null values
272-
using :manual:`$convert </reference/operator/aggregation/convert>` with an
273-
empty string ``onNull`` value.
270+
{+dl+} interprets empty strings (``""``) as null values when parsing
271+
filenames. If you want {+dl+} to generate filenames that it can parse
272+
correctly, you must wrap the field references that could have null values
273+
using :manual:`$convert </reference/operator/aggregation/convert>` with
274+
an empty string ``onNull`` value. The following example shows how to handle
275+
null values in the ``year`` field when creating a filename from the field
276+
value.
277+
278+
.. code-block:: json
279+
:copyable: false
280+
281+
{
282+
$out: {
283+
s3: {
284+
bucket: "my-s3-bucket",
285+
region: "us-east-1",
286+
filename: {
287+
$concat: [
288+
"big-box-store/",
289+
{$convert: {input: "year", to: "string", onNull: ""}}
290+
]
291+
},
292+
format: {
293+
name: "json.gz",
294+
maxFileSize: "200MiB"
295+
}
296+
}
297+
}
298+
}
274299

275300
.. _adl-out-stage-errors:
276301

0 commit comments

Comments
 (0)