File tree Expand file tree Collapse file tree 1 file changed +29
-4
lines changed
source/reference/pipeline Expand file tree Collapse file tree 1 file changed +29
-4
lines changed Original file line number Diff line number Diff line change @@ -267,10 +267,35 @@ is a combination of the following:
267
267
Limitations
268
268
-----------
269
269
270
- {+dl+} interprets null values in partitions as empty strings (``""``). If you
271
- want {+dl+} to handle null values in filenames, you must wrap the null values
272
- using :manual:`$convert </reference/operator/aggregation/convert>` with an
273
- empty string ``onNull`` value.
270
+ {+dl+} interprets empty strings (``""``) as null values when parsing
271
+ filenames. If you want {+dl+} to generate filenames that it can parse
272
+ correctly, you must wrap the field references that could have null values
273
+ using :manual:`$convert </reference/operator/aggregation/convert>` with
274
+ an empty string ``onNull`` value. The following example shows how to handle
275
+ null values in the ``year`` field when creating a filename from the field
276
+ value.
277
+
278
+ .. code-block:: json
279
+ :copyable: false
280
+
281
+ {
282
+ $out: {
283
+ s3: {
284
+ bucket: "my-s3-bucket",
285
+ region: "us-east-1",
286
+ filename: {
287
+ $concat: [
288
+ "big-box-store/",
289
+ {$convert: {input: "year", to: "string", onNull: ""}}
290
+ ]
291
+ },
292
+ format: {
293
+ name: "json.gz",
294
+ maxFileSize: "200MiB"
295
+ }
296
+ }
297
+ }
298
+ }
274
299
275
300
.. _adl-out-stage-errors:
276
301
You can’t perform that action at this time.
0 commit comments