Skip to content

csv processor can't parse empty_value option #55643

@martijnvg

Description

@martijnvg

The following put pipeline call fails:

PUT /_ingest/pipeline/test
{
    "description": "My test pipeline",
    "processors": [
        {
            "csv": {
                "field": "name",
                "target_fields": [
                    "targetField1",
                    "targetField2"
                ],
                "trim": true,
                "empty_value": {
                    "x": "y"
                }
            }
        }
    ]
}

With:

{
    "error": {
        "root_cause": [
            {
                "type": "parse_exception",
                "reason": "processor [csv] doesn't support one or more provided configuration parameters [empty_value]",
                "processor_type": "csv"
            }
        ],
        "type": "parse_exception",
        "reason": "processor [csv] doesn't support one or more provided configuration parameters [empty_value]",
        "processor_type": "csv"
    },
    "status": 400
}

However empty_value is a valid option.

This is caused by a bug in the processor parsing code:
https://github.com/elastic/elasticsearch/blob/master/modules/ingest-common/src/main/java/org/elasticsearch/ingest/common/CsvProcessor.java#L104

It checks for the existence of emptyValue field and then tries to parse it as empty_value field. So even if this option is specified in both ways it can never be parsed.

This bug was found in elastic/elasticsearch-net#4652

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions