Skip to content
This repository was archived by the owner on May 17, 2024. It is now read-only.

fix crashing when parsing column names including _a, _b #636

Merged
merged 2 commits into from
Jul 6, 2023

Conversation

vvkh
Copy link
Contributor

@vvkh vvkh commented Jul 6, 2023

Fix exceptions when using --json for datasets including columns with _a or _b in the name

traceback
         ERROR    Field dataset1 requires a value                                                                                                         __main__.py:326
Traceback (most recent call last):
  File "/Users/dan/repos/demo/env/bin/data-diff", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/dan/repos/demo/env/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dan/repos/demo/env/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/dan/repos/demo/env/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dan/repos/demo/env/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dan/repos/demo/env/lib/python3.11/site-packages/data_diff/__main__.py", line 311, in main
    dbt_diff(
  File "/Users/dan/repos/demo/env/lib/python3.11/site-packages/data_diff/dbt.py", line 132, in dbt_diff
    _local_diff(diff_vars, json_output)
  File "/Users/dan/repos/demo/env/lib/python3.11/site-packages/data_diff/dbt.py", line 313, in _local_diff
    jsonify(
  File "/Users/dan/repos/demo/env/lib/python3.11/site-packages/data_diff/format.py", line 42, in jsonify
    diff_rows_jsonified.append(_jsonify_diff(row, key_columns))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dan/repos/demo/env/lib/python3.11/site-packages/data_diff/format.py", line 229, in _jsonify_diff
    return {column: JsonDiffRowValue(**data) for column, data in columns.items()}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dan/repos/demo/env/lib/python3.11/site-packages/data_diff/format.py", line 229, in <dictcomp>
    return {column: JsonDiffRowValue(**data) for column, data in columns.items()}
                    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 7, in __init__
  File "/Users/dan/repos/demo/env/lib/python3.11/site-packages/runtype/dataclass.py", line 227, in __post_init__
    _post_init(self, config=config, should_cast=check_types == 'cast', sampler=sampler)
  File "/Users/dan/repos/demo/env/lib/python3.11/site-packages/runtype/dataclass.py", line 104, in _post_init
    raise TypeError(f"Field {name} requires a value")
TypeError: Field dataset1 requires a value

Note: currently there are handy removesuffix and removeprefix methods, but they were added in Python 3.9, so we can't use them.

@vvkh vvkh requested a review from dlawin July 6, 2023 16:26
Copy link
Contributor

@dlawin dlawin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested and it resolved the issue

@dlawin dlawin merged commit 1094999 into master Jul 6, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants