Skip to content

hstore_to_array() does not escape single quotes in value #57

@drdee

Description

@drdee

We have a column with an hstore datatype in a table that we are replicating using logical replication.

The select command that is being issued is:

SELECT hstore_to_array('"nickname"=>"Dave's Courtyard"')

the single quote should be escaped because this throws the following error:

2019-07-03 13:57:31,042Z    tap - Traceback (most recent call last):
2019-07-03 13:57:31,042Z    tap -   File "tap-env/bin/tap-postgres", line 10, in <module>
2019-07-03 13:57:31,043Z    tap -     sys.exit(main())
2019-07-03 13:57:31,043Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/__init__.py", line 705, in main
2019-07-03 13:57:31,043Z    tap -     raise exc
2019-07-03 13:57:31,043Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/__init__.py", line 702, in main
2019-07-03 13:57:31,043Z    tap -     main_impl()
2019-07-03 13:57:31,043Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/__init__.py", line 696, in main_impl
2019-07-03 13:57:31,043Z    tap -     do_sync(conn_config, args.catalog.to_dict() if args.catalog else args.properties, args.config.get('default_replication_method'), state)
2019-07-03 13:57:31,043Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/__init__.py", line 671, in do_sync
2019-07-03 13:57:31,043Z    tap -     state = sync_logical_streams(conn_config, list(streams), state, end_lsn)
2019-07-03 13:57:31,043Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/__init__.py", line 581, in sync_logical_streams
2019-07-03 13:57:31,043Z    tap -     state = logical_replication.sync_tables(conn_config, logical_streams, state, end_lsn)
2019-07-03 13:57:31,043Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/sync_strategies/logical_replication.py", line 355, in sync_tables
2019-07-03 13:57:31,044Z    tap -     state = consume_message(logical_streams, state, msg, time_extracted, conn_info, end_lsn)
2019-07-03 13:57:31,044Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/sync_strategies/logical_replication.py", line 263, in consume_message
2019-07-03 13:57:31,044Z    tap -     record_message = row_to_singer_message(target_stream, col_vals, stream_version, col_names, time_extracted, stream_md_map, conn_info)
2019-07-03 13:57:31,044Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/sync_strategies/logical_replication.py", line 203, in row_to_singer_message
2019-07-03 13:57:31,044Z    tap -     cleaned_elem = selected_value_to_singer_value(elem, sql_datatype, conn_info)
2019-07-03 13:57:31,044Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/sync_strategies/logical_replication.py", line 189, in selected_value_to_singer_value
2019-07-03 13:57:31,044Z    tap -     return selected_value_to_singer_value_impl(elem, sql_datatype, conn_info)
2019-07-03 13:57:31,044Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/sync_strategies/logical_replication.py", line 165, in selected_value_to_singer_value_impl
2019-07-03 13:57:31,044Z    tap -     return create_hstore_elem(conn_info, elem)
2019-07-03 13:57:31,044Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/sync_strategies/logical_replication.py", line 75, in create_hstore_elem
2019-07-03 13:57:31,044Z    tap -     cur.execute(sql)
2019-07-03 13:57:31,044Z    tap - psycopg2.ProgrammingError: syntax error at or near "s"
2019-07-03 13:57:31,044Z    tap - LINE 1: SELECT hstore_to_array('"nickname"=>"Dave's Courtyard"')
2019-07-03 13:57:31,044Z    tap -                                                   ^

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions