api: switch from JSON data type to ARROW IPC

[rampage644]

Branch

Snowflake converter for FIXED datatype via arrow ipc natively supports: int8, int16, int32, int64, DECIMAL128 and no uint types at all:
https://github.com/snowflakedb/snowflake-connector-python/blob/main/src/snowflake/connector/nanoarrow_cpp/ArrowIterator/CArrowChunkIterator.cpp#L173

[YaroslavLitvinov]

It's possible that resolving this issue will fix another issue #67.

[YaroslavLitvinov]

Enabling arrow format causes errors on connector's side.
Failed to convert current row, cause: [Snowflake Exception] unknown arrow internal data type(260913075) for FIXED data

[YaroslavLitvinov]

Following data snippet (in data variable) causes above error ^^ in snowflake connector, when
being returned as a part of response in rowSetBase64 field.
Also the same data snipped parsed without errors in a code below, by pyarrow library version 19.0, which is the same version is used in snowflake connector in my environment.

Parsed result :

pyarrow.Table
count: uint64 not null
----
count: [[0]]

Code:

import base64
import pyarrow
from pyarrow import RecordBatchStreamReader  # NOQA
from pyarrow import RecordBatch, RecordBatchStreamWriter

def read_arrrow_base_64(data):
    decoded_data = base64.b64decode(data)
    reader = RecordBatchStreamReader(decoded_data)
    return reader.read_all()

data = "/////ygBAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAYAAAAAAASABoAFAAAABMACAAAAAwABAASAAAANAAAABgAAAAcAAAAAAAAAhgAAAAAAAYACAAEAAYAAABAAAAAAAAAAAUAAABjb3VudAAAAAQAAACAAAAAUAAAACwAAAAEAAAAmP///wgAAAAMAAAAAQAAADAAAAAKAAAAY2hhckxlbmd0aAAAvP///wgAAAAMAAAAAQAAADAAAAAFAAAAc2NhbGUAAADc////CAAAAAwAAAABAAAANgAAAAsAAABsb2dpY2FsVHlwZQAIAAwACAAEAAgAAAAIAAAADAAAAAIAAAAzOAAACQAAAHByZWNpc2lvbgAAAP////+IAAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAEAAAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAAAsAAAAEAAAAAEAAAAAAAAAAAAAAAEAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAIAAAAAAAAAAAAAAAEAAAAAAAAACAAAAAAAAAAIAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAD/////AAAAAA=="

print(read_arrrow_base_64(data))

[YaroslavLitvinov]

Embucket supports different set of datatypes than Snowflake do

Snowflake’s support of FIXED types is pretty limited, no unsigned types supported at all, like Uint64. Respectively Snowflake’s connector expects following signed types: int8, int16, int32, int64, decimal128.
In count batch response replacing UInt64 by Int64 fixes issue.

[YaroslavLitvinov]

There is an issue showing how crucial could be having a right alignment:
apache/arrow-rs#5052