-
Notifications
You must be signed in to change notification settings - Fork 117
Introduce models for SeaDatabricksClient
#595
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
50 commits
Select commit
Hold shift + click to select a range
138c2ae
[squash from exec-sea] bring over execution phase changes
varun-edachali-dbx 3e3ab94
remove excess test
varun-edachali-dbx 4a78165
add docstring
varun-edachali-dbx 0dac4aa
remvoe exec func in sea backend
varun-edachali-dbx 1b794c7
remove excess files
varun-edachali-dbx da5a6fe
remove excess models
varun-edachali-dbx 686ade4
remove excess sea backend tests
varun-edachali-dbx 31e6c83
cleanup
varun-edachali-dbx 69ea238
re-introduce get_schema_desc
varun-edachali-dbx 66d7517
remove SeaResultSet
varun-edachali-dbx 71feef9
clean imports and attributes
varun-edachali-dbx ae9862f
pass CommandId to ExecResp
varun-edachali-dbx d8aa69e
remove changes in types
varun-edachali-dbx db139bc
add back essential types (ExecResponse, from_sea_state)
varun-edachali-dbx b977b12
fix fetch types
varun-edachali-dbx da615c0
excess imports
varun-edachali-dbx 0da04a6
reduce diff by maintaining logs
varun-edachali-dbx ea9d456
fix int test types
varun-edachali-dbx 8985c62
[squashed from exec-sea] init execution func
varun-edachali-dbx d9bcdbe
remove irrelevant changes
varun-edachali-dbx ee9fa1c
remove ResultSetFilter functionality
varun-edachali-dbx 24c6152
remove more irrelevant changes
varun-edachali-dbx 67fd101
remove more irrelevant changes
varun-edachali-dbx 271fcaf
even more irrelevant changes
varun-edachali-dbx bf26ea3
remove sea response as init option
varun-edachali-dbx ed7cf91
exec test example scripts
varun-edachali-dbx dae15e3
formatting (black)
varun-edachali-dbx db5bbea
[squashed from sea-exec] merge sea stuffs
varun-edachali-dbx d5d3699
remove excess changes
varun-edachali-dbx 6137a3d
remove excess removed docstring
varun-edachali-dbx 75b0773
remove excess changes in backend
varun-edachali-dbx 4494dcd
remove excess imports
varun-edachali-dbx 4d0aeca
remove accidentally removed _get_schema_desc
varun-edachali-dbx 7cece5e
remove unnecessary init with sea_response tests
varun-edachali-dbx 8977c06
rmeove unnecessary changes
varun-edachali-dbx 0216d7a
formatting (black)
varun-edachali-dbx 4cb15fd
improved models and filters from cloudfetch-sea branch
varun-edachali-dbx dee47f7
filters stuff (align with JDBC)
varun-edachali-dbx e385d5b
backend from cloudfetch-sea
varun-edachali-dbx 484064e
remove filtering, metadata ops
varun-edachali-dbx 030edf8
raise NotImplementedErrror for metadata ops
varun-edachali-dbx 8bd12d8
Merge branch 'sea-migration' into exec-models-sea
varun-edachali-dbx ffded6e
remove un-necessary changes
varun-edachali-dbx 227f6b3
remove un-necessary backend cahnges
varun-edachali-dbx 3940eec
remove un-needed GetChunksResponse
varun-edachali-dbx 267c9f4
reduce code duplication
varun-edachali-dbx 2967119
more clear docstrings
varun-edachali-dbx 47fd60d
introduce strongly typed ChunkInfo
varun-edachali-dbx 982fdf2
remove is_volume_operation from response
varun-edachali-dbx 9e14d48
add is_volume_op and more ResultData fields
varun-edachali-dbx File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,95 @@ | ||
""" | ||
Base models for the SEA (Statement Execution API) backend. | ||
|
||
These models define the common structures used in SEA API requests and responses. | ||
""" | ||
|
||
from typing import Dict, List, Any, Optional, Union | ||
from dataclasses import dataclass, field | ||
|
||
from databricks.sql.backend.types import CommandState | ||
|
||
|
||
@dataclass | ||
class ServiceError: | ||
"""Error information returned by the SEA API.""" | ||
|
||
message: str | ||
error_code: Optional[str] = None | ||
|
||
|
||
@dataclass | ||
class StatementStatus: | ||
"""Status information for a statement execution.""" | ||
|
||
state: CommandState | ||
error: Optional[ServiceError] = None | ||
sql_state: Optional[str] = None | ||
|
||
|
||
@dataclass | ||
class ExternalLink: | ||
"""External link information for result data.""" | ||
|
||
external_link: str | ||
expiration: str | ||
chunk_index: int | ||
byte_count: int = 0 | ||
row_count: int = 0 | ||
row_offset: int = 0 | ||
next_chunk_index: Optional[int] = None | ||
next_chunk_internal_link: Optional[str] = None | ||
http_headers: Optional[Dict[str, str]] = None | ||
|
||
|
||
@dataclass | ||
class ChunkInfo: | ||
"""Information about a chunk in the result set.""" | ||
|
||
chunk_index: int | ||
byte_count: int | ||
row_offset: int | ||
row_count: int | ||
|
||
|
||
@dataclass | ||
class ResultData: | ||
"""Result data from a statement execution.""" | ||
|
||
data: Optional[List[List[Any]]] = None | ||
external_links: Optional[List[ExternalLink]] = None | ||
byte_count: Optional[int] = None | ||
chunk_index: Optional[int] = None | ||
next_chunk_index: Optional[int] = None | ||
next_chunk_internal_link: Optional[str] = None | ||
row_count: Optional[int] = None | ||
row_offset: Optional[int] = None | ||
attachment: Optional[bytes] = None | ||
|
||
|
||
@dataclass | ||
class ColumnInfo: | ||
"""Information about a column in the result set.""" | ||
|
||
name: str | ||
type_name: str | ||
type_text: str | ||
nullable: bool = True | ||
precision: Optional[int] = None | ||
scale: Optional[int] = None | ||
ordinal_position: Optional[int] = None | ||
|
||
|
||
@dataclass | ||
class ResultManifest: | ||
"""Manifest information for a result set.""" | ||
|
||
format: str | ||
schema: Dict[str, Any] # Will contain column information | ||
total_row_count: int | ||
total_byte_count: int | ||
total_chunk_count: int | ||
truncated: bool = False | ||
chunks: Optional[List[ChunkInfo]] = None | ||
result_compression: Optional[str] = None | ||
is_volume_operation: Optional[bool] = None |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.