Skip to content

Conversation

@CTTY
Copy link
Contributor

@CTTY CTTY commented Jul 28, 2025

Which issue does this PR close?

What changes are included in this PR?

  • Use ParquetWriter's inner AsyncArrowWriter's bytes_written() and in_progress_size to track current written size

Are these changes tested?

No new tests added

@CTTY
Copy link
Contributor Author

CTTY commented Jul 28, 2025

Seems like the docker setup in CI is failing

Copy link
Contributor

@liurenjie1024 liurenjie1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @CTTY for this fix!

@liurenjie1024 liurenjie1024 merged commit 6846be4 into apache:main Jul 29, 2025
17 checks passed
@CTTY CTTY deleted the ctty/parquet-size branch July 29, 2025 02:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ParquetWriter should account for the buffer of the inner writer when checking current_written_size

2 participants