fix: UnexpectedEof on truncated input
#412
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added a generic truncated stream test to the test_cases! macro that automatically tests all decoders (bzip2, gzip, deflate, zlib, xz, lzma, lz4, zstd, brotli) for proper handling of incomplete streams. The test compresses data, truncates it, then attempts decompression. Decoders should return UnexpectedEof errors for truncated streams instead of silently accepting incomplete data. This test currently fails for bzip2, lz4, and zstd decoders, which will be fixed in subsequent commits.
baszalmstra
force-pushed
the
claude/implement-issue-411-011CUpzAf5yUB1s58RpG9Fzp
branch
from
4bfae42 to
6c387f9
Compare
NobodyXu
requested changes
Nov 5, 2025
Collaborator
NobodyXu
left a comment
NobodyXu
left a comment
There was a problem hiding this comment.
Thank you!
Just some feedback with the flush impl:
Fixes Nullus157#411 The async BzDecoder was silently accepting truncated bzip2 streams, returning Ok(0) instead of raising an error. This contrasts with the synchronous bzip2::read::BzDecoder which properly returns an UnexpectedEof error. Added state tracking to BzDecoder: - Added stream_ended field to track if Status::StreamEnd was received - Modified decode() to set stream_ended = true on Status::StreamEnd - Updated finish() to check stream_ended and return UnexpectedEof if false This ensures applications cannot accidentally accept corrupted or incomplete compressed data as valid, matching the behavior of the synchronous decoder. The generic truncated test now passes for bzip2.
The LZ4 decoder was silently accepting truncated streams by not validating stream completion in finish(). This issue was discovered by the generic truncated stream test. Added state tracking to Lz4Decoder: - Added stream_ended field to track if remaining == 0 was seen - Modified decode() to set stream_ended = true when stream completes - Updated finish() to check stream_ended and return UnexpectedEof if false This matches the behavior of other decoders (bzip2, gzip, etc.) and ensures applications cannot accidentally accept corrupted or incomplete LZ4 data as valid. The generic truncated test now passes for LZ4.
baszalmstra
force-pushed
the
claude/implement-issue-411-011CUpzAf5yUB1s58RpG9Fzp
branch
from
6c387f9 to
cada816
Compare
The Zstd decoder was silently accepting truncated streams by not validating stream completion in finish(). This issue was discovered by the generic truncated stream test. Added state tracking to ZstdDecoder: - Added stream_ended field to track if remaining == 0 was seen - Modified decode() to set stream_ended = true when stream completes - Updated finish() to check stream_ended and return UnexpectedEof if false - Updated all constructors to initialize stream_ended = false This matches the behavior of other decoders (bzip2, gzip, lz4, etc.) and ensures applications cannot accidentally accept corrupted or incomplete zstd data as valid. The generic truncated test now passes for Zstd.
baszalmstra
force-pushed
the
claude/implement-issue-411-011CUpzAf5yUB1s58RpG9Fzp
branch
from
cada816 to
64b9392
Compare
Collaborator
|
I will try to cut a release after merging |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #412 +/- ##
===========================
===========================
☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #411
This adds a test that verifies if receiving truncated compressed data yields an
UnexpectedEoferror.While adding the test, I found three decoders that fail this test. I fixed all of them.