std:tar Copy Go tar test suite and make them pass #18261
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ianic
force-pushed
the
tar_tests
branch
2 times, most recently
from
9ae3edd to
bf53d4e
Compare
Split reading/parsing tar file and writing results to the disk in two separate steps. So we can later test parsing part without need to write everyting to the disk.
Move reader into Buffer and make it BufferedReader. This doesn't introduce any new functionality just grouping similar things.
Just adding tests, without changing functionality.
Name of symbolic link can be also found in pax attribute.
Make it more readable.
Reference: https://www.gnu.org/software/tar/manual/html_node/Extensions.html#Extensions If the leading byte is 0x80 (128), the non-leading bytes of the field are concatenated in big-endian order, with the result being a positive number expressed in binary form.
That makes names strings stable during the iteration. Otherwise string buffers can be overwritten while reading file content.
Skip tests if env is not set.
To make it little easier to filter from all stdlib tests.
So we have information to set executable bit on write to file system.
Make it little readable.
Use explicit buffers for name, link_name instead. It is cleaner that way.
Create std/tar/test.zig for test which uses cases from testdata.
Like in other tests which uses testdata files (compress). That enables wasi testing also, was failing because file system operations in tests.
Itarator has `next` function, iterates over tar files. When using from
outside of module with `tar.` prefix makes more sense.
var iter = tar.iterator(reader, null);
while (try iter.next()) |file| {
...
}
Using Python testtar file (mentioned in ziglang#14310) to test diagnostic reporting. Added computing checksum by using both unsigned and signed header bytes values. Added skipping gnu exteneded sparse headers while reporting unsupported header in diagnostic. Note on testing: wget https://github.com/python/cpython/raw/3.11/Lib/test/testtar.tar -O /tmp/testtar.tar ``` test "Python testtar.tar file" { const file_name = "testtar.tar"; var file = try std.fs.cwd().openFile("/tmp/" ++ file_name, .{}); defer file.close(); var diag = Options.Diagnostics{ .allocator = std.testing.allocator }; defer diag.deinit(); var iter = iterator(file.reader(), &diag); while (try iter.next()) |f| { std.debug.print("supported: {} {s} {d}\n", .{ f.kind, f.name, f.size }); try f.skip(); } for (diag.errors.items) |e| { switch (e) { .unsupported_file_type => |u| { std.debug.print("unsupported: {} {s}\n", .{ u.file_type, u.file_name }); }, else => unreachable, } } } ```
Member
|
Nice work! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #14310.
Go test cases are copied to
lib/std/tar/testdatafolder.New functionality added to pass tests:
This passes all relevant Go tests expect those connected with sparse files. Those functionality is not yet implemented. We are using this for package manager, meaning that most of the files will be source code files, pretty unlikely that we will find there tar with sparse files.