[HADOOP-10724] better interoperation with sort -h
#3
+147
−147
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ngUtils.TraditionalBinaryPrefix.long2String to work better with `sort -h`
mekasone
pushed a commit
to mekasone/hadoop
that referenced
this pull request
Feb 19, 2017
Commit script from Keith
tangzhankun
pushed a commit
to tangzhankun/hadoop
that referenced
this pull request
Feb 24, 2017
Tensorflow doc
isimonenko
pushed a commit
to metamx/hadoop
that referenced
this pull request
Sep 15, 2017
Update jets3t and enable AWSv4 signature
steveloughran
added a commit
to steveloughran/hadoop
that referenced
this pull request
Sep 8, 2020
* Fix S3A test failures caused by DeleteOperation bugs added * Extend S3A tests * Fix FileContext base test so that raised IOEs are always rethrown regarding change apache#3, I think those IOEs should be thrown first, but as the existing code does the asserts in finally(), I'm leaving it alone Change-Id: Ia3159db7760424264d84fa2e38d2cbb452a9854a
12 tasks
qinghui-xu
pushed a commit
to qinghui-xu/hadoop
that referenced
this pull request
Feb 4, 2022
…for datanode on ext4. Contributed by Chris Trezzo." (apache#3) This reverts commit 2c8496e. Co-authored-by: William Montaz <[email protected]>
Yifan122
added a commit
to Yifan122/hadoop-apache
that referenced
this pull request
Apr 22, 2023
NyteKnight
pushed a commit
to NyteKnight/hadoop
that referenced
this pull request
Jun 25, 2024
steveloughran
added a commit
to steveloughran/hadoop
that referenced
this pull request
Oct 1, 2024
This sets a different timeout for put/post calls to all other requests, so that option fs.s3a.connection.part.upload.timeout default 15m Although itests show the option is being applied to put/part uploads, and interpreted by the SDK, a full command line tests is showing a failure at 60s, as before. fMrSmr7TgYCqPw1C2tI_tM0A_TzaYJWPfwVxnE9MyC on hadoop-3.4.1.tar.gz._COPYING_: Retried 1: org.apache.hadoop.fs.s3a.AWSApiCallTimeoutException: upload part apache#3 on hadoop-3.4.1.tar.gz._COPYING_: software.amazon.awssdk.core.exception.ApiCallAttemptTimeoutException: HTTP request execution did not complete before the specified timeout configuration: 60000 millis This actually validates that the upload recovery is good, which makes me happy 2024-10-01 18:30:50,287 [s3a-transfer-stevel-london-bounded-pool1-t1] INFO impl.UploadContentProviders (UploadContentProviders.java:newStream(278)) - Stream created more than once: FileWithOffsetContentProvider{file=/tmp/hadoop-stevel/s3a/s3ablock-0001-751923718162888182.tmp, offset=0} BaseContentProvider{size=67108864, streamCreationCount=7, currentStream=null} Change-Id: I84e594eae55746a85f58b05ad376173ddbbc3ad1
4 tasks
singer-bin
pushed a commit
to singer-bin/hadoop
that referenced
this pull request
Dec 19, 2024
Original from the old repo: Parquet/parquet-mr#410 JIRA: https://issues.apache.org/jira/browse/PARQUET-2 These changes allow primitive types to be requested as different types than what is stored in the file format using a flag to turn off strict type checking (default is on). Types are cast to the requested type where possible and will suffer precision loss for casting where necessary (e.g. requesting a double as an int). No performance penalty is imposed for using the type defined in the file type. A flag exists to A 6x6 test case is provided to test conversion between the primitive types. Author: Daniel Weeks <[email protected]> Closes apache#3 from dcw-netflix/type-persuasion and squashes the following commits: 97f4e9a [Daniel Weeks] Added documentation as suggested by code review 1c3c0c7 [Daniel Weeks] Fixed test with strict checking off f3cb495 [Daniel Weeks] Added type persuasion for primitive types with a flag to control strict type checking for conflicting schemas, which is strict by default.
singer-bin
pushed a commit
to singer-bin/hadoop
that referenced
this pull request
Dec 19, 2024
…2 api Currently for creating a user defined predicate using the new filter api, no value can be passed to create a dynamic filter at runtime. This reduces the usefulness of the user defined predicate, and meaningful predicates cannot be created. We can add a generic Object value that is passed through the api, which can internally be used in the keep function of the user defined predicate for creating many different types of filters. For example, in spark sql, we can pass in a list of filter values for a where IN clause query and filter the row values based on that list. Author: Yash Datta <[email protected]> Author: Alex Levenson <[email protected]> Author: Yash Datta <[email protected]> Closes apache#73 from saucam/master and squashes the following commits: 7231a3b [Yash Datta] Merge pull request apache#3 from isnotinvain/alexlevenson/fix-binary-compat dcc276b [Alex Levenson] Ignore binary incompatibility in private filter2 class 7bfa5ad [Yash Datta] Merge pull request apache#2 from isnotinvain/alexlevenson/simplify-udp-state 0187376 [Alex Levenson] Resolve merge conflicts 25aa716 [Alex Levenson] Simplify user defined predicates with state 51952f8 [Yash Datta] PARQUET-116: Fix whitespace d7b7159 [Yash Datta] PARQUET-116: Make UserDefined abstract, add two subclasses, one accepting udp class, other accepting serializable udp instance 40d394a [Yash Datta] PARQUET-116: Fix whitespace 9a63611 [Yash Datta] PARQUET-116: Fix whitespace 7caa4dc [Yash Datta] PARQUET-116: Add ConfiguredUserDefined that takes a serialiazble udp directly 0eaabf4 [Yash Datta] PARQUET-116: Move the config object from keep method to a configure method in udp predicate f51a431 [Yash Datta] PARQUET-116: Adding type safety for the filter object to be passed to user defined predicate d5a2b9e [Yash Datta] PARQUET-116: Enforce that the filter object to be passed must be Serializable dfd0478 [Yash Datta] PARQUET-116: Add a test case for passing a filter object to user defined predicate 4ab46ec [Yash Datta] PARQUET-116: Pass a filter object to user defined predicate in filter2 api
4 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
do not insert a space between number and units in StringUtils.TraditionalBinaryPrefix.long2String to work better with
sort -h