Skip to content

Commit f57025b

Browse files
author
Stephen Wu
committed
address comments
1 parent 7d4c4f1 commit f57025b

File tree

2 files changed

+15
-3
lines changed
  • hadoop-tools/hadoop-aws/src

2 files changed

+15
-3
lines changed

hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1204,8 +1204,17 @@ private Constants() {
12041204
*/
12051205
public static final int DEFAULT_AWS_S3_VECTOR_READS_MAX_MERGED_READ_SIZE = 1253376; //1M
12061206

1207-
public static final String FAIL_ON_AWS_BAD_REQUEST = "fs.s3a.retry.failOnAwsBadRequest";
1207+
/**
1208+
* Flag for immediate failure when observing a {@link AWSBadRequestException}.
1209+
* If it's disabled and set to false, the failure is treated as retryable.
1210+
* Value {@value}.
1211+
*/
1212+
public static final String FAIL_ON_AWS_BAD_REQUEST = "fs.s3a.fail.on.aws.bad.request";
12081213

1214+
/**
1215+
* Default value for immediate failure when observing a
1216+
* {@link AWSBadRequestException}: {@value}.
1217+
*/
12091218
public static final boolean DEFAULT_FAIL_ON_AWS_BAD_REQUEST = true;
12101219

12111220
}

hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1118,12 +1118,15 @@ from them.
11181118

11191119
* Connection timeout: `ConnectTimeoutException`. Timeout before
11201120
setting up a connection to the S3 endpoint (or proxy).
1121-
* HTTP response status code 400, "Bad Request"
1121+
* HTTP response status code 400, "Bad Request" aka `AWSBadRequestException`
11221122

11231123
The status code 400, Bad Request usually means that the request
11241124
is unrecoverable; it's the generic "No" response. Very rarely it
11251125
does recover, which is why it is in this category, rather than that
1126-
of unrecoverable failures.
1126+
of unrecoverable failures. The default behavior fails immediately
1127+
without retry. If your system is failure sensitive, you can
1128+
configure `fs.s3a.fail.on.aws.bad.request` to `false` and allow
1129+
to retry when observing a Bad Request with status code 400.
11271130

11281131
These failures will be retried with an exponential sleep interval set in
11291132
`fs.s3a.retry.interval`, up to the limit set in `fs.s3a.retry.limit`.

0 commit comments

Comments
 (0)