diff --git a/docs/content.zh/docs/connectors/table/filesystem.md b/docs/content.zh/docs/connectors/table/filesystem.md index b3dfeb2bfcc79..effd8fb49b209 100644 --- a/docs/content.zh/docs/connectors/table/filesystem.md +++ b/docs/content.zh/docs/connectors/table/filesystem.md @@ -229,7 +229,7 @@ To define when to commit a partition, providing partition commit trigger:
sink.partition-commit.watermark-time-zone
UTC String - The time zone to parse the long watermark value to TIMESTAMP value, the parsed watermark timestamp is used to compare with partition time to decide the partition should commit or not. This option is only take effect when `sink.partition-commit.trigger` is set to 'partition-time'. If this option is not configured correctly, e.g. source rowtime is defined on TIMESTAMP_LTZ column, but this config is not configured, then users may see the partition committed after a few hours. The default value is 'UTC', which means the watermark is defined on TIMESTAMP column or not defined. If the watermark is defined on TIMESTAMP_LTZ column, the time zone of watermark is the session time zone. The option value is either a full name such as 'America/Los_Angeles', or a custom timezone id such as 'GMT-8:00'. + The time zone to parse the long watermark value to TIMESTAMP value, the parsed watermark timestamp is used to compare with partition time to decide the partition should commit or not. This option is only take effect when `sink.partition-commit.trigger` is set to 'partition-time'. If this option is not configured correctly, e.g. source rowtime is defined on TIMESTAMP_LTZ column, but this config is not configured, then users may see the partition committed after a few hours. The default value is 'UTC', which means the watermark is defined on TIMESTAMP column or not defined. If the watermark is defined on TIMESTAMP_LTZ column, the time zone of watermark is the session time zone. The option value is either a full name such as 'America/Los_Angeles', or a custom timezone id such as 'GMT-08:00'. diff --git a/docs/content/docs/connectors/table/filesystem.md b/docs/content/docs/connectors/table/filesystem.md index d9c53b4605d9b..4fa3458999ff5 100644 --- a/docs/content/docs/connectors/table/filesystem.md +++ b/docs/content/docs/connectors/table/filesystem.md @@ -229,7 +229,7 @@ To define when to commit a partition, providing partition commit trigger:
sink.partition-commit.watermark-time-zone
UTC String - The time zone to parse the long watermark value to TIMESTAMP value, the parsed watermark timestamp is used to compare with partition time to decide the partition should commit or not. This option is only take effect when `sink.partition-commit.trigger` is set to 'partition-time'. If this option is not configured correctly, e.g. source rowtime is defined on TIMESTAMP_LTZ column, but this config is not configured, then users may see the partition committed after a few hours. The default value is 'UTC', which means the watermark is defined on TIMESTAMP column or not defined. If the watermark is defined on TIMESTAMP_LTZ column, the time zone of watermark is the session time zone. The option value is either a full name such as 'America/Los_Angeles', or a custom timezone id such as 'GMT-8:00'. + The time zone to parse the long watermark value to TIMESTAMP value, the parsed watermark timestamp is used to compare with partition time to decide the partition should commit or not. This option is only take effect when `sink.partition-commit.trigger` is set to 'partition-time'. If this option is not configured correctly, e.g. source rowtime is defined on TIMESTAMP_LTZ column, but this config is not configured, then users may see the partition committed after a few hours. The default value is 'UTC', which means the watermark is defined on TIMESTAMP column or not defined. If the watermark is defined on TIMESTAMP_LTZ column, the time zone of watermark is the session time zone. The option value is either a full name such as 'America/Los_Angeles', or a custom timezone id such as 'GMT-08:00'. diff --git a/docs/data/sql_functions.yml b/docs/data/sql_functions.yml index 51df9d1bb30de..73abcf2ae0747 100644 --- a/docs/data/sql_functions.yml +++ b/docs/data/sql_functions.yml @@ -471,7 +471,7 @@ temporal: table: timestampDiff(TIMEPOINTUNIT, TIMEPOINT1, TIMEPOINT2) description: 'Returns the (signed) number of timepointunit between timepoint1 and timepoint2. The unit for the interval is given by the first argument, which should be one of the following values: SECOND, MINUTE, HOUR, DAY, MONTH, or YEAR.' - sql: CONVERT_TZ(string1, string2, string3) - description: Converts a datetime string1 (with default ISO timestamp format 'yyyy-MM-dd HH:mm:ss') from time zone string2 to time zone string3. The format of time zone should be either an abbreviation such as "PST", a full name such as "America/Los_Angeles", or a custom ID such as "GMT-8:00". E.g., CONVERT_TZ('1970-01-01 00:00:00', 'UTC', 'America/Los_Angeles') returns '1969-12-31 16:00:00'. + description: Converts a datetime string1 (with default ISO timestamp format 'yyyy-MM-dd HH:mm:ss') from time zone string2 to time zone string3. The format of time zone should be either an abbreviation such as "PST", a full name such as "America/Los_Angeles", or a custom ID such as "GMT-08:00". E.g., CONVERT_TZ('1970-01-01 00:00:00', 'UTC', 'America/Los_Angeles') returns '1969-12-31 16:00:00'. - sql: FROM_UNIXTIME(numeric[, string]) description: Returns a representation of the numeric argument as a value in string format (default is 'yyyy-MM-dd HH:mm:ss'). numeric is an internal timestamp value representing seconds since '1970-01-01 00:00:00' UTC, such as produced by the UNIX_TIMESTAMP() function. The return value is expressed in the session time zone (specified in TableConfig). E.g., FROM_UNIXTIME(44) returns '1970-01-01 00:00:44' if in UTC time zone, but returns '1970-01-01 09:00:44' if in 'Asia/Tokyo' time zone. - sql: UNIX_TIMESTAMP() diff --git a/docs/layouts/shortcodes/generated/table_config_configuration.html b/docs/layouts/shortcodes/generated/table_config_configuration.html index ff14b8bd4ffe4..72226bc27f271 100644 --- a/docs/layouts/shortcodes/generated/table_config_configuration.html +++ b/docs/layouts/shortcodes/generated/table_config_configuration.html @@ -30,7 +30,7 @@
table.local-time-zone

Batch Streaming "default" String - The local time zone defines current session time zone id. It is used when converting to/from <code>TIMESTAMP WITH LOCAL TIME ZONE</code>. Internally, timestamps with local time zone are always represented in the UTC time zone. However, when converting to data types that don't include a time zone (e.g. TIMESTAMP, TIME, or simply STRING), the session time zone is used during conversion. The input of option is either a full name such as "America/Los_Angeles", or a custom timezone id such as "GMT-8:00". + The local time zone defines current session time zone id. It is used when converting to/from <code>TIMESTAMP WITH LOCAL TIME ZONE</code>. Internally, timestamps with local time zone are always represented in the UTC time zone. However, when converting to data types that don't include a time zone (e.g. TIMESTAMP, TIME, or simply STRING), the session time zone is used during conversion. The input of option is either a full name such as "America/Los_Angeles", or a custom timezone id such as "GMT-08:00".
table.planner

Batch Streaming diff --git a/flink-python/pyflink/datastream/connectors.py b/flink-python/pyflink/datastream/connectors.py index 4d34d1f75db0c..e6b279f1e45d2 100644 --- a/flink-python/pyflink/datastream/connectors.py +++ b/flink-python/pyflink/datastream/connectors.py @@ -573,7 +573,7 @@ def date_time_bucket_assigner(format_str: str = "yyyy-MM-dd--HH", timezone_id: s :param format_str: The format string used to determine the bucket id. :param timezone_id: The timezone id, either an abbreviation such as "PST", a full name such as "America/Los_Angeles", or a custom timezone_id such as - "GMT-8:00". Th e default time zone will b used if it's None. + "GMT-08:00". Th e default time zone will b used if it's None. """ if timezone_id is not None and isinstance(timezone_id, str): j_timezone = get_gateway().jvm.java.time.ZoneId.of(timezone_id) diff --git a/flink-python/pyflink/table/table_config.py b/flink-python/pyflink/table/table_config.py index 7305102f7accb..5ecb855ac16ee 100644 --- a/flink-python/pyflink/table/table_config.py +++ b/flink-python/pyflink/table/table_config.py @@ -56,7 +56,7 @@ def get_local_timezone(self) -> str: """ Returns the local timezone id for timestamp with local time zone, either an abbreviation such as "PST", a full name such as "America/Los_Angeles", or a custom timezone_id such - as "GMT-8:00". + as "GMT-08:00". """ return self._j_table_config.getLocalTimeZone().getId() @@ -66,7 +66,7 @@ def set_local_timezone(self, timezone_id: str): :param timezone_id: The timezone id, either an abbreviation such as "PST", a full name such as "America/Los_Angeles", or a custom timezone_id such as - "GMT-8:00". + "GMT-08:00". """ if timezone_id is not None and isinstance(timezone_id, str): j_timezone = get_gateway().jvm.java.time.ZoneId.of(timezone_id) diff --git a/flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/operators/windowing/TimeWindowTest.java b/flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/operators/windowing/TimeWindowTest.java index 752c69d128127..57a4367851496 100644 --- a/flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/operators/windowing/TimeWindowTest.java +++ b/flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/operators/windowing/TimeWindowTest.java @@ -53,7 +53,7 @@ public void testGetWindowStartWithOffset() { Assert.assertEquals(TimeWindow.getWindowStartWithOffset(7, offset, 7), 5); Assert.assertEquals(TimeWindow.getWindowStartWithOffset(12, offset, 7), 12); - // for GMT+8:00 + // for GMT+08:00 offset = -TimeUnit.HOURS.toMillis(8); long size = TimeUnit.DAYS.toMillis(1); Assert.assertEquals( diff --git a/flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/TableConfig.java b/flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/TableConfig.java index 2e0ec26528c8a..c14db3b0f16d2 100644 --- a/flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/TableConfig.java +++ b/flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/TableConfig.java @@ -182,7 +182,7 @@ private void validateTimeZone(String zone) { throw new IllegalArgumentException( String.format( "The supported Zone ID is either a full name such as 'America/Los_Angeles'," - + " or a custom timezone id such as 'GMT-8:00', but configured Zone ID is '%s'.", + + " or a custom timezone id such as 'GMT-08:00', but configured Zone ID is '%s'.", zone)); } } diff --git a/flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/config/TableConfigOptions.java b/flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/config/TableConfigOptions.java index 9485d8dea9972..ddd9b417f9462 100644 --- a/flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/config/TableConfigOptions.java +++ b/flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/config/TableConfigOptions.java @@ -90,7 +90,7 @@ private TableConfigOptions() {} + "TIMESTAMP WITH LOCAL TIME ZONE. Internally, timestamps with local time zone are always represented in the UTC time zone. " + "However, when converting to data types that don't include a time zone (e.g. TIMESTAMP, TIME, or simply STRING), " + "the session time zone is used during conversion. The input of option is either a full name " - + "such as \"America/Los_Angeles\", or a custom timezone id such as \"GMT-8:00\"."); + + "such as \"America/Los_Angeles\", or a custom timezone id such as \"GMT-08:00\"."); @Documentation.TableOption(execMode = Documentation.ExecMode.BATCH_STREAMING) public static final ConfigOption MAX_LENGTH_GENERATED_CODE = diff --git a/flink-table/flink-table-api-java/src/test/java/org/apache/flink/table/api/TableConfigTest.java b/flink-table/flink-table-api-java/src/test/java/org/apache/flink/table/api/TableConfigTest.java index 6b22dee800219..299048ca6e752 100644 --- a/flink-table/flink-table-api-java/src/test/java/org/apache/flink/table/api/TableConfigTest.java +++ b/flink-table/flink-table-api-java/src/test/java/org/apache/flink/table/api/TableConfigTest.java @@ -66,23 +66,36 @@ public void testSetAndGetLocalTimeZone() { assertEquals(ZoneId.of("Asia/Shanghai"), configByMethod.getLocalTimeZone()); assertEquals(ZoneId.of("Asia/Shanghai"), configByConfiguration.getLocalTimeZone()); + + configuration.setString("table.local-time-zone", "GMT-08:00"); + configByConfiguration.addConfiguration(configuration); + configByMethod.setLocalTimeZone(ZoneId.of("GMT-08:00")); + + assertEquals(ZoneId.of("GMT-08:00"), configByMethod.getLocalTimeZone()); + assertEquals(ZoneId.of("GMT-08:00"), configByConfiguration.getLocalTimeZone()); } @Test public void testSetInvalidLocalTimeZone() { expectedException.expectMessage( "The supported Zone ID is either a full name such as 'America/Los_Angeles'," - + " or a custom timezone id such as 'GMT-8:00', but configured Zone ID is 'UTC-10:00'."); + + " or a custom timezone id such as 'GMT-08:00', but configured Zone ID is 'UTC-10:00'."); configByMethod.setLocalTimeZone(ZoneId.of("UTC-10:00")); } + @Test + public void testInvalidGmtLocalTimeZone() { + expectedException.expectMessage("Invalid ID for offset-based ZoneId: GMT-8:00"); + configByMethod.setLocalTimeZone(ZoneId.of("GMT-8:00")); + } + @Test public void testGetInvalidLocalTimeZone() { configuration.setString("table.local-time-zone", "UTC+8"); configByConfiguration.addConfiguration(configuration); expectedException.expectMessage( "The supported Zone ID is either a full name such as 'America/Los_Angeles'," - + " or a custom timezone id such as 'GMT-8:00', but configured Zone ID is 'UTC+8'."); + + " or a custom timezone id such as 'GMT-08:00', but configured Zone ID is 'UTC+8'."); configByConfiguration.getLocalTimeZone(); } @@ -92,7 +105,7 @@ public void testGetInvalidAbbreviationLocalTimeZone() { configByConfiguration.addConfiguration(configuration); expectedException.expectMessage( "The supported Zone ID is either a full name such as 'America/Los_Angeles'," - + " or a custom timezone id such as 'GMT-8:00', but configured Zone ID is 'PST'."); + + " or a custom timezone id such as 'GMT-08:00', but configured Zone ID is 'PST'."); configByConfiguration.getLocalTimeZone(); } diff --git a/flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/filesystem/FileSystemOptions.java b/flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/filesystem/FileSystemOptions.java index bfc17e487162b..11137805b7b72 100644 --- a/flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/filesystem/FileSystemOptions.java +++ b/flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/filesystem/FileSystemOptions.java @@ -229,7 +229,7 @@ public class FileSystemOptions { + " The default value is 'UTC', which means the watermark is defined on TIMESTAMP column or not defined." + " If the watermark is defined on TIMESTAMP_LTZ column, the time zone of watermark is user configured time zone," + " the the value should be the user configured local time zone. The option value is either a full name" - + " such as 'America/Los_Angeles', or a custom timezone id such as 'GMT-8:00'."); + + " such as 'America/Los_Angeles', or a custom timezone id such as 'GMT-08:00'."); public static final ConfigOption SINK_PARTITION_COMMIT_POLICY_KIND = key("sink.partition-commit.policy.kind") diff --git a/flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/filesystem/FileSystemTableFactory.java b/flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/filesystem/FileSystemTableFactory.java index 49eb2355ffbd4..96e5d4fc272f4 100644 --- a/flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/filesystem/FileSystemTableFactory.java +++ b/flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/filesystem/FileSystemTableFactory.java @@ -133,7 +133,7 @@ private void validate(FactoryUtil.TableFactoryHelper helper) { throw new ValidationException( String.format( "The supported watermark time zone is either a full name such as 'America/Los_Angeles'," - + " or a custom time zone id such as 'GMT-8:00', but configured time zone is '%s'.", + + " or a custom time zone id such as 'GMT-08:00', but configured time zone is '%s'.", watermarkTimeZone)); } } diff --git a/flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/util/TimeWindowUtil.java b/flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/util/TimeWindowUtil.java index 0fd20947e339a..56ffd493ae520 100644 --- a/flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/util/TimeWindowUtil.java +++ b/flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/util/TimeWindowUtil.java @@ -43,8 +43,8 @@ public class TimeWindowUtil { /** * Convert a epoch mills to timestamp mills which can describe a locate date time. * - *

For example: The timestamp string of epoch mills 5 in GMT+8:00 is 1970-01-01 08:00:05, the - * timestamp mills is 8 * 60 * 60 * 1000 + 5. + *

For example: The timestamp string of epoch mills 5 in GMT+08:00 is 1970-01-01 08:00:05, + * the timestamp mills is 8 * 60 * 60 * 1000 + 5. * * @param epochMills the epoch mills. * @param shiftTimeZone the timezone that the given timestamp mills has been shifted. diff --git a/flink-table/flink-table-runtime-blink/src/test/java/org/apache/flink/table/filesystem/FileSystemTableFactoryTest.java b/flink-table/flink-table-runtime-blink/src/test/java/org/apache/flink/table/filesystem/FileSystemTableFactoryTest.java index 9bc33a9982844..332e68c240550 100644 --- a/flink-table/flink-table-runtime-blink/src/test/java/org/apache/flink/table/filesystem/FileSystemTableFactoryTest.java +++ b/flink-table/flink-table-runtime-blink/src/test/java/org/apache/flink/table/filesystem/FileSystemTableFactoryTest.java @@ -166,7 +166,7 @@ public void testUnsupportedWatermarkTimeZone() { .contains( "The supported watermark time zone is either a full name such " + "as 'America/Los_Angeles', or a custom time zone id such " - + "as 'GMT-8:00', but configured time zone is 'UTC+8'.")); + + "as 'GMT-08:00', but configured time zone is 'UTC+8'.")); return; } diff --git a/flink-table/flink-table-runtime-blink/src/test/java/org/apache/flink/table/runtime/operators/aggregate/window/SlicingWindowAggOperatorTest.java b/flink-table/flink-table-runtime-blink/src/test/java/org/apache/flink/table/runtime/operators/aggregate/window/SlicingWindowAggOperatorTest.java index 483979cef60fa..50c469a55cc8a 100644 --- a/flink-table/flink-table-runtime-blink/src/test/java/org/apache/flink/table/runtime/operators/aggregate/window/SlicingWindowAggOperatorTest.java +++ b/flink-table/flink-table-runtime-blink/src/test/java/org/apache/flink/table/runtime/operators/aggregate/window/SlicingWindowAggOperatorTest.java @@ -692,8 +692,8 @@ public void testProcessingTimeTumblingWindows() throws Exception { final SliceAssigner assigner = SliceAssigners.tumbling(-1, shiftTimeZone, Duration.ofHours(5)); - // the assigned windows should like as following, e.g. the given timeZone is GMT+8:00: - // local windows(timestamp in GMT+8:00) <=> epoch windows(timestamp in UTC) + // the assigned windows should like as following, e.g. the given timeZone is GMT+08:00: + // local windows(timestamp in GMT+08:00) <=> epoch windows(timestamp in UTC) // [1970-01-01 00:00, 1970-01-01 05:00] <=> [1969-12-31 16:00, 1969-12-31 21:00] // [1970-01-01 05:00, 1970-01-01 10:00] <=> [1969-12-31 21:00, 1970-01-01 02:00]