Skip to content

Conversation

@jonathanc-n
Copy link
Contributor

Which issue does this PR close?

Closes #13302 .

Rationale for this change

What changes are included in this PR?

Added random timezones to fuzz test for timestamps

Copy link
Contributor

@LeslieKid LeslieKid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍This PR is great.

The only thing i am concerned is -- the timestamp part in baseline_config() function in aggr_fuzz.rs is:

ColumnDescr::new("timestamp_s", DataType::Timestamp(TimeUnit::Second, None)),
ColumnDescr::new(
    "timestamp_ms",
    DataType::Timestamp(TimeUnit::Millisecond, None),
),
ColumnDescr::new(
    "timestamp_us",
    DataType::Timestamp(TimeUnit::Microsecond, None),
),
ColumnDescr::new(
    "timestamp_ns",
    DataType::Timestamp(TimeUnit::Nanosecond, None),
),

The implementation will generate the timezone randomly. So I think the None here may confuse the reader. Maybe we can add some comments here, or use generate_timezone() function to replace the None directly to make the code clear?

@jonathanc-n
Copy link
Contributor Author

@LeslieKid Yeah I can definitely add a comment, as using generate_timezone() directly in the function makes it difficult to pass into the generate_primitive_array, as that function is used for other primitive types.

@github-actions github-actions bot added the core Core DataFusion crate label Nov 12, 2024
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great to me -- thank you @jonathanc-n and @LeslieKid

ColumnDescr::new("time32_ms", DataType::Time32(TimeUnit::Millisecond)),
ColumnDescr::new("time64_us", DataType::Time64(TimeUnit::Microsecond)),
ColumnDescr::new("time64_ns", DataType::Time64(TimeUnit::Nanosecond)),
// `None` is passed in here however when generating the array, it will generate
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

use arrow::datatypes::DataType;
use rand::rngs::StdRng;
use rand::Rng;
use chrono_tz::{Tz, TZ_VARIANTS};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alamb alamb merged commit 99cdbcc into apache:main Nov 13, 2024
27 checks passed
@jonathanc-n jonathanc-n deleted the add-random-ts branch November 27, 2024 22:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add randomized timezones for timestamps

3 participants