Skip to content

update min year for acquisition to 2017 #1582

@aysim319

Description

@aysim319

google symptoms later introduced issue dates preceding the current min_year for acquisition
when attempting to patch data, patch failed with the message

{"detail": 20171028, "file": "/common/covidcast/batch_issue_upload/issue_20171028", "event": "invalid issue directory day", "logger": "find_issue_specific_csv_files", "level": "warning", "timestamp": "2025-01-25T05:47:04.151974Z"}
{"detail": 20171029, "file": "/common/covidcast/batch_issue_upload/issue_20171029", "event": "invalid issue directory day", "logger": "find_issue_specific_csv_files", "level": "warning", "timestamp": "2025-01-25T05:47:04.152044Z"}
{"detail": 20171030, "file": "/common/covidcast/batch_issue_upload/issue_20171030", "event": "invalid issue directory day", "logger": "find_issue_specific_csv_files", "level": "warning", "timestamp": "2025-01-25T05:47:04.152130Z"}
{"detail": 20171031, "file": "/common/covidcast/batch_issue_upload/issue_20171031", "event": "invalid issue directory day", "logger": "find_issue_specific_csv_files", "level": "warning", "timestamp": "2025-01-25T05:47:04.152198Z"}

  MIN_YEAR = 2019

 def is_sane_day(value):
    """Return whether `value` is a sane (maybe not valid) YYYYMMDD date.

    Truthy return is is a datetime.date object representing `value`."""

    year, month, day = value // 10000, (value % 10000) // 100, value % 100

    nearby_year = CsvImporter.MIN_YEAR <= year <= CsvImporter.MAX_YEAR
    valid_month = 1 <= month <= 12
    sensible_day = 1 <= day <= 31

    if not (nearby_year and valid_month and sensible_day):
      return False
    return date(year=year,month=month,day=day)

blocks cmu-delphi/covidcast-indicators#2107

Metadata

Metadata

Assignees

Labels

acquisitionchanges acquisition logic

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions