Skip to content

Conversation

@raebot
Copy link
Contributor

@raebot raebot commented Oct 31, 2025

This PR removes siuba from shared_utils/schedule_rt_utils by converting siuba queries to use SQLAlchemy ORM model querying. It also adds test coverage for the impacted functions and introduces pytest-recording for requests to BigQuery. This allows us to record and replay real data from the DB without having to make the expensive requests when we run the tests.

Relates to #1553

@raebot raebot force-pushed the mov/1553-remove-siuba-from-shared-utils branch from a49ed6c to 5b04c91 Compare October 31, 2025 18:58
@raebot raebot marked this pull request as draft October 31, 2025 21:34
@raebot raebot force-pushed the mov/1553-remove-siuba-from-shared-utils branch 5 times, most recently from befda73 to 0ebda36 Compare November 4, 2025 02:54
@raebot raebot marked this pull request as ready for review November 4, 2025 02:58
@raebot raebot requested review from edasmalchi and erikamov November 4, 2025 02:59
@raebot raebot self-assigned this Nov 4, 2025
Copy link

@erikamov erikamov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed these models with Rae comparing to dbt and the tables and PK looks right!

@raebot raebot requested a review from amandaha8 November 6, 2025 18:03
@raebot
Copy link
Contributor Author

raebot commented Nov 7, 2025

I asked @amandaha8 to try out these changes with a few files that use the sample_gtfs_dataset_key_to_organization_crosswalk function

  • gtfs_digest/18_rm_siuba.ipynb
  • gtfs_funnel/crosswalk_gtfs_dataset_key_to_organization.py
  • rt_scheduled_v_ran/06_vp_usable_exploration.ipynb

@raebot raebot force-pushed the mov/1553-remove-siuba-from-shared-utils branch from 440651b to 5e7f93d Compare November 7, 2025 19:13
raebot and others added 2 commits November 7, 2025 11:16
* add SQLAlchemy models for all impacted tables
* tests for all changed functions
* setup GitHub Actions to run tests
* remove siuba
* pytest-recording for recording and replaying BigQuery requests
* add dependencies needed for testing to requirements.txt

Co-authored-by: Erika Pacheco <[email protected]>
@raebot raebot force-pushed the mov/1553-remove-siuba-from-shared-utils branch from 5e7f93d to 2df4a42 Compare November 7, 2025 19:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants