-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GTFS RT date, time, and trip labeling updates #2489
Conversation
cea799d
to
8677f14
Compare
3355760
to
2084be3
Compare
2084be3
to
593891e
Compare
f0da795
to
8ba2fd0
Compare
d86d494
to
f55f156
Compare
Warehouse report 📦 Checks/potential follow-upsChecks indicate the following action items may be necessary.
New models 🌱calitp_warehouse.intermediate.gtfs.int_gtfs_rt__service_alerts_day_map_grouping calitp_warehouse.intermediate.gtfs.int_gtfs_rt__service_alerts_trip_day_map_grouping calitp_warehouse.intermediate.gtfs.int_gtfs_rt__trip_updates_trip_day_map_grouping calitp_warehouse.intermediate.gtfs.int_gtfs_rt__vehicle_positions_trip_day_map_grouping Changed incremental models 🔀calitp_warehouse.mart.gtfs_quality.fct_daily_service_alerts_message_age_summary calitp_warehouse.mart.ad_hoc.fct_daily_trip_update_status_counts calitp_warehouse.mart.gtfs_quality.fct_daily_trip_updates_message_age_summary calitp_warehouse.mart.ad_hoc.fct_daily_vehicle_location_feed_trip_counts calitp_warehouse.mart.gtfs_quality.fct_daily_vehicle_positions_message_age_summary calitp_warehouse.mart.gtfs.fct_service_alerts_messages_unnested calitp_warehouse.mart.gtfs.fct_trip_updates_no_stop_times calitp_warehouse.mart.gtfs.fct_vehicle_locations DAGLegend (in order of precedence)
|
ded96a8
to
d12a173
Compare
6c27670
to
d1dd10c
Compare
decided this PR is getting too unwieldy, I'm going to break it up further and do each RT message type in dedicated PRs |
... forgot about |
7df6b24
to
9995925
Compare
53cfe91
to
6aa31a1
Compare
ca62470
to
5d393bd
Compare
* update profile based on observed behavior * address failing test and update keying * only test on sample * update intermediate tests * working fix for schedule change issue * sampled where updates * add schedule URL to intermediate key * add comments * forward date URL to dataset mapping * get vp test passing * fix gtfs dataset join to use new url mapping * coalesce feed time zone and refactor schedule join to use url map * deprecate trip status count model per slack convo * minor refactors on vehicle locations * working -- get dataset info from url map
Description
Describe your changes and why you're making them. Please include the context, motivation, and relevant dependencies.
This PR fundamentally restructures date/time handling and trip keying in GTFS RT models. See this release note doc for a full description of the issues and fix approaches.
It also deletes the dbt model for
mart_ad_hoc.fct_daily_vehicle_location_trip_counts
, rather than update it to make it compatible with these changes, because it was causing issue #2732 and the model generally seems a bit redundant with other models that are available.Also adds
vehicle.wheelchairAccessible
to the trip updates raw messages models, I am not sure why it was missing before.Decided not to do:
Resolves #2413
Resolves #2347
Resolves #2732 by deleting the model
Fixes CAL-ITP-DATA-INFRA-25HX (x)
Type of change
How has this been tested?
Include commands/logs/screenshots as relevant.
Ran an unqualified
dbt run
anddbt test
, confirmed everything runs and no new tests failPost-merge follow-ups
Document any actions that must be taken post-merge to deploy or otherwise implement the changes in this PR (for example, running a full refresh of some incremental model in dbt). If these actions will take more than a few hours after the merge or if they will be completed by someone other than the PR author, please create a dedicated follow-up issue and link it here to track resolution.