Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updating ntd_id_2022 using airtable #3367

Merged
merged 1 commit into from
Jun 11, 2024
Merged

Updating ntd_id_2022 using airtable #3367

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions airflow/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
logs/*
airflow.sh
3 changes: 3 additions & 0 deletions ...gs/create_external_tables/airtable/external_airtable_california_transit_organizations.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -254,3 +254,6 @@ schema_fields:
- name: raw_ntd_id
type: STRING
mode: NULLABLE
- name: ntd_id_2022
type: STRING
mode: NULLABLE
1 change: 1 addition & 0 deletions ...dels/intermediate/transit_database/dimensions/int_transit_database__organizations_dim.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ int_transit_database__organizations_dim AS (
hq_county_geography,
is_public_entity,
raw_ntd_id,
ntd_id_2022,
public_currently_operating,
public_currently_operating_fixed_route,
_is_current,
Expand Down
2 changes: 1 addition & 1 deletion warehouse/models/mart/transit_database/_mart_transit_database.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ models:
renamed to `_deprecated__ntd_agency_to_organizations`).
- name: ntd_id_2022
description: |
Just pulls out the 5 digit component of the raw_ntd_id
Pulled from airtable, the 5 digit component of raw_ntd_id
- name: name
description: Organization name
- name: organization_type
Expand Down
4 changes: 1 addition & 3 deletions warehouse/models/mart/transit_database/dim_organizations.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,7 @@ dim_organizations AS (
WHEN _valid_from >= '2023-05-23' THEN raw_ntd_id
ELSE ntd_to_org.ntd_id
END AS ntd_id,
IF(LENGTH(ntd_id) >= 10,
SUBSTR(ntd_id, -5),
ntd_id) AS ntd_id_2022,
ntd_id_2022,
public_currently_operating,
public_currently_operating_fixed_route,
_is_current,
Expand Down
1 change: 1 addition & 0 deletions warehouse/models/staging/transit_database/stg_transit_database__organizations.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ stg_transit_database__organizations AS (
hq_county_geography,
is_public_entity = "Yes" AS is_public_entity,
raw_ntd_id,
ntd_id_2022,
public_currently_operating = "Yes" AS public_currently_operating,
public_currently_operating_fixed_route = "Yes" AS public_currently_operating_fixed_route,
FROM once_daily_organizations
Expand Down
Loading