Exclude private datasets from 6 geoportal datasets, GTFS digest #1223
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
gtfs_utils_v2
to grab either a list or the df of public datasets - normally, this is preferred because we get a list ofschedule_gtfs_dataset_keys
we can keep.publish_utils
-- sometimes in GTFS digest, we do not haveschedule_gtfs_dataset_key
, so we useschedule_gtfs_dataset_name
to filter.publish_open_data
via parquet filtering andpublish_public_gcs
merge_*
scripts so that all the dates are concatenated, and before exporting, private datasets are dropped. This should allow the portfolio yaml to build and report to run as usual.Big Blue Bus Swiftly Schedule
)TODO related GTFS Digest:
some redundancies were noticed while updating merge scripts. wherever references are shared, adapted functions to take additional uses.
service hours
, having weekday / weekend columns in addition to rows holding those values is confusing, since column values are mixing normalized (per day) and not normalized (sums). alsomonth
changed tomonth_year
becausemonth
typically holds month values (1-12 or Jan-Dec).merge_*
scripts to create new concatenated digest tables