Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: ShapeIncreasingDistanceValidator #1083

Merged
merged 24 commits into from
Jan 25, 2022
Merged

fix: ShapeIncreasingDistanceValidator #1083

Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
e8e93dc
fix(ShapeIncreasingDistanceValidator): change rule logic
Dec 22, 2021
d7c3795
apply goJF
Dec 22, 2021
ca1e532
apply 1 meter threshold
Dec 23, 2021
4baa553
Merge branch 'master' into fix-ShapeIncreasingDistanceValidator
Jan 10, 2022
308631b
apply code review suggestions
Jan 10, 2022
cff81ed
modify rule logic and update documentation
Jan 10, 2022
737adab
refine rule logic
Jan 10, 2022
cbf9e79
remove unused method
Jan 10, 2022
574cd7b
modify rule logic
Jan 10, 2022
f2c5c77
increase threshold for test
Jan 11, 2022
a34def9
retrieve actual distance between shapepoints for analysis
Jan 12, 2022
0d80342
fix typo and comment test
Jan 12, 2022
66a8c5d
fix typo and comment test
Jan 12, 2022
6fe6240
modify rule logic
Jan 18, 2022
54242ea
format code ð
Jan 18, 2022
2a2f13f
modify rule logic
Jan 21, 2022
e1be2fa
remove changes on StopTimeIncreasingDistanceValidator
Jan 21, 2022
11ba8b6
additional unit test
Jan 21, 2022
29fd823
clarify documentation
Jan 21, 2022
3b91380
apply suggestions from code review [acceptance test skip]
lionel-nj Jan 21, 2022
0ee67a7
Merge branch 'fix-ShapeIncreasingDistanceValidator' of github.com:Mob…
Jan 21, 2022
e36a9c8
format code
Jan 21, 2022
757df8b
doc updates [acceptance test skip]
Jan 24, 2022
bd48299
doc updates [acceptance test skip]
Jan 24, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 24 additions & 4 deletions RULES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,13 +71,14 @@ Additional details regarding the notices' context is provided in [`NOTICES.md`](
|-----------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------|
| [`BlockTripsWithOverlappingStopTimesNotice`](#BlockTripsWithOverlappingStopTimesNotice) | Block trips with overlapping stop times. |
| [`CsvParsingFailedNotice`](#CsvParsingFailedNotice) | Parsing of a CSV file failed. |
| [`DecreasingOrEqualShapeDistanceNotice`](#DecreasingOrEqualShapeDistanceNotice) | Decreasing or equal `shape_dist_traveled` in `shapes.txt`. |
| [`DecreasingShapeDistanceNotice`](#DecreasingShapeDistanceNotice) | Decreasing `shape_dist_traveled` in `shapes.txt`. |
| [`DecreasingOrEqualStopTimeDistanceNotice`](#DecreasingOrEqualStopTimeDistanceNotice) | Decreasing or equal `shape_dist_traveled` in `stop_times.txt`. |
| [`DuplicatedColumnNotice`](#DuplicatedColumnNotice) | Duplicated column in CSV. |
| [`DuplicateFareRuleZoneIdFieldsNotice`](#DuplicateFareRuleZoneIdFieldsNotice) | Duplicate rows from `fare_rules.txt` based on `fare_rules.route_id`, `fare_rules.origin_id`, `fare_rules.contains_id` and `fare_rules.destination_id`. |
| [`DuplicateKeyNotice`](#DuplicateKeyNotice) | Duplicated entity. |
| [`EmptyColumnNameNotice`](#EmptyColumnNameNotice) | A column name is empty. |
| [`EmptyFileNotice`](#EmptyFileNotice) | A CSV file is empty. |
| [`EqualShapeDistanceDiffCoordinatesNotice`](#EqualShapeDistanceDiffCoordinatesNotice) | Equal `shape_dist_traveled` in `shapes.txt`. |
lionel-nj marked this conversation as resolved.
Show resolved Hide resolved
| [`ForeignKeyViolationNotice`](#ForeignKeyViolationNotice) | Wrong foreign key. |
| [`InconsistentAgencyTimezoneNotice`](#InconsistentAgencyTimezoneNotice) | Inconsistent Timezone among agencies. |
| [`InvalidColorNotice`](#InvalidColorNotice) | A field contains an invalid color value. |
Expand Down Expand Up @@ -128,6 +129,7 @@ Additional details regarding the notices' context is provided in [`NOTICES.md`](
| [`AttributionWithoutRoleNotice`](#AttributionWithoutRoleNotice) | Attribution with no role. |
| [`DuplicateRouteNameNotice`](#DuplicateRouteNameNotice) | Two distinct routes have either the same `route_short_name`, the same `route_long_name`, or the same combination of `route_short_name` and `route_long_name`. |
| [`EmptyRowNotice`](#EmptyRowNotice) | A row in the input file has only spaces. |
| [`EqualShapeDistanceSameCoordinatesNotice`](#EqualShapeDistanceSameCoordinatesNotice) | Equal `shape_dist_traveled` in `shapes.txt`. |
lionel-nj marked this conversation as resolved.
Show resolved Hide resolved
| [`FastTravelBetweenConsecutiveStopsNotice`](#FastTravelBetweenConsecutiveStopsNotice) | A transit vehicle moves too fast between two consecutive stops. |
| [`FastTravelBetweenFarStopsNotice`](#FastTravelBetweenFarStopsNotice) | A transit vehicle moves too fast between two far stops. |
| [`FeedExpirationDateNotice`](#FeedExpirationDateNotice) | Dataset should be valid for at least the next 7 days. Dataset should cover at least the next 30 days of service. |
Expand Down Expand Up @@ -199,11 +201,11 @@ Trips with the same block id have overlapping stop times.

Parsing of a CSV file failed. One common case of the problem is when a cell value contains more than 4096 characters.

<a name="DecreasingOrEqualShapeDistanceNotice"/>
<a name="DecreasingShapeDistanceNotice"/>

#### DecreasingOrEqualShapeDistanceNotice
#### DecreasingShapeDistanceNotice

When sorted by `shape.shape_pt_sequence`, two consecutive shape points should have increasing values for `shape_dist_traveled`. If the values are equal, this is considered as an error.
When sorted by `shape.shape_pt_sequence`, two consecutive shape points must not have decreasing values for `shape_dist_traveled`.

##### References:
* [shapes.txt specification](https://gtfs.org/reference/static#shapestxt)
Expand Down Expand Up @@ -262,6 +264,15 @@ Empty csv file found in the archive: file does not have any headers, or is a req
##### References:
* [GTFS files requirements](https://gtfs.org/reference/static#file-requirements)

#### EqualShapeDistanceDiffCoordinatesNotice

<a name="EqualShapeDistanceDiffCoordinatesNotice"/>

When sorted by `shape.shape_pt_sequence`, two consecutive shape points with different coordinates must not have equal values for `shape_dist_traveled`.
lionel-nj marked this conversation as resolved.
Show resolved Hide resolved

##### References:
* [shapes.txt specification](https://gtfs.org/reference/static#shapestxt)

<a name="ForeignKeyViolationNotice"/>

#### ForeignKeyViolationNotice
Expand Down Expand Up @@ -685,6 +696,15 @@ A row in the input file has only spaces.
##### References:
* [GTFS file requirements](http://gtfs.org/reference/static/#file-requirements)

#### EqualShapeDistanceSameCoordinatesNotice

<a name="EqualShapeDistanceSameCoordinatesNotice"/>

When sorted by `shape.shape_pt_sequence`, two consecutive shape points should not have equal values for `shape_dist_traveled`.
lionel-nj marked this conversation as resolved.
Show resolved Hide resolved

##### References:
* [shapes.txt specification](https://gtfs.org/reference/static#shapestxt)

<a name="FastTravelBetweenConsecutiveStopsNotice"/>

#### FastTravelBetweenConsecutiveStopsNotice
Expand Down
41 changes: 39 additions & 2 deletions docs/NOTICES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,14 @@
|-------------------------------------------------------- |----------------------------------------------------------------------------------------------------------------- |
| `block_trips_with_overlapping_stop_times` | [`BlockTripsWithOverlappingStopTimesNotice`](#BlockTripsWithOverlappingStopTimesNotice) |
| `csv_parsing_failed` | [`CsvParsingFailedNotice`](#CsvParsingFailedNotice) |
| `decreasing_or_equal_shape_distance` | [`DecreasingOrEqualShapeDistanceNotice`](#DecreasingOrEqualShapeDistanceNotice) |
| `decreasing_shape_distance` | [`DecreasingShapeDistanceNotice`](#DecreasingShapeDistanceNotice) |
| `decreasing_or_equal_stop_time_distance` | [`DecreasingOrEqualStopTimeDistanceNotice`](#DecreasingOrEqualStopTimeDistanceNotice) |
| `duplicated_column` | [`DuplicatedColumnNotice`](#DuplicatedColumnNotice) |
| `duplicate_fare_rule_zone_id_fields` | [`DuplicateFareRuleZoneIdFieldsNotice`](#DuplicateFareRuleZoneIdFieldsNotice) |
| `duplicate_key` | [`DuplicateKeyNotice`](#DuplicateKeyNotice) |
| `empty_column_name` | [`EmptyColumnNameNotice`](#EmptyColumnNameNotice) |
| `empty_file` | [`EmptyFileNotice`](#EmptyFileNotice) |
| `equal_shape_distance_diff_coordinates` | [`EqualShapeDistanceDiffCoordinatesNotice`](#EqualShapeDistanceDiffCoordinatesNotice) |
| `foreign_key_violation` | [`ForeignKeyViolationNotice`](#ForeignKeyViolationNotice) |
| `inconsistent_agency_timezone` | [`InconsistentAgencyTimezoneNotice`](#InconsistentAgencyTimezoneNotice) |
| `invalid_color` | [`InvalidColorNotice`](#InvalidColorNotice) |
Expand Down Expand Up @@ -88,7 +89,7 @@
##### Affected files
[All GTFS files supported by the specification.](http://gtfs.org/reference/static#dataset-files)

#### [`DecreasingOrEqualShapeDistanceNotice`](/RULES.md#DecreasingOrEqualShapeDistanceNotice)
#### [`DecreasingShapeDistanceNotice`](/RULES.md#DecreasingShapeDistanceNotice)
##### Fields description

| Field name | Description | Type |
Expand Down Expand Up @@ -183,6 +184,23 @@
##### Affected files
[All GTFS files supported by the specification.](http://gtfs.org/reference/static#dataset-files)

#### [`EqualShapeDistanceDiffCoordinatesNotice`](/RULES.md#EqualShapeDistanceDiffCoordinatesNotice)
##### Fields description

| Field name | Description | Type |
|----------------------- |------------------------------------------------------------------------------------------------- |--------- |
| `shapeId` | The id of the faulty shape. | String |
| `csvRowNumber` | The row number from `shapes.txt`. | Long |
| `shapeDistTraveled` | Actual distance traveled along the shape from the first shape point to the faulty record. | Double |
| `shapePtSequence` | The faulty record's `shapes.shape_pt_sequence`. | Integer |
| `prevCsvRowNumber` | The row number from `shapes.txt` of the previous shape point. | Long |
| `prevShapeDistTraveled` | Actual distance traveled along the shape from the first shape point to the previous shape point. | Double |
| `prevShapePtSequence` | The previous record's `shapes.shape_pt_sequence`. | Integer |

##### Affected files
* [`stops.txt`](http://gtfs.org/reference/static#stopstxt)
* [`shapes.txt`](http://gtfs.org/reference/static#shapestxt)

#### [`ForeignKeyViolationNotice`](/RULES.md#ForeignKeyViolationNotice)
##### Fields description

Expand Down Expand Up @@ -731,6 +749,7 @@
| `attribution_without_role` | [`AttributionWithoutRoleNotice`](#AttributionWithoutRoleNotice) |
| `duplicate_route_name` | [`DuplicateRouteNameNotice`](#DuplicateRouteNameNotice) |
| `empty_row` | [`EmptyRowNotice`](#EmptyRowNotice) |
| `equal_shape_distance_same_coordinates` | [`EqualShapeDistanceSameCoordinatesNotice`](#EqualShapeDistanceSameCoordinatesNotice) |
| `fast_travel_between_consecutive_stops` | [`FastTravelBetweenConsecutiveStopsNotice`](#FastTravelBetweenConsecutiveStopsNotice) |
| `fast_travel_between_far_stops` | [`FastTravelBetweenFarStopsNotice`](#FastTravelBetweenFarStopsNotice) |
| `feed_expiration_date` | [`FeedExpirationDateNotice`](#FeedExpirationDateNotice) |
Expand Down Expand Up @@ -805,6 +824,24 @@
##### Affected files
[All GTFS files supported by the specification.](http://gtfs.org/reference/static#dataset-files)


#### [EqualShapeDistanceSameCoordinatesNotice](/RULES.md#EqualShapeDistanceSameCoordinatesNotice)
##### Fields description

| Field name | Description | Type |
|----------------------- |------------------------------------------------------------------------------------------------- |--------- |
| `shapeId` | The id of the faulty shape. | String |
| `csvRowNumber` | The row number from `shapes.txt`. | Long |
| `shapeDistTraveled` | Actual distance traveled along the shape from the first shape point to the faulty record. | Double |
| `shapePtSequence` | The faulty record's `shapes.shape_pt_sequence`. | Integer |
| `prevCsvRowNumber` | The row number from `shapes.txt` of the previous shape point. | Long |
| `prevShapeDistTraveled` | Actual distance traveled along the shape from the first shape point to the previous shape point. | Double |
| `prevShapePtSequence` | The previous record's `shapes.shape_pt_sequence`. | Integer |

##### Affected files
* [`stops.txt`](http://gtfs.org/reference/static#stopstxt)
* [`shapes.txt`](http://gtfs.org/reference/static#shapestxt)

#### [FeedExpirationDateNotice](/RULES.md#FeedExpirationDateNotice)
##### Fields description

Expand Down
Loading