-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deal with misaligned triplines #13
Comments
This issue is due to a data error on the provider's part. In certain cases, the first message in the feed is provided with an incomplete stop sequence—specifically, one missing the first stop in the route. This is then corrected in the follow-up feed message. Here is a minimal failure case I found whilst looking into this issue:
Inserting new stops after-the-fact in this way breaks the station sequence logic in
If this failure was silent, it'd be hard to know what to do about it. Luckily in the case of the MTA 7 feed every message having this problem has another schema violation (lol)—a timestamp on the first vehicle update that is set to zero. By removing messages with this schema violation we can also get rid of this style of data error. |
This turns out to be a very difficult error to recover from. The zero timestamp is extremely common in the dataset for some reason and leads to extremely high fragmentation in the trip-line data. The additional code complexity and time cost in |
A highly visible parsing artifact are trips which are for some reason coded as going linearly and then backwards in time:
The text was updated successfully, but these errors were encountered: