Issue with hourly updates

[francois2metz]

Hi,

I have some trouble this morning with the hourly updates. The 097702 state and diff has an error: https://osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com/planet/replication/hour/000/097/702.state.txt

NoSuchKeyThe specified key does not exist.planet/replication/hour/000/097/702.state.txtSGQH7G1ERTA7G8Y0shqOMkDwPE7yiTFrMCRcS+8OJLd9G/wYyNRqBbeBLMDt0rB1+sJIQzgQ5DBq7uWO+qTwYVcq09U=

https://osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com/planet/replication/hour/000/097/702.osc.gz

NoSuchKeyThe specified key does not exist.planet/replication/hour/000/097/702.osc.gzAQC5FDHGV6V6N2AAcHrZObuOCZPUF2hDUiGUtrnqMDCNZpdl1PG4DJXIC4BbrXMd7n/ujCOxOOIl59o4O5k3y9hk+/k=

The next sequence number seems ok.

[tomhughes]

I've uploaded those manually but this does reveal a glitch in the S3 publishing so I'm going to leave this open for now.

Basically I think what happened is that osmosis produced two files when the hourly replication job ran because the database issues meant there was more than one hour's worth of data but we only synced the last one.

[francois2metz]

Thanks. I still cannot get the state file or the diff. Is there is any delay?

[tomhughes]

Sorry... Looks like I got the copy wrong. It should really be fixed now...

[francois2metz]

I's working again. Thanks!

[dieterdreist]

I am not sure if it is related, but I get problems with the daily updates:

--2023-11-10 18:40:36--  https://planet.osm.org/replication/day/000/004/067.osc.gz
Resolving planet.osm.org (planet.osm.org)... 2001:470:1:fa1::11, 184.104.179.145
Connecting to planet.osm.org (planet.osm.org)|2001:470:1:fa1::11|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com/planet/replication/day/000/004/067.osc.gz [following]
--2023-11-10 18:40:36--  https://osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com/planet/replication/day/000/004/067.osc.gz
Resolving osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com (osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com)... 2a05:d050:40af:8488:34db:a90a::, 2a05:d050:40a0:409:34db:4bdc::, 2a05:d050:402f:8501:34db:aa36::, ...
Connecting to osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com (osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com)|2a05:d050:40af:8488:34db:a90a::|:443... failed: Operation timed out.
Connecting to osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com (osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com)|2a05:d050:40a0:409:34db:4bdc::|:443... failed: Operation timed out.
Connecting to osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com (osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com)|2a05:d050:402f:8501:34db:aa36::|:443... failed: Operation timed out.
Connecting to osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com (osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com)|2a05:d050:40af:8490:34db:a912::|:443... failed: Operation timed out.

basically it downloads one daily change file every 10 minutes, the rest is timeouts and Network is down errors

[tomhughes]

No that is unrelated - that just shows that your machine is unable to reach AWS over IPv6 for some reason. It's some kind of routing issue with your ISP or something further down the chain between you and Amazon.

[francois2metz]

This issue happened again this morning:

• https://osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com/planet/replication/hour/000/104/277.osc.gz
• https://osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com/planet/replication/hour/000/104/277.state.txt
• https://osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com/planet/replication/hour/000/104/278.osc.gz
• https://osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com/planet/replication/hour/000/104/278.state.txt
• https://osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com/planet/replication/hour/000/104/279.osc.gz
• https://osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com/planet/replication/hour/000/104/279.state.txt

https://planet.openstreetmap.org/replication/hour/000/104/280.osc.gz seems ok.

There was a lag this morning generating theses files, it may be related.

[tomhughes]

I've synced the missing files.

[francois2metz]

Thanks, it worked.