You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Multiple values at a given when_captured for a single sensor
Example data
Expand the query below. Note that this shows the sensor reporting 13 different lnd_7318c values for the same when_captured.
As far as @matschaffer knows this shouldn't happen unless possibly the device had 13 physical sensors. It should only have 1, and even if it had 13 we'd expect it to report them under different keys similar to how the multiple air sensors on the solarcast are reported via different prefixes.
from 2018
safecast=> select id, device_id, created_at, payload->>'when_captured' as when_captured, payload->>'lnd_7318c' as lnd_7318c from measurements where device_id = 4249659165 and payload->>'when_captured' like '2018-04-13T10:30%' limit 20;
safecast=> select id, device_id, created_at, payload->>'when_captured' as when_captured, payload->>'lnd_7318c' as lnd_7318c from measurements where device_id = 2152053642 and payload->>'when_captured' = '2020-04-07T15:59:56Z' order by created_at desc limit 50;
safecast=> select id, device_id, created_at, payload->>'when_captured' as when_captured, payload from measurements where device_id = 2152053642 and payload->>'when_captured' = '2020-04-07T15:59:56Z' order by created_at desc limit 20;
TTServe disk storage - can observe two different lnd_7318u values (33 & 48) for two payloads both captured/uploaded at 2020-04-07T15:59:56Z via device-tcp:172.31.14.80 to i-0c65ac97805549e0d
Expand the query below. Note how the when_captured value flips between 2020-05-15 and 2020-07-09 while service_uploaded is all 2020-07-09
safecast=> select id, device_id, payload->>'loc_lat' as loc_lat, payload->>'loc_lon' as loc_lon, created_at, payload->>'when_captured' as when_captured, payload->>'service_uploaded' as service_uploaded from measurements where device_id = 4249659165 and created_at between '2020-07-09 00:00:00' and '2020-07-09 23:00:00' order by created_at desc;
Multiple values at a given
when_capturedfor a single sensorExample data
Expand the query below. Note that this shows the sensor reporting 13 different
lnd_7318cvalues for the samewhen_captured.As far as @matschaffer knows this shouldn't happen unless possibly the device had 13 physical sensors. It should only have 1, and even if it had 13 we'd expect it to report them under different keys similar to how the multiple air sensors on the solarcast are reported via different prefixes.
from 2018
safecast=> select id, device_id, created_at, payload->>'when_captured' as when_captured, payload->>'lnd_7318c' as lnd_7318c from measurements where device_id = 4249659165 and payload->>'when_captured' like '2018-04-13T10:30%' limit 20;from 2020 (lnd_7318c field only)
safecast=> select id, device_id, created_at, payload->>'when_captured' as when_captured, payload->>'lnd_7318c' as lnd_7318c from measurements where device_id = 2152053642 and payload->>'when_captured' = '2020-04-07T15:59:56Z' order by created_at desc limit 50;from 2020 (full payloads)
safecast=> select id, device_id, created_at, payload->>'when_captured' as when_captured, payload from measurements where device_id = 2152053642 and payload->>'when_captured' = '2020-04-07T15:59:56Z' order by created_at desc limit 20;TTServe disk storage - can observe two different lnd_7318u values (33 & 48) for two payloads both captured/uploaded at 2020-04-07T15:59:56Z via device-tcp:172.31.14.80 to i-0c65ac97805549e0d
http://tt.safecast.org/device-log/2020-04-2152053642-30025.json
Possible cause
Measurements seem to be coming in without
when_capturedthen it's being set by ttserve https://github.com/Safecast/TTServe/blob/master/stamp.go#L264-L266 toNow()The
when_capturedmay be missing due to a compression glitch. https://safecast.slack.com/archives/C014YRM0E7R/p1595216845201100"Time Traveling" data series from a given device
Example data
Expand the query below. Note how the
when_capturedvalue flips between 2020-05-15 and 2020-07-09 whileservice_uploadedis all 2020-07-09safecast=> select id, device_id, payload->>'loc_lat' as loc_lat, payload->>'loc_lon' as loc_lon, created_at, payload->>'when_captured' as when_captured, payload->>'service_uploaded' as service_uploaded from measurements where device_id = 4249659165 and created_at between '2020-07-09 00:00:00' and '2020-07-09 23:00:00' order by created_at desc;Likely cause from @rayozzie on slack
the way that the solarcast/scnano code base works is that whenever the gps or network report a new timedate, the device saves:
All of this is fine except “seconds since boot” is derived by asking the nordic firmware how many ticks since boot, and dividing it, etc.
Unfortunately, I can see now that nordic does 32768 ticks per second, which means that it isn’t going to take long for us to wrap.
What is striking to me is that the difference between the May date and the July date is approximately 65K seconds, plus or minus.
I think there is a wrap issue here.
(airnote uses in64 to hold ms; solarcast uses uint32 to hold ‘ticks’)
The text was updated successfully, but these errors were encountered: