SIAP metrics do not support gaps in detections

[JulienRoussel77]

🐛 Bug Report

Description

When no object is detected at a given timestamp, there is no track for this timestamp even if there is a groundtruth. This should be supported by the Accuracy metrics, but it produces an error.

Current Behavior

When there is a gap in the detections, the function accuracy_at_time tries to get a non existing timestamp key from the tracks and returns an error.

{
	"name": "IndexError",
	"message": "timestamp not found in states",
	"stack": "---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[13], line 1
----> 1 metrics = metric_manager.generate_metrics()

File ~/Library/Caches/pypoetry/virtualenvs/cosita-K1LlpWRX-py3.12/lib/python3.12/site-packages/stonesoup/metricgenerator/manager.py:91, in MultiManager.generate_metrics(self)
     89     print(\"appel self.associate_tracks(generator)\")
     90     self.associate_tracks(generator)
---> 91 metric_list = generator.compute_metric(self)
     92 if not isinstance(metric_list, list):  # If not already a list, force it to be one
     93     metric_list = [metric_list]

File ~/Library/Caches/pypoetry/virtualenvs/cosita-K1LlpWRX-py3.12/lib/python3.12/site-packages/stonesoup/metricgenerator/tracktotruthmetrics.py:114, in SIAPMetrics.compute_metric(self, manager, **kwargs)
    112 Nt = self.num_tracks_at_time(tracks, timestamp)
    113 N_sum += Nt
--> 114 PAt = self.accuracy_at_time(manager, timestamp, self.position_measure)
    115 PA_sum += PAt
    116 VAt = self.accuracy_at_time(manager, timestamp, self.velocity_measure)

File ~/Library/Caches/pypoetry/virtualenvs/cosita-K1LlpWRX-py3.12/lib/python3.12/site-packages/stonesoup/metricgenerator/tracktotruthmetrics.py:327, in SIAPMetrics.accuracy_at_time(self, manager, timestamp, measure)
    325 for association in associations:
    326     truth, track = self.truth_track_from_association(association)
--> 327     error_sum += measure(track[timestamp], truth[timestamp])
    328 return error_sum

File ~/Library/Caches/pypoetry/virtualenvs/cosita-K1LlpWRX-py3.12/lib/python3.12/site-packages/stonesoup/types/state.py:307, in StateMutableSequence.__getitem__(self, index)
    305             return state
    306     else:
--> 307         raise IndexError('timestamp not found in states')
    308 elif isinstance(index, slice):
    309     return StateMutableSequence(self.states.__getitem__(index))

IndexError: timestamp not found in states"
}

Expected Behavior

The accuracy metrics should ignore timestamps where the track is not defined. Additionnaly the function num_tracks_at_time should take into account timestamps when the track is associated to a truth in terms of time interval, but not defined at the given timestamp.

---

Steps to Reproduce

Provide the steps to reproduce the problem:

1. Download the notebook https://stonesoup.readthedocs.io/en/v1.3/auto_examples/metrics/Metrics.html
2. Replace the detector, suppressing a few detections:

class PartialPlatformDetectionSimulator(PlatformDetectionSimulator):
    @BufferedGenerator.generator_method
    def detections_gen(self):
        for time, truths in self.groundtruth:
            print(time, time.second % 10)
            if time.second % 10 == 0:
                continue
            # Move platforms and carry out sensor actions.
            for platform in self.platforms:
                platform.move(time)
                for sensor in platform.sensors:
                    sensor.act(time)

            # Make measurements from sensors
            for platform in self.platforms:
                for sensor in platform.sensors:
                    truths_to_be_measured = truths.union(self.platforms) - {platform}
                    detections = sensor.measure(truths_to_be_measured)
                    yield time, detections

detector = PartialPlatformDetectionSimulator(ground_truth_gen, platforms=[platform])

---

Additional Information

Environment

• Operating System: macOS Sonoma 14
• Stonesoup Version: 1.4