Performance data not shown after loading output

[brucelaughlin]

Hi. I'm trying to determine an efficient setup for a project requiring multiple runs, so I'm running Opendrift in multiple configurations and seeking to compare the performance of each setup.

If I type "print(o)" or "o.performance" in my terminal after a run finishes but is still in memory, I get the expected performance data printed to the screen. However, if I load a previously stored Opendrift output file, for instance:

o = OceanDrift()
o.io_import_file(tracking_output_file)

or

o = opendrift.open(tracking_output_file),

I see the following when I type "print(o)" or "o.performance" :

Performance:
0 total time
0 configuration

Curiously, if I explore the file itself by typing in the unix terminal:

ncdump -h <output.nc> | less

I see much more information:

performance = "--------------------\nReader performance:\n--------------------\nroms native\n 0:09:30.7 total\n 0:00:00.3 preparing\n 0:09:05.7 reading\n 0:07:57.2 interpolation\n 0:00:03.5 interpolation_time\n 0:00:09.2 masking\n 0:00:15.3 rotating vectors\n--------------------\nglobal_landmask\n 0:00:05.7 total\n 0:00:00.2 preparing\n 0:00:05.4 reading\n 0:00:00.0 masking\n--------------------\nPerformance:\n 0 total time\n 45.0 configuration\n 2.6 preparing main loop\n 0.8 moving elements to ocean\n14:08.6 main loop\n 16.4 updating elements\n 0 cleaning up\n--------------------\n" ;

I'm trying to figure out what "io_import_file()" or "opendrift.open()" are doing, when it comes to reading performance data, but I am struggling to find the relevant code.

Bigger picture, is there something I could be doing incorrectly, either when saving or loading Opendrift output, that is leading to the performance data not being loaded for easy viewing (ie with print(o) or o.performance)?

Thanks,
Bruce

[knutfrode]

Hi,
I see that the total runtime is missing from performance stored in netCDF metadata, since it is written before the runtime is calculated.
I now fixed this, so that total runtime is also included: https://github.com/OpenDrift/opendrift/pull/1217/files
Thus you will get this if updating to the latest from Git. If using conda/mamba, you might need to wait for the next release in some days.

The performance dictionary is stored in memory after the run, but only exported to netCDF as a string.
Thus you might prefer to store the o.timing dictionary after the run, to access the data numerically. (This works also without updating to newest version).
Alternatively you may retrieve it from netCDF file e.g. using Xarray:
>>> print(xr.open_dataset(<netCDF-file>).performance)

Thus no need to use OpenDrift to re-import. Though if using OpenDrift, you should do

import opendrift
o = opendrift.open(<netCDF-file>)
print(o.performance)

[brucelaughlin]

Wonderful, thank you so much! The Xarray solution is a great option. Will also update using mamba when the new release is available. Appreciate the quick help and the updates to the software!