Announcing version 2.0 of the fastMRI dataset

[mmuckley]

Hello everyone,

We recently rolled out a new version of the fastMRI dataset. If you sign up at https://fastmri.med.nyu.edu/ you should receive an email with the new files. There are two main changes: 1) batching and 2) release of the brain test set.

Batching

A common thread we've heard from many users is that the fastMRI dataset is too big. It can take too long to download. In some cases, people don't even have enough disk space for it. These users would like to work with the data, but the sizes of the files (and the lack of options) has been prohibitive.

Now if you sign up at https://fastmri.med.nyu.edu/, you have the option of only downloading batches of data. The batches have been constructed such that each one is about 100 GB (compressed). If you're wondering what data is in each batched file, you can see them in the manifest files here.

Release of brain test set

To facilitate reproducibility, our recent Radiology: AI paper promised release of the brain test set. This seemed like a reasonable decision at the time, as not many users were using the brain leaderboard. As such, the fully-sampled brain test set is part of the new release. You can download it and reproduce your leaderboard numbers, as well as design new metrics.

Since the test set is now released, there is no point in continuing to allow submissions to the brain leaderboard. As such, as of today (January 30, 2023), the brain leaderboard has been disabled.

Please let us know if you have any thoughts or need further information.

[Ishak96]

Hello @mmuckley, how should we evaluate our model performances since there is no 'reconstruction_rss' in the test .h5 knee multicoil data?

Thank you.

[mmuckley]

Hello @Ishak96, this was previously covered in Discussion #293:

• Evaluation on the brain test set: You can now download the fully-sampled brain test set from https://fastmri.med.nyu.edu/ and run evaluations yourself. The website essentially runs the same code as in evaluate.py You may want to log the current leaderboard information if you want to make comparisons to the other models there.
• Evaluation on knee test set (multicoil and single coil): This will no longer be possible until NYU rebuilds the leaderboard infrastructure on their version of the website. Instead, you will have to use the knee validation set. You can split the knee validation set into two and use half as "val" for improving your models and the other half as "test".
• Historic leaderboards can be accessed via the archive at https://web.archive.org/web/20230324102125mp_/https://fastmri.org/leaderboards.

[Ishak96]

Thank you @mmuckley for your reply and for all your help.