Crash when extracting ~1M particles

[delarosatrevin]

Describe the bug
@JuhaHuiskonen has reported this issue here: I2PC/scipion#2058

Log lines from the failed run with 1M particles:

03433: srun which relion_preprocess_mpi --i micrographs_00001-03460.star --part_star micrographs_00001-03460_particles.star --coord_dir "." --coord_suffix .coords.star --part_dir "." --extract --extract_size 400 --set_angpix 4.240000 --bg_radius 47 --invert_contrast --norm --scale 100 --white_dust 5.000 --black_dust 5.000
03434: === RELION MPI setup ===
03435: + Number of MPI processes = 20
03436: + Master (0) runs on host = r05c20.bullx
03437: + Slave 1 runs on host = r05c20.bullx
03438: + Slave 2 runs on host = r05c20.bullx
...
03455: + Slave 19 runs on host = r05c20.bullx
03456: =================
03457: + Setting pixel size in output STAR file to 4.24 Angstroms
03458: WARNING: You manually changed the pixel size by the --set_angpix option. You can no longer use Bayesian Polishing on the resulting particles.
03459: Extracting particles from the micrographs ...
03460: 12.68/12.68 min ............................................................~~(,_,">
03461: Joining metadata of all particles from 3351 micrographs in one STAR file...
03462: Written out STAR file with 1057682 particles in micrographs_00001-03460_particles.star
03463: The new pixel size of the extracted particles are 16.96 Angstrom/pixel.
03464: Done preprocessing!
03465: FINISHED: extractMicrographListStep, step 1
03466: 2019-10-22 16:12:28.089902
03467: Traceback (most recent call last):
03468: File "/projappl/project_2001566/apps/scipion/2.0/pyworkflow/protocol/protocol.py", line 186, in run
03469: self._run()
03470: File "/projappl/project_2001566/apps/scipion/2.0/pyworkflow/protocol/protocol.py", line 1289, in _run
03471: self._runSteps(startIndex)
03472: File "/projappl/project_2001566/apps/scipion/2.0/pyworkflow/protocol/protocol.py", line 1161, in _runSteps
03473: self._stepsCheckSecs)
03474: File "/projappl/project_2001566/apps/scipion/2.0/pyworkflow/protocol/executor.py", line 133, in runSteps
03475: stepsCheckCallback()
03476: File "/projappl/project_2001566/apps/scipion/2.0/pyworkflow/em/protocol/protocol_particles.py", line 320, in _stepsCheck
03477: self._checkNewOutput()
03478: File "/projappl/project_2001566/apps/scipion/2.0/pyworkflow/em/protocol/protocol_particles.py", line 527, in _checkNewOutput
03479: self._updateOutputPartSet(newDone, streamMode)
03480: File "/projappl/project_2001566/apps/scipion/2.0/pyworkflow/em/protocol/protocol_particles.py", line 581, in _updateOutputPartSet
03481: self.readPartsFromMics(micList, outputParts)
03482: File "/projappl/project_2001566/apps/scipion/2.0/software/lib/python2.7/site-packages/relion/protocols/protocol_extract_particles.py", line 305, in readPartsFromMics
03483: sortByLabel=md.RLN_MICROGRAPH_NAME):
03484: File "/projappl/project_2001566/apps/scipion/2.0/pyworkflow/em/metadata/utils.py", line 97, in iterRows
03485: md.sort(sortByLabel)
03486: XmippError: Error code: 21 message: no such table: MDTable_2
03487: Sqlite query: INSERT INTO MDTable_3( "rlnCoordinateX", "rlnCoordinateY", "rlnImageName", "rlnMicrographName", "rlnMagnification", "rlnVoltage", "rlnDefocusU", "rlnDefocusV", "rlnDefocusAngle", "rlnSphericalAberration", "rlnBfactor", "rlnCtfScalefactor", "rlnPhaseShift", "rlnAmplitudeContrast", "rlnOriginX", "rlnOriginY", "rlnDetectorPixelSize") SELECT "rlnCoordinateX", "rlnCoordinateY", "rlnImageName", "rlnMicrographName", "rlnMagnification", "rlnVoltage", "rlnDefocusU", "rlnDefocusV", "rlnDefocusAngle", "rlnSphericalAberration", "rlnBfactor", "rlnCtfScalefactor", "rlnPhaseShift", "rlnAmplitudeContrast", "rlnOriginX", "rlnOriginY", "rlnDetectorPixelSize" FROM MDTable_2
03488: ------------------- PROTOCOL FAILED (DONE 1/2)

From the error log it seems like a bug in the Xmipp metadata class, when trying to execute the line:

md.sort(sortByLabel)  # while iterating thrown the star file rows

We might consider to replace the use of Xmipp's metadata class here.

[delarosatrevin]

Will be addressed in a more general issue