WIP: Concurrent loading of ensemble

[wouterjdb]

Work towards addressing #11

[berland]

Good start. Next up for grabs should be ScratchEnsemble.load_file() which is where there is really potential for speedup.

[wouterjdb]

Good start. Next up for grabs should be ScratchEnsemble.load_file() which is where there is really potential for speedup.

@berland, something like this?

[wouterjdb]

Managed to get a few tests further now.... now I got a segmentation error 😱 when running the test_ensemble_aggregations...... to be continued...

[wouterjdb]

@berland OK, first working version of the code. We already had a chat about some of the downsides of the code as it is per today, but maybe you can have a look and identify what really needs to be done before moving towards a merge.

[wouterjdb]

This looks like a more elegant way of writing it - getting rid of all the boilerplate code! 👍

There is another use of concurrent in the same file - did you consider the same approach there? Or was that any different?

[wouterjdb]

Why did you use dummy arguments?

[wouterjdb]

I updated those lines in the meanwhile by the way - let me know if that was not what you had in mind.

And I see that for the other usage there is not this additional wrapper (there is one, but it is different.). Though, we might consider changing that to the submit() method as well - just to use the same thing.

[berland]

dummydir was just for laziness. Need to find a pretty solution, but it is only used for logging.

[berland]

I had a look at how to remove the _load_smry() wrapper as well, but that is not that easy. But more importantly, it uncovers a flaw in the usage pattern - this way of working, calling load_ functions from the Ensemble object to update the realization object does not scale. Check the comment in #11

[berland]

This PR is replaced by #106