Too many files after moving from v1.2.0 to v2.0.0 and v2.2.0

[jkleckner]

In a large Scala repo with submodules, I tested a lot of combinations of sbt versions and sbt-assembly versions and found that moving from sbt-assembly version 1.2.0 to 2.0.0 or 2.2.0 resulted in the message when running sbt assembly:

Could not create file /Users/.../data/target/streams/test/_global/_global/compileOutputs/previousjava.io.IOException: Too many open files

As mentioned in #463 (comment), there is increased memory use, but that is expected with the amount of caching going on.

The sbt-assembly version 1.2.0 worked with all sbt versions up to and including 1.10.1
But moving to 2.0.0 started getting that error on my Mac (64GB Intel Sonomo 14.5).
As described here, running ulimit -n 10240 (the maximum on a Mac) allowed sbt assembly to complete.

• https://discussions.apple.com/thread/251000125?sortBy=best

However, sbt Test/assembly would not work and would give "too many open files".
This part of an upgrade process from Scala 2.11.12 and Java 8 so perhaps that is a contributing factor.
Any suggestions on what to try?