1.0.0.6
- Update 3rd party libraries
- Use common-crawl 2018-43 by default
- Write accumulated mimetypes to a separate text-file after each index-file
- Add some support for detecting duplicate files and moving them out of the list to not re-process the same file over and over by the post-processing steps
- Some small adjustments for behavior changes in Java 11