fetch fix

[tttp]

so simplify fetch

• get the source file from trust
• check if correct (eg not empty)
• db.clear
• loop over every item and write them (no need to check, it should be there already)

and voila, we have a tiny (<0.3 sec) between db.clear and the end of the loop where a lookup can happen AND possibly not be paused until the loop is finished AND be an email that should be there AND that hasn't been already

it's an acceptable limitation

[1v4n4]

@tttp
The problem:
Level DB opens on the top Level
(const db = new Level<string, Record>(process.env.DB_PATH || './emails.db', { valueEncoding: 'json' });) This line can't be for example inside of Cron job, or any other function.
If it is on top, and the job is using it, nvm whether it is running once or 100 times a day, the database is always opened just for this one process, and no other can access it.

This brings me back to the 'database copy' solution #7 (comment)

It is 73263 hashes at the moment