Faster upload many small files and transfer api functionality

Small files will be uploaded in parallel now.
This also includes internal changes for transfer api functionality.
There is still no support for using the transfer api from the command line.