Skip to content

Parallel directory copy for distributed filesystems

License

Notifications You must be signed in to change notification settings

CEA-LIST/parsync

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

parsync

Parsync lets you recursively copy missing files to a directory.

This is equivalent to cp -r --ignore-existing SRC DST but using concurrent copies to accelerate the process.

Example

Copy the content of the current directory:

parsync * /dest

Copy with 2 threads:

parsync -p 2 src dest

Copy only jpeg files and skip a specific directory, note the usage of python regex syntax:

parsync --filter='\.jpg$' --exclude='/somedir/' src dest

Usage

usage: parsync [-h] [--procs PROCS] [--filter [FILTER ...]] [--exclude [EXCLUDE ...]] [--version] SRC [SRC ...] DST

Recursively copy missing files to a directory.

This is equivalent to: cp -r --ignore-existing SRC DST

Note: matching patterns use python regex syntax.

positional arguments:
  SRC                   source files and directories
  DST                   destination directory

options:
  -h, --help            show this help message and exit
  --procs PROCS, -p PROCS
                        maximum number of concurrent copies
  --filter [FILTER ...], -f [FILTER ...]
                        pattern to filter source files
  --exclude [EXCLUDE ...], -e [EXCLUDE ...]
                        pattern to exclude source files
  --version, -v         show program's version number and exit