Skip to content
This repository has been archived by the owner on Aug 26, 2022. It is now read-only.

transfers/inbound: clean up zero-byte files #602

Open
adamdecaf opened this issue Oct 16, 2020 · 1 comment · Fixed by #612
Open

transfers/inbound: clean up zero-byte files #602

adamdecaf opened this issue Oct 16, 2020 · 1 comment · Fixed by #612
Assignees
Labels
enhancement New feature or request

Comments

@adamdecaf
Copy link
Member

PayGate Version: v0.9.0-dev

During periods of no activity some vendor systems create zero-byte files that PayGate needs to process and delete. These files will not properly parse as incoming/returned files and need to just be deleted, however zero-byte files might exist during the middle of a transfer from the ODFI's side. We don't want to delete these files right away, but after they've been sitting on the remote machine for a while (e.g. 10m by default).

Similar to KeepRemoteFiles: <boolean> I think we want a configuration to remove these outside of that option.

RemoveZeroByteFilesAfter: <duration> # default 10 minutes

These files would be removed as part of transfers/inbound's cleanup.go file: https://github.com/moov-io/paygate/blob/v0.9.0-dev1/pkg/transfers/inbound/cleanup.go

@adamdecaf adamdecaf added the enhancement New feature or request label Oct 16, 2020
@jmbrown412 jmbrown412 self-assigned this Oct 26, 2020
@jmbrown412 jmbrown412 linked a pull request Oct 27, 2020 that will close this issue
@adamdecaf
Copy link
Member Author

There's a bug with the current implementation of this. When we download a file onto the local filesystem the created and modified times are set to "now", but the file on the SFTP server would have been created "in the past." The logic expects to stat the local file and have it represent the old ctime.

We need to either write the ctime/mtime from the server (only viable on unix systems) or stat the remote file and decide then. I believe the FTP library we use doesn't support STAT, but our sftp library does. We could leverage just the SFTP side for now.

@adamdecaf adamdecaf reopened this Nov 23, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants