Cannot upload bigger file size or larger number of files

[goldenratio]

Stacktrace,

events.js:141
      throw er; // Unhandled 'error' event
      ^
 Error: Failure
    at SFTPStream._transform (F:\gulp-ssh\node_modules\ssh2\node_modules\ssh2-streams\lib\sftp.js:384:27)
    at SFTPStream.Transform._read (_stream_transform.js:167:10)
    at SFTPStream._read (F:\node_modules\gulp-ssh\node_modules\ssh2\node_modules\ssh2-streams\lib\sftp.js:170:15)
    at SFTPStream.Transform._write (_stream_transform.js:155:12)
    at doWrite (_stream_writable.js:300:12)
    at writeOrBuffer (_stream_writable.js:286:5)
    at SFTPStream.Writable.write (_stream_writable.js:214:11)
    at Channel.ondata (_stream_readable.js:542:20)
    at emitOne (events.js:77:13)
    at Channel.emit (events.js:169:7)

Related to this issue: teambition/gulp-ssh#51
Summary:

1. When trying to upload 100> files, I am getting this error
2. When I try to upload one file of size 30 MB + , I am getting this error.

[mscdex]

For the one file scenario, can you set debug: console.log in your ssh config object and post the resulting console output?

[goldenratio]

Not able to reproduce second scenario by myself.
But first is still valid.. log for the last few files

[18:25:21] Writing '/var/www/html/jenkins/url-gen-tool/latest/lib/@angular/compi
ler/src/util.d.ts'
DEBUG[SFTP]: Outgoing: Writing OPEN
DEBUG: Outgoing: Writing CHANNEL_DATA (0)
DEBUG: Parser: IN_PACKETBEFORE (expecting 16)
DEBUG: Parser: IN_PACKET
DEBUG: Parser: Decrypting
DEBUG: Parser: pktLen:44,padLen:17,remainLen:32
DEBUG: Parser: IN_PACKETDATA
DEBUG: Parser: Decrypting
DEBUG: Parser: HMAC size:32
DEBUG: Parser: IN_PACKETDATAVERIFY
DEBUG: Parser: Verifying MAC
DEBUG: Parser: IN_PACKETDATAVERIFY (Valid HMAC)
DEBUG: Parser: IN_PACKETDATAAFTER, packet: CHANNEL_DATA (0)
DEBUG[SFTP]: Parser: Response: HANDLE
DEBUG[SFTP]: Outgoing: Writing WRITE
DEBUG: Outgoing: Writing CHANNEL_DATA (0)
DEBUG: Parser: IN_PACKETBEFORE (expecting 16)
DEBUG: Parser: IN_PACKET
DEBUG: Parser: Decrypting
DEBUG: Parser: pktLen:44,padLen:6,remainLen:32
DEBUG: Parser: IN_PACKETDATA
DEBUG: Parser: Decrypting
DEBUG: Parser: HMAC size:32
DEBUG: Parser: IN_PACKETDATAVERIFY
DEBUG: Parser: Verifying MAC
DEBUG: Parser: IN_PACKETDATAVERIFY (Valid HMAC)
DEBUG: Parser: IN_PACKETDATAAFTER, packet: CHANNEL_DATA (0)
DEBUG[SFTP]: Parser: Response: STATUS
[18:25:21] Finished writing '/var/www/html/jenkins/url-gen-tool/latest/lib/@angu
lar/compiler/src/util.d.ts'
[18:25:21] Preparing to write "/var/www/html/jenkins/url-gen-tool/latest/lib/@an
gular/compiler/src/util.js"
DEBUG[SFTP]: Outgoing: Writing STAT
DEBUG: Outgoing: Writing CHANNEL_DATA (0)
DEBUG: Parser: IN_PACKETBEFORE (expecting 16)
DEBUG: Parser: IN_PACKET
DEBUG: Parser: Decrypting
DEBUG: Parser: pktLen:60,padLen:9,remainLen:48
DEBUG: Parser: IN_PACKETDATA
DEBUG: Parser: Decrypting
DEBUG: Parser: HMAC size:32
DEBUG: Parser: IN_PACKETDATAVERIFY
DEBUG: Parser: Verifying MAC
DEBUG: Parser: IN_PACKETDATAVERIFY (Valid HMAC)
DEBUG: Parser: IN_PACKETDATAAFTER, packet: CHANNEL_DATA (0)
DEBUG[SFTP]: Parser: Response: ATTRS
[18:25:21] Writing '/var/www/html/jenkins/url-gen-tool/latest/lib/@angular/compi
ler/src/util.js'
DEBUG[SFTP]: Outgoing: Writing OPEN
DEBUG: Outgoing: Writing CHANNEL_DATA (0)
DEBUG: Parser: IN_PACKETBEFORE (expecting 16)
DEBUG: Parser: IN_PACKET
DEBUG: Parser: Decrypting
DEBUG: Parser: pktLen:44,padLen:6,remainLen:32
DEBUG: Parser: IN_PACKETDATA
DEBUG: Parser: Decrypting
DEBUG: Parser: HMAC size:32
DEBUG: Parser: IN_PACKETDATAVERIFY
DEBUG: Parser: Verifying MAC
DEBUG: Parser: IN_PACKETDATAVERIFY (Valid HMAC)
DEBUG: Parser: IN_PACKETDATAAFTER, packet: CHANNEL_DATA (0)
DEBUG[SFTP]: Parser: Response: STATUS
DEBUG: Outgoing: Writing DISCONNECT (BY_APPLICATION)
Deploy Error: Error in plugin 'gulp-ssh'
Message:
    Failure
Details:
    code: 4
    lang:
DEBUG: Outgoing: Writing CHANNEL_EOF (0)
DEBUG: Outgoing: Writing CHANNEL_CLOSE (0)

[mscdex]

It could be due to limits in place on the server. I'm not sure why, but it appears that gulp-ssh is not automatically closing files after they are written. That could leave resources open on the server (e.g. open file descriptors) and causing a potential limit issue? What also makes me think this may be the case is that the error happens when the next file is opened for writing on the server and not during a write or other request.

On an unrelated note, fastPut() will transfer (local) files faster than normal streaming due to parallel writes (vs sequential writes when streaming). The gulp-ssh authors may want to look into that at some point.

[goldenratio]

ok thanks for input.