Improve error message about unsupported commands on SSH destination (was: SSH backup to Hetzner Storage Box: "test" command not available)

[philipkbh]

I am trying to set up a backup profile for my Hetzner Storage Box under a sub-account. The sub-account is accessible from outside the Hetzner network and SSH access is enabled. I have also installed the public SSH key on this sub-account so that I can use SSH without entering my password.

The problem I'm running into is that backintime tells me that it Couldn't create remote path.: /home. In the config tab I set Path to /home.

I think it's due to the structure of a Hetzner Storage Box sub-account, because when I SSH into that sub-account, the starting directory is the /home folder of that sub-account. When you create a sub-account for your Hetzner Storage Box you define a base directory and this base directory will be the /home folder. Outside this /home folder (under /) a subaccount has no permissions to do anything.

Running Back in Time 1.4.3 under Pop!_OS 22.04 LTS (Linux-6.8.0-76060800daily20240311-generic-x86_64-with-glibc2.35).

[buhtz]

Reminds me of #1674. Maybe this is a similar problem?

EDIT: Sorry, I realized that the linked issue is written in German. The issue was about a Synology NAS device. To my understanding the difference was between using ssh and sshfs. Both are used by BIT with the assumption that they "start" in the same folder after logging in. But on a Synology NAS it seems to be different.
I never used sshfs directly on shell. But maybe you are able to test this hypothesis somehow trying ssh and sshfs on your machine?
One of my other team mates is more into this topic and will catch up to you.

[philipkbh]

I tried a different approach by mounting the Storage Box sub-account via sshfs on my PC and doing a backup in Local mode to this mounted path.
When I try to save this configuration, I get the following error:

Destination filesystem for /mnt/backup is an sshfs-mounted share. sshfs doesn't support hard-links. Please use mode 'SSH' instead.

[buhtz]

Hello Phillip,
currently I see no way how I can help you.

We have to wait for a response from my teammate "aryoda". He is deeper into that topic.

Hetzner itself refused to provide us with a test account in the past. They also won't be helpful.

Don't hesitate to warm up the Issue if you don't get any response in some weeks.

Christian

[buhtz]

Did you checked #1674 ? Did this help you? Is your Issue a duplicate of it?

[philipkbh]

I didn't investigate further because I switched to restic with resticprofile.

[buhtz]

Thank you for reporting back.

[rwgroenenberg]

I am facing the same issue, with a Hetzner account:
The debug output for checking the remote folder is:

jul 15 15:10:41 penacho backintime[445693]: Hetzner(6) :: DEBUG: [common/sshtools.py:608 SSH.checkRemoteFolder] Check remote folder
jul 15 15:10:41 penacho backintime[445693]: Hetzner(6) :: DEBUG: [common/sshtools.py:635 SSH.checkRemoteFolder] Call command: ssh -o ServerAliveInterval=240 -o LogLevel=Error -o IdentityFile=/home/robert/.ssh/id_rsa -p 23 [email protected] d=0;test -e "./" || d=1;test $d -eq 1 && mkdir "./"; err=$?;test $d -eq 1 && exit $err;test -d "./" || exit 11;test -w "./" || exit 12;test -x "./" || exit 13;exit 20
jul 15 15:10:41 penacho backintime[445693]: Hetzner(6) :: DEBUG: [common/sshtools.py:645 SSH.checkRemoteFolder] Command returncode: 8

Trimming down the ssh command to just test already fails due to the restricted shell:

$ ssh -p 23 [email protected] test -e "./"
Command not found. Use 'help' to get a list of available commands.

[rwgroenenberg]

The list of available commands is just:

/home > help
+-----------------------------------------------------------------------------+
| The following commands are available:                                       |
|   ls                                  list directory content                |
|   tree                                list directory content                |
|   cd                                  change current working directory      |
|   pwd                                 show current working directory        |
|   mkdir                               create new directory                  |
|   rmdir                               delete directory                      |
|   du                                  disk usage of files/directories       |
|   df                                  show disk usage                       |
|   dd                                  read and write files                  |
|   cat                                 output file content                   |
|   touch                               create new file                       |
|   cp                                  copy files/directories                |
|   rm                                  delete files/directories              |
|   unlink                              delete file/directory                 |
|   mv                                  move files/directories                |
|   chmod                               change file/directory permissions     |
|   md5|sha1|sha256|sha512              create hash sum of file               |
|   md5sum|sha1sum|sha256sum|sha512sum  create hash sum of file               |
|   head                                show first lines of file              |
|   tail                                show last lines of file               |
|   grep                                search for specific string in files   |
|   stat                                stat files/directory                  |
|                                                                             |
| Available as server side backend:                                           |
|   borg                                                                      |
|   rsync                                                                     |
|   scp                                                                       |
|   sftp                                                                      |
|   rclone serve restic --stdio                                               |
|                                                                             |
| Please note that this is only a restricted shell which do not               |
| support shell features like redirects or pipes.                             |
|                                                                             |
| You can find more information in our Docs:                                  |
|   https://docs.hetzner.com/robot/storage-box/                               |
+-----------------------------------------------------------------------------+

I'm afraid this may be a bit too restricted to get BIT working :-/

[buhtz]

For my own and because Hetzner once reject my requests, I am not much motivated investing resources into this. But I am also open for suggestions how to solve, RPs and test scenarios. The "test -e" command just tests if the folder does exist. Maybe "stat" could be used for this instead. But I am unable to test this because Hetzner won't provide us an account.

They provide rsync, restic, borg on their servers. If they wont to support Back In Time they do have enough man power doing so. We are open for PRs.

[rwgroenenberg]

Hetzner has confirmed that support for BackInTime is on their roadmap, but can't say when it will be available...
I'll see if I can get it to work with mounting the storage box locally (in the mean time I'm using Borg+Vorta)

[buhtz]

Great to here that Hetzner will jump in. Let's see if they suggest the use of another command then test or if they add test to their platform.

[buhtz]

Can you please do me a favour and test this command (replace login credentials):

ssh -o ServerAliveInterval=240 -o LogLevel=Error -o IdentityFile=/home/robert/.ssh/id_rsa -p 23 [email protected] d=0;[ -e "./" ] || d=1

I replaced test -e with [ -e ].

Can you please also provide the output of echo $SHELL after logging into the Hetzern shell.

[rwgroenenberg]

Unfortunately their shell is _really_ restricted. The suggested command doesn't work:

/home > d=0;[ -e "./" ] || d=1 Command not found. Use 'help' to get a list of available commands.

Even a simple a simple assignment fails:

…

/home > d=0 Command not found. Use 'help' to get a list of available commands. On 26-08-2024 13:33, buhtz wrote: Can you please do me a favour and test this command (replace login credentials): |ssh -o ServerAliveInterval=240 -o LogLevel=Error -o IdentityFile=/home/robert/.ssh/id_rsa -p 23 ***@***.*** d=0;[ -e "./" ] || d=1 | I replaced |test -e| with |[ -e ]|. — Reply to this email directly, view it on GitHub <#1745 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEY2JRDZZEYQCKPHXLQ2EM3ZTMHBRAVCNFSM6AAAAABIURJVIWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBZHE4DQMJRHA>. You are receiving this because you commented.Web Bug from https://github.com/notifications/beacon/AEY2JRFEXXIX557ELUJRF7LZTMHBRA5CNFSM6AAAAABIURJVIWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTUJV6PRM.gifMessage ID: ***@***.***> [ { ***@***.***": "http://schema.org", ***@***.***": "EmailMessage", "potentialAction": { ***@***.***": "ViewAction", "target": "#1745 (comment)", "url": "#1745 (comment)", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { ***@***.***": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

[buhtz]

OK, lets try stat

ssh -o ServerAliveInterval=240 -o LogLevel=Error -o IdentityFile=/home/robert/.ssh/id_rsa -p 23 [email protected] d=0;stat "./" || d=1

This should work. But the question is if we should use stat by default, or just as a fallback if test is not available.

[buhtz]

Hello Robert,
can you please test the last command I suggested on your Hetzer Box please?

[rwgroenenberg]

It will not work as already the variable assignment 'd=0' is not supported.

[buhtz]

It will not work as already the variable assignment 'd=0' is not supported.

Mhm... It worked on my bash. So this is another Hetzer-restriction?

Do you have any other ideas? Or can we just say Hetzer is not supported and improving the error message.

[buhtz]

Suggested approach

Do not support restricted systems like Hetzer is using. Improve the error message to make it more clear to the user what is going on. Maybe do a previous check if commands are available or not. Also add a FAQ entry about known error messages in context of Hetzner boxes.

[buhtz]

• Incompatible remote shell (was: Can't create folder (because of fish shell on remote)) #1905