Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

'#' symbol ignored by when reading real path on 16.08 and 18.03 #250

Open
khurtado opened this issue May 15, 2019 · 2 comments · May be fixed by #251
Open

'#' symbol ignored by when reading real path on 16.08 and 18.03 #250

khurtado opened this issue May 15, 2019 · 2 comments · May be fixed by #251

Comments

@khurtado
Copy link

khurtado commented May 15, 2019 •
edited
Loading

Hello,

I'm trying to use the --volume option on Blue Waters, with shifter v16.08 which works fine, except when I have # in the directory. The same thing happens on a test machine with v18.03. None of the parent directories are symbolic links.

I tried escaping '#', ie.: '#` but that didn't work either. Do you have a workaround for it? I'm doing this as part of an automated process and renaming the directory names created is not a trivial thing to do.

Any ideas or suggestions?

For example:

# Blue Waters
> pwd
/mnt/a/u/sciteam/hurtadoa
> mkdir -p test1#1
> shifter --image=docker:centos:latest --volume=/mnt/a/u/sciteam/hurtadoa/test1#1:/reana --  cat /etc/centos-release
shifter_realpath: failed to lstat /var/udiMount/mnt/a/u/sciteam/hurtadoa/test11
FAILED to find real path for volume "from": /mnt/a/u/sciteam/hurtadoa/test11
FAILED to setup user-requested mounts.
FAILED to setup image.
> module avail shifter

------------------------------------------------------------- /opt/cray/gem/modulefiles -------------------------------------------------------------
shifter/16.08.3-1.0502.8871-cray_gem shifter/16.08.3-2(default)

# Test Machine
$ shifter --image=python:2.7 --volume=/home/khurtado/test1#1:/reana-- cat /etc/issue
shifter_realpath: failed to lstat /var/udiMount/home/khurtado/test11
FAILED to find real path for volume "from": /home/khurtado/test11
FAILED to setup user-requested mounts.
FAILED to setup image.
$ rpm -qa | grep shifter
shifter-imagegw-18.03.0-1.nersc.el7.x86_64
shifter-runtime-18.03.0-1.nersc.el7.x86_64
shifter-18.03.0-1.nersc.el7.x86_64
@craigsteffen
Copy link

I'm the User Support person who took the Blue Waters ticket about this issue.

I presume that you're stripping characters from filenames for security reasons. I agree and understand that. However, I think that SILENTLY stripping characters from user input and then operating on that input as if it's valid is very bad and itself very dangerous interface design.

If shifter encounters input that is invalid or a security risk, shifter should stop, flag the input as invalid, complain very specifically to the user what the problem is with the user input, and then abort and exit. It should NOT blindly try to sanitize the input (thus making it very definitely invalid) and then continue to try to operate with now-modified inputs.

khurtado added a commit to khurtado/shifter that referenced this issue May 15, 2019
@khurtado
Fixes NERSC#250
3384437 
Allow '#' as part of the string name patterns.
@khurtado khurtado linked a pull request May 15, 2019 that will close this issue
Open
@khurtado
Copy link
Author

khurtado commented May 15, 2019 •
edited
Loading

I made a PR allowing '#', in case shifter developers consider this is acceptable.
My use case is for mounting a specific path with '#' characters on it that are passed by the HTCondor GT2/5 grid universe as part of the GlobalJobId variable (so, Torque is submitting grid universe condor pilots that are joining an existing condor pool), so I can't really change the pattern.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fixes #250 khurtado/shifter
2 participants
@craigsteffen @khurtado