Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Create temporary files w/o race conditions whenever possible (fixes #1145) #1177

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

alexvong243f
Copy link
Collaborator

This PR could be too ambitious but adding 2 new functions is the cleanest way I can think of to fix #1145 once and for all. These functions cannot be private because they are needed by tests so we append __ to their names.

Alex Vong added 2 commits July 6, 2022 16:17
Try to create temporary file in the most secure and portable way
possible.

Partially fixes gnu-octave#1145.

* inst/make_temp_file__.m: New function.
* inst/@sym/sym.m: Use it.
* inst/private/python_ipc_system.m: Use it.
Try to create temporary directory in the most secure and portable way
possible.

Partially fixes gnu-octave#1145.

* inst/make_temp_dir__.m: New function.
* inst/@sym/function_handle.m: Use it.
fd = fopen (filename, 'r+');

else
error ('make_temp_file__: cannot create temp file: please enable java');
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this is "please enable java or use GNU Octave"?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, we should recommend Octave as well

@@ -953,7 +954,7 @@
%! syms x
%! y = 2*x;
%! a = 42;
%! myfile = tempname ();
%! [fd, myfile] = make_temp_file__ (tempdir (), 'octsympy-myfile-');
%! save (myfile, 'x', 'y', 'a')
%! clear x y a
%! load (myfile)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks scary: fd is an open file id...? not closed? does save open another fd on the same file...?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, should have closed the unused fd here

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but then this is no better than any other racy way... i.e., does this offer any benefit over tempname?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is better than tempname because tempname does not choose the filename and create the file in a single step (aka atomic operation). The fd is opened for convenience but it's not essential. The mktemp shell command similarly returns a filename instead of a fd, but it's still secure.

cmd = {'import tempfile'
'(tmpdir, prefix) = _ins'
'return tempfile.mkdtemp(dir=tmpdir, prefix=prefix)'};
path = pycall_sympy__ (cmd, tmpdir, prefix);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not a huge fan of having temp dir creation depend on Python...!

Is this really not do-able on Octave?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK octave has no function to create temporary directory.

If java is enabled, we can just use java. For *nix, we can use the shell command mktemp -d... So basically we are left with the case of octave without java running on windows. Do you know if all octave builds on windows include java? If it's the case, problem solved!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know if all octave builds on windows include java?

Java isn't included in the installer of Octave for Windows. Octave just tries to detect if a compatible JRE "happens" to be installed that it can use.
So, you shouldn't rely on being able to use Java.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Since cygwin / msys2 is part of octave in windows (as indicated in #1182), I think we can use the mktemp program.

%!error <Python exception: FileNotFoundError>
%! if (exist ('OCTAVE_VERSION', 'builtin'))
%! make_temp_dir__ ('/nonexistent', '');
%! end
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bikeshedding here but seems to me this should verify that /nonexistent is in fact non-existent after line 47.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. Debian-based distros use /nonexistent as a convention, but we should make sure it's indeed non-existent

@cbm755
Copy link
Collaborator

cbm755 commented Jul 6, 2022

I dunno about this...

These functions cannot be private because ...

This bothers me. I guess I'd rather they were private without __ on their names?

@alexvong243f
Copy link
Collaborator Author

I dunno about this...

These functions cannot be private because ...

This bothers me. I guess I'd rather they were private without __ on their names?

Is it possible to call a private function in bist? We don't need the __ trick If there's a way to do so w/o duplicating the code.

@mmuetzel
Copy link
Member

mmuetzel commented Aug 2, 2022

Is it possible to call a private function in bist?

That's bug #38776.

@alexvong243f alexvong243f changed the title Fixes #1145 (Create temporary files w/o race conditions whenever possible) [WIP] Fixes #1145 (Create temporary files w/o race conditions whenever possible) Sep 4, 2022
@alexvong243f alexvong243f changed the title [WIP] Fixes #1145 (Create temporary files w/o race conditions whenever possible) [WIP] Create temporary files w/o race conditions whenever possible (fixes #1145) Sep 7, 2022
@alexvong243f alexvong243f self-assigned this Sep 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create temporary files w/o race conditions whenever possible
3 participants