Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow loading files fail (over 1 hour) #77

Open
GoogleCodeExporter opened this issue Mar 16, 2015 · 1 comment
Open

Slow loading files fail (over 1 hour) #77

GoogleCodeExporter opened this issue Mar 16, 2015 · 1 comment

Comments

@GoogleCodeExporter
Copy link
Contributor

What steps will reproduce the problem?
1. Load a large file, or load over a slow network
2. Watch the the tempfile table in the database
3. Watch the actual file on the storage node

What is the expected output? What do you see instead?
I expect my slow loading file to be loaded.

After an hour the tempfile entry disappears and the file on the storage node is 
deleted. My load thread completes but MogileFS never responds to create_close 
command (timed out after 10 mins for me) and the file is not present.

I'm no good with perl but I think the problem seems to be in the Delete worker:
    # dig up some temporary files to purge
    my $sto = Mgd::get_store();
    my $too_old = int($ENV{T_TEMPFILE_TOO_OLD} || 3600);
    my $tempfiles = $sto->old_tempfiles($too_old);
    return 0 unless $tempfiles && @$tempfiles;

Fixes
 - it would be best if the worker could check to see if the file is still been written to before culling it so I don't need to raise the value.
 - If I have to raise the value can we set it in the config file rather than an environmental variable?

Thanks!

Original issue reported on code.google.com by [email protected] on 20 Mar 2013 at 9:33

@GoogleCodeExporter
Copy link
Contributor Author

Just confirming starting mogilefsd with env T_TEMPFILE_TOO_OLD=12345 fixes the 
problem as expected.

Original comment by [email protected] on 22 Mar 2013 at 12:37

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant