Fix #1965: About gunicorn [CRITICAL] Worker Timeout

[skytoup]

Fixes #1965

Refer to your suggestions, use time.monotonic and os.utime to fix system time changes cause the woker timeout

[tilgovi]

Thanks! One small change and it should be great.

[tilgovi]

It would also be great to know if this solves your original issue!

[benoitc]

@skytoup does it solve your issue?

[benoitc]

the fchmod stuff still needs to be fixed. Can you change it?

[tilgovi]

Sorry, I approve too quickly. 😅

Still need to change self._tmp.name back to self._tmp.fileno().

[benoitc]

@skytoup we are doing a new release this week and it would be useful to include such change now, have you any questions about the proposed changes?

[skytoup]

@skytoup we are doing a new release this week and it would be useful to include such change now, have you any questions about the proposed changes?

I did the modified time test in the test environment and found no problem temporarily.

However, I observed the previous timeout error on my server and found that timeout still occurs every day.

Timeout may be caused by other reasons every day, but the reason has not been found yet.

I have changed a machine to deploy the server. Timeout does not occur every day. It may be caused by other factors such as the machine environment.

[skytoup]

I found the reason. Disk error due to system backup and affected gunicorn to modify the file information.

[tilgovi]

Do we still want to merge this PR? It makes sense if it works and is portable.

[benoitc]

@tilgovi the idea is still interresting. Provided we replace os.utime by os.fchmod since we need to work on the file descriptor.

[tilgovi]

@benoitc os.utime accepts a file descriptor.

[tkc17]

Hi,

I am facing the same issue, any plans when/if this will be merged? We are running a service during boot and time sync is causing premature/false timeouts.

[tkc17]

I can confirm that the fix works on 18.04, the test case which was failing always passed. It would be good to get this merged.

[tilgovi]

@benoitc can you take another look at this please? I think it's correct. The change to using utime instead of fchmod and checking st_mtime instead of st_ctime is fine and maybe more portable to Windows filesystems in the future. If the resolution concerns us, we can use the ns option of utime and check st_mtime_ns (though many filesystems won't be that accurate).

[kylemanna]

Ran into this issue on an embedded device that doesn't have a RTC so the time is always wrong on a start-up and jumps when it's connected to the Internet. Applied this patch and the issue seems resolved.

Any updates on getting this merged, would love to see it incorporated?

[flixr]

Bumping this, would be great to get this merged to finally resolve this issue.
See also #1709

[tilgovi]

Superseded by #3095.

[tilgovi]

Actually, #3095 does not use the monotonic trick. Feel free to reopen and rebase and we can merge this to fix the other issues.

[tilgovi]

I'll just merge, resolving conflicts. This looks goood.