-
Notifications
You must be signed in to change notification settings - Fork 359
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Parallel rpmfc thread-safety - libmagic may fork helpers #756
Comments
For the record, there have been some reports of builds hanging during classification on systems with large number of CPUs using file 5.36 which where fixed by updating to 5.37. Another workaround is to set %_smp_build_nthreads to a smaller number, ultimately 1 to make the whole classification run in serial as it always used to. |
Sadly, I am seeing this when rpm is linked against file 5.37 as well. Here's a couple tracebacks obtained by attaching gdb to running (hanging) rpm processes: 0x00007fc5a9ee22a1 in ?? () from /usr/lib/x86_64-linux-gnu/libgomp.so.1 0x00007f2220f000ee in waitpid () from /home/alexander/development/poky/build-x86-64/tmp/sysroots-uninative/x86_64-linux/lib/libc.so.6 Process tree shows: rpmbuild.real─┬─2*[bzip2] rpmdeps.real─┬─29*[xz] |
TBH, this seems like a serious enough issue in 4.15.0 release, that parallel file classification should be simply disabled for the time being, with a 4.15.0.1 issued to include that. I don't want to set %_smp_build_nthreads to 1, as Yocto project does benefit greatly from parallel binary package creation. |
TBH, the thought of reverting the whole damn threading business has crossed my mind more than once by now, it is (quite expectedly) proving painful. I've resisted the temptation so far... The thing is that this hasn't been an issue on Fedora at all, AFAIK. If we just disable without understanding where the difference comes from, it'll still be there the next time we try enabling it. |
Anyway, there will be a bugfix release to 4.15.x soonish, there's a handful of fairly critical things that needs addressing. I mean, it is a .0 release... |
Let's work together and sort the problem. If you would like me to run additional diagnostics, I'd be happy to help. |
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 (From OE-Core rev: 870c4bfc1c699543dcbd712f97deceb206976cd5) Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 (From OE-Core rev: 3c4a33a24891855fb60934399b3630ee03bcd7b0) Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 (From OE-Core rev: 3c4a33a24891855fb60934399b3630ee03bcd7b0) Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 (From OE-Core rev: 3b35000667cd6084ca14bc04ada55ec01752182f) Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 (From OE-Core rev: 3b35000667cd6084ca14bc04ada55ec01752182f) Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 (From OE-Core rev: ea287491ad1f2e0c92ba9ded33b59892790d00d9) Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 (From OE-Core rev: ea287491ad1f2e0c92ba9ded33b59892790d00d9) Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 (From OE-Core rev: f0526ccf0d244e3ff995604f68c51199154e16ad) Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 (From OE-Core rev: f0526ccf0d244e3ff995604f68c51199154e16ad) Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 (From OE-Core rev: f0526ccf0d244e3ff995604f68c51199154e16ad) Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 (From OE-Core rev: 8514da3601c5631cf9b05a627cac6204519b8d57) Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 (From OE-Core rev: 8514da3601c5631cf9b05a627cac6204519b8d57) Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 (From OE-Core rev: 8514da3601c5631cf9b05a627cac6204519b8d57) Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 (From OE-Core rev: 8514da3601c5631cf9b05a627cac6204519b8d57) Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 (From OE-Core rev: 8514da3601c5631cf9b05a627cac6204519b8d57) Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
I dug a bit deeper: the issue happens when libmagic calls out to external executables, which isn't implemented in a thread-safe way, and locks up. Enabling internal compression support (done via linking to libraries) makes the issue go away. |
Yup, at least one such issue has just been addressed here: file/file@81f15c2b |
Based on internal testing, the above seems to have cured the remaining cases, so considering closed. |
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Armin Kuster <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 (From OE-Core rev: 67257ca87c6fa8e6050a20ecea50daf834c7e869) Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
Drop patches that were merged upstream. 0001-mono-find-provides-requires-do-not-use-monodis-from-.patch modifies a file that was removed upstream. Add a 0001-rpmfc.c-do-not-run-file-classification-in-parallel.patch as unfortunately the new parallel file classification feature from upstream trips over somewhere in libmagic when inspected files are compressed: rpm-software-management/rpm#756 (From OE-Core rev: 67257ca87c6fa8e6050a20ecea50daf834c7e869) Signed-off-by: Alexander Kanavin <[email protected]> Signed-off-by: Richard Purdie <[email protected]>
When using MAGIC_COMPRESS (as rpmfc does), libmagic may fork helper executables to deal with decompressing files. Since commit 41f0e21 we are running the classifier in threads, and mixing fork() and threads tends to mean trouble.
Filing this ticket as a reminder to look into and deal with it one way or the other: use of MAGIC_COMPRESS dates back to 2009 (commit c3770c9), predating the current customizable classifier and availability of path based rules. Looking into compressed files was necessary for at least font provide extraction but we could trivially use paths for that now instead.
The text was updated successfully, but these errors were encountered: