Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use --io-manager=native in lib-suite on Windows #10114

Merged
merged 1 commit into from
Jun 25, 2024

Conversation

jasagredo
Copy link
Collaborator

Closes #9571

@jasagredo jasagredo force-pushed the js/io-manager-lib-suite-windows branch from d799936 to 416255f Compare June 14, 2024 12:09
@jasagredo jasagredo force-pushed the js/io-manager-lib-suite-windows branch from 6f71d6e to ae5c57e Compare June 18, 2024 09:32
@jasagredo
Copy link
Collaborator Author

jasagredo commented Jun 18, 2024

This PR fails consistently in CI for GHC 9.0.2 with a Mingw error.

Mingw-w64 runtime failure:
  VirtualProtect failed with code 0x5afrealgcc.exe: internal compiler error: Aborted signal terminated program cc1
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://github.com/msys2/MINGW-packages/issues> for instructions.
`gcc.exe' failed in phase `C Compiler'. (Exit code: 4)
...
realgcc.exe: fatal error: cannot execute 'C://tools//ghc-9.0.2//mingw//bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/cc1.exe': CreateProcess: No such file or directory
compilation terminated.
`gcc.exe' failed in phase `C Compiler'. (Exit code: 1)
...
realgcc.exe: out of memory allocating 65536 bytes
`gcc.exe' failed in phase `C Compiler'. (Exit code: 1)
...
Unable to start C:\tools\ghc-9.0.2\mingw\bin/realgcc.exe (error code: 1455)
`gcc.exe' failed in phase `C Compiler'. (Exit code: 1)
...
*** Exception: dieVerbatim: user error (Error: <interactive>: 'C:\tools\ghc-9.0.2\bin\ghc.exe' exited with an error:
ghc-9.0.2.exe: osCommitMemory: VirtualAlloc MEM_COMMIT failed: The paging file
is too small for this operation to complete.
)

However I cannot reproduce it locally, cli-suite works with GHC 9.0.2:

=== cabal-install: cabal-testsuite =========================================== 17:03:41 ===
C:\Users\Javier\code\cabal\dist-newstyle-validate-ghc-9.0.2\build\x86_64-windows\ghc-9.0.2\cabal-testsuite-3\build\cabal-tests\cabal-tests.exe --builddir=/c/Users/Javier/code/cabal/dist-newstyle-validate-ghc-9.0.2/build/x86_64-windows/ghc-9.0.2/cabal-testsuite-3 --with-cabal=C:\Users\Javier\code\cabal\dist-newstyle-validate-ghc-9.0.2\build\x86_64-windows\ghc-9.0.2\cabal-install-3.13.0.0\x\cabal\build\cabal\cabal.exe -j4 --with-ghc=ghc --hide-successes --intree-cabal-lib=/c/Users/Javier/code/cabal --test-tmp=/c/Users/Javier/code/cabal/testdb +RTS --io-manager=native
...
PackageTests\Regression\T9756\cabal.test.hs                                                                       SKIP Windows (1.03s)
PackageTests\RequireExplicit\FlagInProject\cabal.test.hs                                                          SKIP Windows (0.79s)
PackageTests\RequireExplicit\MultiPkg\cabal.test.hs                                                               SKIP Windows (0.80s)
PackageTests\ShowBuildInfo\Complex\single.test.hs                                                                 SKIP needs ghc >= 9.2 (1.22s)
PackageTests\TemplateHaskell\dynamic\setup.test.hs                                                                SKIP no shared libs; no shared libs (1.64s)
PackageTests\VersionPriority\0-local.test.hs                                                                      SKIP Windows (1.78s)
PackageTests\VersionPriority\1-local.test.hs                                                                      SKIP Windows (1.54s)
PackageTests\VersionPriority\1-web.test.hs                                                                        SKIP Windows (1.64s)
PackageTests\VersionPriority\2-local.test.hs                                                                      SKIP Windows (1.65s)
PackageTests\VersionPriority\2-web.test.hs                                                                        SKIP Windows (1.54s)
PackageTests\VersionPriority\3-web.test.hs                                                                        SKIP Windows (1.51s)
PackageTests\WarnEarlyOverwrite\clean-install-by-copy.test.hs                                                     SKIP Windows (0.84s)
PackageTests\WarnEarlyOverwrite\clean-install-by-symlink.test.hs                                                  SKIP Windows (0.90s)
PackageTests\WarnEarlyOverwrite\dirty-install.test.hs                                                             SKIP Windows (0.99s)
GHCi exited with ExitFailure (-1073741510) (use -v for more information)
GHCi exited with ExitFailure (-1073741510) (use -v for more information)
GHCi exited with ExitFailure (-1073741510) (use -v for more information)
GHCi exited with ExitFailure (-1073741510) (use -v for more information)
555 tests, 104 skipped, 0 unexpected passes, 0 unexpected fails.
OK
<<< C:\Users\Javier\code\cabal\dist-newstyle-validate-ghc-9.0.2\build\x86_64-windows\ghc-9.0.2\cabal-testsuite-3\build\cabal-tests\cabal-tests.exe --builddir=/c/Users/Javier/code/cabal/dist-newstyle-validate-ghc-9.0.2/build/x86_64-windows/ghc-9.0.2/cabal-testsuite-3 --with-cabal=C:\Users\Javier\code\cabal\dist-newstyle-validate-ghc-9.0.2\build\x86_64-windows\ghc-9.0.2\cabal-install-3.13.0.0\x\cabal\build\cabal\cabal.exe -j4 --with-ghc=ghc --hide-successes --intree-cabal-lib=/c/Users/Javier/code/cabal --test-tmp=/c/Users/Javier/code/cabal/testdb +RTS --io-manager=native (933/951 sec)

Perhaps due to my machine having plenty of RAM memory? I don't really know where this comes from

@Mikolaj
Copy link
Member

Mikolaj commented Jun 18, 2024

If that was a random CI failure after runner images get updated, I'd advocate disabling these tests on Windows. But since the trigger seems to be this PR, I'm worried there may be some disastrous interaction of the added feature with some strange OS settings that some users may have too (or they may have them on their github CI). In which case, are they going to have a similar problem if this PR gets merged?

@mpickering
Copy link
Collaborator

I suspect that WinIO is just broken on 9.0.2 and nothing more can be done about that.

@jasagredo
Copy link
Collaborator Author

I suspect that WinIO is just broken on 9.0.2 and nothing more can be done about that.

However it doesn't fail on my local machine 🧩, only on CI.

@Mikolaj
Copy link
Member

Mikolaj commented Jun 20, 2024

If that's always only 9.0.2 , disabling the tests on Windows only on 9.0.2 seems fine to me, regardless of whether we can reproduce it locally.

@ulysses4ever
Copy link
Collaborator

I'd prefer disabling individual tests over the whole configuration (Windows/GHC 9.0) if that's not too much trouble. But if it is, then it's fine too imo.

@Kleidukos
Copy link
Member

My suggestion is that the validate script doesn't include this option in the GitHub CI environment. We can read environment variables just fine to take this decision.

@jasagredo jasagredo force-pushed the js/io-manager-lib-suite-windows branch from ae5c57e to c76f335 Compare June 20, 2024 20:45
@jasagredo
Copy link
Collaborator Author

jasagredo commented Jun 20, 2024

The validate script now sets --io-manager=native only on Windows non-CI executions.

@jasagredo jasagredo added merge me Tell Mergify Bot to merge and removed attention: needs-review labels Jun 20, 2024
@mergify mergify bot added the merge delay passed Applied (usually by Mergify) when PR approved and received no updates for 2 days label Jun 22, 2024
@Mikolaj Mikolaj force-pushed the js/io-manager-lib-suite-windows branch from c76f335 to 5c6e548 Compare June 22, 2024 22:43
@jasagredo jasagredo force-pushed the js/io-manager-lib-suite-windows branch from 5c6e548 to a307d98 Compare June 25, 2024 09:11
@Mikolaj Mikolaj force-pushed the js/io-manager-lib-suite-windows branch from a307d98 to a562b57 Compare June 25, 2024 11:29
@mergify mergify bot merged commit bf6f26d into haskell:master Jun 25, 2024
52 checks passed
@geekosaur
Copy link
Collaborator

I think this only works by accident:

validate.sh: 332: [[: not found

See for example https://github.com/haskell/cabal/actions/runs/9757762049/job/26944807720?pr=10167#step:13:13.
([[ is a bash/ksh-ism; POSIX shells are not required to support it.)

geekosaur added a commit to geekosaur/cabal that referenced this pull request Jul 4, 2024
Otherwise CI prints `validate.sh: 332: [[: not found` and the line
does nothing (but `validate.sh` continues to run), unless the
system shell is `ksh` / `bash` / `zsh`. This may explain
haskell#10114 (comment).
@geekosaur geekosaur mentioned this pull request Jul 4, 2024
2 tasks
geekosaur added a commit to geekosaur/cabal that referenced this pull request Jul 4, 2024
Otherwise CI prints `validate.sh: 332: [[: not found` and the line
does nothing (but `validate.sh` continues to run), unless the
system shell is `ksh` / `bash` / `zsh`. This may explain
haskell#10114 (comment).
Mikolaj pushed a commit to geekosaur/cabal that referenced this pull request Jul 6, 2024
Otherwise CI prints `validate.sh: 332: [[: not found` and the line
does nothing (but `validate.sh` continues to run), unless the
system shell is `ksh` / `bash` / `zsh`. This may explain
haskell#10114 (comment).
mergify bot pushed a commit that referenced this pull request Aug 30, 2024
Otherwise CI prints `validate.sh: 332: [[: not found` and the line
does nothing (but `validate.sh` continues to run), unless the
system shell is `ksh` / `bash` / `zsh`. This may explain
#10114 (comment).

(cherry picked from commit 0b5cf15)

# Conflicts:
#	validate.sh
@mergify mergify bot mentioned this pull request Aug 30, 2024
2 tasks
geekosaur added a commit that referenced this pull request Aug 30, 2024
Otherwise CI prints `validate.sh: 332: [[: not found` and the line
does nothing (but `validate.sh` continues to run), unless the
system shell is `ksh` / `bash` / `zsh`. This may explain
#10114 (comment).

(cherry picked from commit 0b5cf15)
@geekosaur
Copy link
Collaborator

@mergify backport 3.12

Copy link
Contributor

mergify bot commented Sep 13, 2024

backport 3.12

✅ Backports have been created

mergify bot added a commit that referenced this pull request Sep 14, 2024
Use `--io-manager=native` in `lib-suite` on Windows (backport #10114)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merge delay passed Applied (usually by Mergify) when PR approved and received no updates for 2 days merge me Tell Mergify Bot to merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Running the test-suite terminates abruptly with fd:5: hGetLine: end of file in MSYS2 Windows
6 participants