cmd/cgo: support clang on Windows

[nadiasvertex]

What version of Go are you using (go version)?

go version devel +0cff219 Wed Sep 7 10:43:13 2016 +0000 windows/amd64

What operating system and processor architecture are you using (go env)?

Windows 10 / AMD64

What did you do?

I used CGO with Clang for Windows 3.8.1 (http://llvm.org/releases/3.8.1/LLVM-3.8.1-win64.exe) like this:

set CC=clang
set CGO_CFLAGS="-m64 -Ipath/to/my/includes"
set CGO_LDFLAGS = "-v -Xlinker -libpath:path/to/libs -lwldap32 -ladvapi32 -lws2_32 -ldbghelp -luser32"

go build

What did you expect to see?

Expected to see nothing (ie, successful compile)

What did you see instead?

# editor/dom
clang version 3.8.1 (branches/release_38)
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\LLVM\bin
 "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\bin\\amd64\\link.exe" "-out:C:\\Users\\CHRIST~1\\AppData\\Local\\Temp\\go-build406860094\\editor\\dom\\_obj\\_cgo_.o" -defaultlib:libcmt "-libpath:C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\lib\\amd64" "-libpath:C:\\Program Files (x86)\\Windows Kits\\10\\Lib\\10.0.10240.0\\ucrt\\x64" "-libpath:C:\\Program Files (x86)\\Windows Kits\\8.1\\Lib\\winv6.3\\um\\x64" -nologo "C:\\Users\\CHRIST~1\\AppData\\Local\\Temp\\go-build406860094\\editor\\dom\\_obj\\_cgo_main.o" "C:\\Users\\CHRIST~1\\AppData\\Local\\Temp\\go-build406860094\\editor\\dom\\_obj\\_cgo_export.o" "C:\\Users\\CHRIST~1\\AppData\\Local\\Temp\\go-build406860094\\editor\\dom\\_obj\\binding.cgo2.o" "-libpath:c:\\temp\\go\\src\\editor\\.mm\\windows\\amd64\\release\\lib" core-static.lib ssl-static.lib crypto-static.lib ssh2-static.lib c-ares-static.lib user32.lib ws2_32.lib wldap32.lib advapi32.lib dbghelp.lib
   Creating library $WORK\editor\dom\_obj\_cgo_.lib and object $WORK\editor\dom\_obj\_cgo_.exp
core-static.lib(node.obj) : warning LNK4049: locally defined symbol xmlFree imported
core-static.lib(xml.obj) : warning LNK4049: locally defined symbol xmlFree imported
# editor/dom
clang version 3.8.1 (branches/release_38)
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\LLVM\bin
 "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\bin\\amd64\\link.exe" "-out:C:\\Users\\CHRIST~1\\AppData\\Local\\Temp\\go-build406860094\\editor\\dom\\_obj\\_all.o" "-libpath:C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\lib\\amd64" "-libpath:C:\\Program Files (x86)\\Windows Kits\\10\\Lib\\10.0.10240.0\\ucrt\\x64" "-libpath:C:\\Program Files (x86)\\Windows Kits\\8.1\\Lib\\winv6.3\\um\\x64" -nologo "C:\\Users\\CHRIST~1\\AppData\\Local\\Temp\\go-build406860094\\editor\\dom\\_obj\\_cgo_export.o" "C:\\Users\\CHRIST~1\\AppData\\Local\\Temp\\go-build406860094\\editor\\dom\\_obj\\binding.cgo2.o" -Wl,-r
LINK : warning LNK4044: unrecognized option '/Wl,-r'; ignored
LINK : fatal error LNK1561: entry point must be defined
clang.exe: error: linker command failed with exit code 1561 (use -v to see invocation)

[bradfitz]

I don't believe clang is supported yet. Our installation docs only mention mingw.

That said, we should probably support clang on Windows at some point.

/cc @ianlancetaylor @alexbrainman @crawshaw

[alexbrainman]

I have nothing against clang support. I have never had it installed on my computer. I will try install clang when I have some time.

Alex

[DmitriyMV]

I did some digging around and found out that with -Xlinker /subsystem:console it actually gets past entry point must be defined. But now I'm getting this error:

# runtime/cgo
clang version 3.9.0 (branches/release_39)
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Dmitry\LLVM_\bin
 "C:\\Program Files (x86)\\Microsoft Visual Studio\\VS15Preview\\Common7\\IDE\\VC\\bin\\amd64\\link.exe" "-out:C:\\Users\\Dmitry\\AppData\\Local\\Temp\\go-build586540150\\runtime\\cgo\\_obj\\_cgo_.o" -defaultlib:libcmt -nologo "C:\\Users\\Dmitry\\AppData\\Local\\Temp\\go-build586540150\\runtime\\cgo\\_obj\\_cgo_main.o" "C:\\Users\\Dmitry\\AppData\\Local\\Temp\\go-build586540150\\runtime\\cgo\\_obj\\_cgo_export.o" "C:\\Users\\Dmitry\\AppData\\Local\\Temp\\go-build586540150\\runtime\\cgo\\_obj\\cgo.cgo2.o" "C:\\Users\\Dmitry\\AppData\\Local\\Temp\\go-build586540150\\runtime\\cgo\\_obj\\gcc_context.o" "C:\\Users\\Dmitry\\AppData\\Local\\Temp\\go-build586540150\\runtime\\cgo\\_obj\\gcc_libinit_windows.o" "C:\\Users\\Dmitry\\AppData\\Local\\Temp\\go-build586540150\\runtime\\cgo\\_obj\\gcc_util.o" "C:\\Users\\Dmitry\\AppData\\Local\\Temp\\go-build586540150\\runtime\\cgo\\_obj\\gcc_windows_amd64.o" "
subsystem:console wldap32.lib advapi32.lib ws2_32.lib dbghelp.lib user32.lib kernel32.lib winmm.lib ntdll.lib
cmd/go
# runtime/cgo
clang version 3.9.0 (branches/release_39)
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Dmitry\LLVM_\bin
 "C:\\Program Files (x86)\\Microsoft Visual Studio\\VS15Preview\\Common7\\IDE\\VC\\bin\\amd64\\link.exe" "-out:C:\\Users\\Dmitry\\AppData\\Local\\Temp\\go-build586540150\\runtime\\cgo\\_obj\\_all.o" -nologo "C:\\Users\\Dmitry\\AppData\\Local\\Temp\\go-build586540150\\runtime\\cgo\\_obj\\_cgo_export.o" "C:\\Users\\Dmitry\\AppData\\Local\\Temp\\go-build586540150\\runtime\\cgo\\_obj\\cgo.cgo2.o" "C:\\Users\\Dmitry\\AppData\\Local\\Temp\\go-build586540150\\runtime\\cgo\\_obj\\gcc_context.o" "C:\\Users\\Dmitry\\AppData\\Local\\Temp\\go-build586540150\\runtime\\cgo\\_obj\\gcc_libinit_windows.o" "C:\\Users\\Dmitry\\AppData\\Local\\Temp\\go-build586540150\\runtime\\cgo\\_obj\\gcc_util.o" "C:\\Users\\Dmitry\\AppData\\Local\\Temp\\go-build586540150\\runtime\\cgo\\_obj\\gcc_windows_amd64.o" "C:\\Users\\Dmitry\\AppData\\Local\\Temp\\go-build586540150\\runtime\\cgo\\_obj\\gcc_amd64.o" /subsystem:console -r
LINK : warning LNK4044: unrecognized option '/r'; ignored
gcc_context.o : error LNK2019: unresolved external symbol __stdio_common_vsprintf referenced in function _vsnprintf_l
gcc_libinit_windows.o : error LNK2001: unresolved external symbol __stdio_common_vsprintf
gcc_util.o : error LNK2001: unresolved external symbol __stdio_common_vsprintf
gcc_windows_amd64.o : error LNK2001: unresolved external symbol __stdio_common_vsprintf
gcc_libinit_windows.o : error LNK2019: unresolved external symbol __imp_CreateEventA referenced in function _cgo_preinit_init
gcc_libinit_windows.o : error LNK2019: unresolved external symbol __acrt_iob_func referenced in function _cgo_preinit_init
gcc_util.o : error LNK2001: unresolved external symbol __acrt_iob_func
gcc_windows_amd64.o : error LNK2001: unresolved external symbol __acrt_iob_func
gcc_libinit_windows.o : error LNK2019: unresolved external symbol abort referenced in function _cgo_preinit_init
gcc_util.o : error LNK2001: unresolved external symbol abort
gcc_windows_amd64.o : error LNK2001: unresolved external symbol abort
gcc_libinit_windows.o : error LNK2019: unresolved external symbol __imp_InitializeCriticalSection referenced in function _cgo_preinit_init
gcc_libinit_windows.o : error LNK2019: unresolved external symbol __imp_Sleep referenced in function _cgo_maybe_run_preinit
gcc_libinit_windows.o : error LNK2019: unresolved external symbol _beginthread referenced in function x_cgo_sys_thread_create
gcc_windows_amd64.o : error LNK2001: unresolved external symbol _beginthread
gcc_libinit_windows.o : error LNK2019: unresolved external symbol _errno referenced in function x_cgo_sys_thread_create
gcc_windows_amd64.o : error LNK2001: unresolved external symbol _errno
gcc_libinit_windows.o : error LNK2019: unresolved external symbol __imp_EnterCriticalSection referenced in function _cgo_is_runtime_initialized
gcc_libinit_windows.o : error LNK2019: unresolved external symbol __imp_LeaveCriticalSection referenced in function _cgo_is_runtime_initialized
gcc_libinit_windows.o : error LNK2019: unresolved external symbol __imp_WaitForSingleObject referenced in function _cgo_wait_runtime_init_done
gcc_libinit_windows.o : error LNK2019: unresolved external symbol __imp_SetEvent referenced in function x_cgo_notify_runtime_init_done
gcc_libinit_windows.o : error LNK2019: unresolved external symbol __stdio_common_vfprintf referenced in function _vfprintf_l
gcc_util.o : error LNK2001: unresolved external symbol __stdio_common_vfprintf
gcc_windows_amd64.o : error LNK2001: unresolved external symbol __stdio_common_vfprintf
gcc_util.o : error LNK2019: unresolved external symbol malloc referenced in function x_cgo_thread_start
gcc_windows_amd64.o : error LNK2019: unresolved external symbol free referenced in function threadentry
LINK : error LNK2001: unresolved external symbol mainCRTStartup
C:\Users\Dmitry\AppData\Local\Temp\go-build586540150\runtime\cgo\_obj\_all.o : fatal error LNK1120: 16 unresolved externals
clang.exe: error: linker command failed with exit code 1120 (use -v to see invocation)

With:
set CGO_LDFLAGS=-v -Xlinker /subsystem:console -lwldap32 -ladvapi32 -lws2_32 -ldbghelp -luser32 -lkernel32 -lwinmm -lntdll

I guess, it cannot find the C libs, but then when I tried to add -libpath:"C:\Program Files (x86)\Microsoft Visual Studio\VS15Preview\Common7\IDE\VC\lib\" I'm sure I did something wrong because it started to produce this:

# runtime/cgo
clang version 3.9.0 (branches/release_39)
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Dmitry\LLVM_\bin
clang.exe: error: no such file or directory: 'Files'
clang.exe: error: no such file or directory: '(x86)/Microsoft'
clang.exe: error: no such file or directory: 'Visual'
clang.exe: error: no such file or directory: 'Studio/VS15Preview/Common7/IDE/VC/lib"'

@nadiasvertex I would really appreciate an advice on proper libpath setting.

[mattn]

Maybe it's related on #16455 ?

[DmitriyMV]

Yes - I created symlink because of that. It solved the second part of the problem, but the first(and main) remains.

[nadiasvertex]

@DmitriyMV I don't have much to tell you. The fundamental problem is that clang for Windows uses Microsoft's LIB as its backend. You do not need to tell LIB where to find the C library, it already knows that. I believe that the problem is that Go does not currently pass paths to clang in a way that LIB knows how to consume. I have not had success getting clang to compile with Go by playing with the commands. I think that the Go backend needs to be modified to send different values in order to make this work.

Sorry. I haven't had much time to work on this, but it is something that I would like to get working, or see someone else get working. :-)

[minux]

I think core issue is that the target for clang is: x86_64-pc-windows-msvc i.e. it's meant to be MSVC compatible. However, Go on windows rely on mingw for cgo. If we want to do anything here, it's probably to support msvc directly, rather than clang's emulation of it.

[nadiasvertex]

I prefer Clang's emulation. It makes the compiler part simpler since MSVC
is notoriously slow to get up to date with the standard, among other
frustrations. It would be really nice to be able to create MSVC-compatible
binaries without having to use MSVC. Not that I'm against also supporting
MSVC. We support a lot of C++ code that must compile under MSVC for now,
but we are very interested in dropping MSVC as soon as it is feasible.

On Tue, Nov 8, 2016, 2:48 AM Minux Ma [email protected] wrote:

I think core issue is that the target for clang is: x86_64-pc-windows-msvc
i.e. it's meant to be MSVC compatible.

However, Go on windows rely on mingw for cgo.

If we want to do anything here, it's probably to support msvc directly,
rather than clang's emulation of it.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#17014 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AB8JvBeiKJxeAx5mwRnS6oOx7G8k00lBks5q8Ck1gaJpZM4J2_91
.

[sbhushan87]

will this feature be available in GO1.8 release as I saw this issue has been linked to GO1.8Maybe issue list?

Any update will be helpful. thanks.

[minux]

If an issue remains open, then it's not in any of the releases.

[blizzardplus]

I'd like to get an update on this? I might be able to push this forward, if there's interest.

[alexbrainman]

I'd like to get an update on this?

Nothing new to report here.

I might be able to push this forward, if there's interest.

Go ahead if you know what to do. Everyone will help, if we can.
Maybe related #20982.

Alex

[as]

I guess, it cannot find the C libs, but then when I tried to add -libpath:"C:\Program Files (x86)\Microsoft Visual Studio\VS15Preview\Common7\IDE\VC\lib" I'm sure I did something wrong because it started to produce this:

# runtime/cgo
clang version 3.9.0 (branches/release_39)
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Dmitry\LLVM_\bin
clang.exe: error: no such file or directory: 'Files'
clang.exe: error: no such file or directory: '(x86)/Microsoft'
clang.exe: error: no such file or directory: 'Visual'
clang.exe: error: no such file or directory: 'Studio/VS15Preview/Common7/IDE/VC/lib"'

@DmitriyMV
It's splitting the path due to spaces, maybe try setting -libpath to a legacy 8.3 path to the same directory

-libpath C:\progra~2\microsoft\visual~1\VS15Preview\Common7\IDE\VC\lib

[blizzardplus]

I found a bunch of issues. If you could tell me where in the code these things are processed, I might be able to fix them, unless someone else wants to take care of them:

1. The Lib path in LDFLAGS should be passed as -L"path/to/libs"
2. If there is space in the lib path, it does not compile for me. I don't know how spaces are handled by cgo, but when I copied my lib folder to "C:", it had no issue finding it.
3. For some reason, the clang linker command is missing the _cgo_main.o object when linking objects.
4. For me, the linker command is called with "-nostdlib" option and it causes this error: "LINK : error LNK2001: unresolved external symbol mainCRTStartup"

Note: I used the go build -x -work command to see the clang commands and keep the temp working directory.

[alexbrainman]

The Lib path in LDFLAGS should be passed as -L"path/to/libs"

OK. What is your question?

If there is space in the lib path, it does not compile for me. I don't know how spaces are handled by cgo, but when I copied my lib folder to "C:", it had no issue finding it.

I would copy all code into directories without spaces at the start. Once that works, we can deal with paths with spaces.

For some reason, the clang linker command is missing the _cgo_main.o object when linking objects.

According to $GOROOT/src/cmd/cgo/doc.go file:

the build process generates an object file using dynamic
linkage to the desired libraries. The main function is provided by
_cgo_main.c:
	int main() { return 0; }
	void crosscall2(void(*fn)(void*, int, uintptr_t), void *a, int c, uintptr_t ctxt) { }
	uintptr_t _cgo_wait_runtime_init_done() { return 0; }
	void _cgo_release_context(uintptr_t ctxt) { }
	char* _cgo_topofstack(void) { return (char*)0; }
	void _cgo_allocate(void *a, int c) { }
	void _cgo_panic(void *a, int c) { }
	void _cgo_reginit(void) { }
The extra functions here are stubs to satisfy the references in the C
code generated for gcc. The build process links this stub, along with
_cgo_export.c and *.cgo2.c, into a dynamic executable and then lets
cgo examine the executable. 

So it seems to me Go creates _cgo_main.c file and then uses gcc to compile it into _cgo_main.o. Does _cgo_main.c gets created? It should be compiled into _cgo_main.o by invoking gcc, but you do not have gcc. Do you use clang to compile _cgo_main.c? Does compilation succeeds or fails?

For me, the linker command is called with "-nostdlib" option and it causes this error: "LINK : error LNK2001: unresolved external symbol mainCRTStartup"

Searching for "-nostdlib", I find many instances in $GOROOT/src/cmd/go/internal/work/build.go. If clang does not accepts "-nostdlib", maybe try and adjust there?

Note: I used the go build -x -work command to see the clang commands and keep the temp working directory.

The "go build -x -work" only controls go command itself. You, probably, want to do the same for the cmd/link as well. See #20982 (comment) for how to access cmd/link temp files. You might also find -v cmd/link flag useful (or "-v -v").

I hope it helps.

Alex

[ghost]

The main issue that I received when using the MSVC linker is a segmentation fault that occurred while attempting to bind stdout. I can help to resolve the unresolved symbols but the segmentation fault issue is a bit trickier. My plan for that is to basically manually link everything with MinGW and then use DLL linking to factor out each object at a time until I receive the segmentation fault. I think then I will need to compare assembly to see what the problem is, although maybe there is someone more talented than I am that has a better idea? I am probably stretching the limits of my capability so I am open to as much help as I can get.

[alexbrainman]

The main issue that I received when using the MSVC linker is a segmentation fault that occurred while attempting to bind stdout.

@xoviat are you taking about issue #20982? Did you manage to build executable successfully? But the executable crashes when it runs? Perhaps if you can provide the steps, I will try and debug it too.

Maybe reply on #20982 so we don't confuse things.

Thank you.

Alex

[blizzardplus]

Searching for "-nostdlib", I find many instances in $GOROOT/src/cmd/go/internal/work/build.go. If clang does not accepts "-nostdlib", maybe try and adjust there?

I tried to remove the -nostdlib option from the source file and build Go from source, but having some linking issues when building the source code, so stuck there! The problem is that the build.go seems to be appending the option at the end of LDFLAGS, so no matter what LDFLAG options user provides, -nostdlib will be appended. I'm wondering if there is a way to completely override the flags?

Note: I used the go build -x -work command to see the clang commands and keep the temp working directory.
The "go build -x -work" only controls go command itself. You, probably, want to do the same for the cmd/link as well. See #20982 (comment) for how to access cmd/link temp files. You might also find -v cmd/link flag useful (or "-v -v").

I can link the files, by manually adding missing object files and modifying linker options, but don't know the next step the build process takes, so I can't move further.

[ghost]

@blizzardplus Do you have libgo?

[neonphog]

I'm trying use a go library through rust ffi tx5-go-pion-sys

A bunch of rust deps will only build on windows through msvc, and go will only build with mingw. So right now we're using dynamic linking, but it'd be nice if go supported either msvc directly, or clang and i could use clang-for-msvc so I could statically link the library.

[vault-thirteen]

Golang positions itself as a "cross-platform language".
Why is there still no support for a cross-platform technology, the LLVM, in a cross-platform language Go ?

Why are the Go wrapper libraries which use CGO store its source code as on Unix or Linux O.S. ?
Why not store the results as on MS-DOS ? Is MS-DOS bad ?

Some libraries written in C language have portability issues when imported as Unix-Linux source code and compiled on Windows O.S. This fact does not allow me to say that Golang is a truly cross-platfrom language. At least its CGO feature is not truly cross-platform.

Moreover, the GCC compiler is not a best solution for compiling code for Windows O.S.
GCC was created for GNU/Linux and BSD and has issues when used on Windows.

[neptoess]

Moreover, the GCC compiler is not a best solution for compiling code for Windows O.S. GCC was created for GNU/Linux and BSD and has issues when used on Windows.

The rest is a bit philosophical, but I’m going to disagree on this point. There are certainly codebases where other compilers will produce “better” output on Windows, but I’ve shipped production Go code using MinGW gcc for the Windows cgo dependencies, and it’s been quite reliable, given me performant code, etc.

I’m not even positive that LLVM on Windows doesn’t work with cgo, because it certainly does on macOS.

[ianlancetaylor]

@vault-thirteen Thanks for the questions. Go is an open source project. As such, the answer to these kinds of questions tends to be "because nobody has fixed it yet." This issue is tagged "help wanted". Would you like to fix it?

[vault-thirteen]

Hello @ianlancetaylor.

There are several moments about the Go language.

1. I saw several merge requests (they are called pull requests on GitHub) about fixing missing C compilers for CGO. These MRs (PRs) were intended to change source code of Go language related to CLang and MSVC. These requests were closed without merging. So, to me it looks not like a question of open-source-ness.

Open-source means literally source which is published to people.
Open-source does not mean that it allows anybody to add anything to the product.

2. Go language is controlled by Google corporation. Open-sourceness does not affect features of the language if Google does not want to allow feature X or feature Y for some reason.

Would you like to fix it ?

Yes, I would like to fix this, but I am not a creator of this language.

[gophun]

@vault-thirteen The Go project does not do GitHub pull requests, it uses Gerrit, as has been pointed out in the PR you seem to be referring to. Also, it was the author who closed it.

[vault-thirteen]

@gophun thank you for correcting me.
So, does that mean that we have chances to see alternative C compilers in CGO ? :-)

[gophun]

I don't see why not, if someone steps up and pulls through the whole contribution process and does not abandon their change halfway through.

[vault-thirteen]

Does CGO provide a way to configure the process of compiling and linking ?
I think it is not giving a user such an opportunity. Correct me if it is not true.
I think the main reason is there.

[gophun]

Yes, it does. The environment variables are called CC, CXX, CGO_CFLAGS, CGO_CPPFLAGS, CGO_CXXFLAGS, CGO_FFLAGS, CGO_LDFLAGS. https://pkg.go.dev/cmd/cgo#hdr-Using_cgo_with_the_go_command

[vault-thirteen]

Thank you, @gophun.
I will try to study all this when I have more time.

[vault-thirteen]

Where can I find the source code of the CGO mechanism ?

I have found these parts so far:

• https://github.com/golang/go/tree/master/misc/cgo
• https://github.com/golang/go/tree/master/src/cmd/cgo

Are there any other parts available ?

[vault-thirteen]

Why are there a lot of unresolved references in Go's source code ? How do you build it ?
I will have to open a new issue ...

[TBBle]

How do you build it ?

https://go.dev/doc/install/source appears to be the instructions for building Go from source.

[ianlancetaylor]

Where can I find the source code of the CGO mechanism ?

What are you looking for more precisely? As you've discovered, cmd/cgo is the tool that takes code that transform Go code that uses import "C" into Go code that does not. The generated Go code relies on support in cmd/compile and in runtime, as described in the long comment at the end of cmd/cgo/doc.go.

Why are there a lot of unresolved references in Go's source code ?

There aren't. For questions like this we don't use the issue tracker. Please see https://go.dev/wiki/Questions. Thanks.

[voltagex]

I have tried the patch from Gerrit to enable MSVC's clang to be used, unfortunately the following program crashes

package main

/*
#include <stdio.h>
#include <stdlib.h>

void myprint() {
 printf("Hello world\n");
}
*/
import "C"

func main() {
 C.myprint()
}

PS C:\git\cgo-test> go run hello.go
# command-line-arguments
   Creating library $WORK\b001\exe\hello.lib and object $WORK\b001\exe\hello.exp
minpc= 0x7ff70dad5079 min= 0x7ff70dad9ee0 maxpc= 0x7ff70db39b2a max= 0x7ff70db39b2a
fatal error: minpc or maxpc invalid
runtime: panic before malloc heap initialized

runtime stack:
runtime.stopTheWorldWithSema(0x40)
        c:/git/goroot/src/runtime/proc.go:1321 fp=0xba6acffa08 sp=0xba6acffa00 pc=0x7ff70db07b25
runtime: g 0: unexpected return pc for runtime.stopTheWorldWithSema called from 0xba6acffa08
stack: frame={sp:0xba6acffa00, fp:0xba6acffa08} stack=[0xba6acefb90,0xba6acffb90)
0x000000ba6acff900:  0x000000ba6acff970  0x00007ff70db0837e <runtime.forEachP+0xffffffffffffb35e>
0x000000ba6acff910:  0x00007ff70dbddc9b  0x000000ba6acff948
0x000000ba6acff920:  0x00007ff70db0bad1 <runtime.schedule+0xffffffffffffb1f1>  0x00007ff70dc78080
0x000000ba6acff930:  0x0000000000000000  0x01007ff70db0bb00
0x000000ba6acff940:  0x0000000000000008  0x000000ba6acff970
0x000000ba6acff950:  0x00007ff70db08192 <runtime.mexit+0xffffffffffffb3d2>  0x00007ff70dbe232d
0x000000ba6acff960:  0x00007ff70db07b25 <runtime.stopTheWorldWithSema+0xffffffffffffb3a5>  0x000000ba6acffa00
0x000000ba6acff970:  0x000000ba6acff9b0  0x00007ff70db07ea5 <runtime.mstart1+0xffffffffffffb225>
0x000000ba6acff980:  0x00007ff70dc78080  0x00007ff70dbf51e8
0x000000ba6acff990:  0x0000000000000001  0x000000ba6acffa00
0x000000ba6acff9a0:  0x00007ff70db07b25 <runtime.stopTheWorldWithSema+0xffffffffffffb3a5>  0x00007ff70dc78080
0x000000ba6acff9b0:  0x000000ba6acff9f0  0x00007ff70db07e25 <runtime.startTheWorldWithSema+0xffffffffffffb405>
0x000000ba6acff9c0:  0x000000ba6acff9d0  0x00007ff70dc78080
0x000000ba6acff9d0:  0x00007ff70db07e60 <runtime.startTheWorldWithSema+0xffffffffffffb440>  0x00007ff70dc78080
0x000000ba6acff9e0:  0x00007ff70db07b25 <runtime.stopTheWorldWithSema+0xffffffffffffb3a5>  0x000000ba6acffa00
0x000000ba6acff9f0:  0x000000ba6acffa20  0x00007ff70db07b25 <runtime.stopTheWorldWithSema+0xffffffffffffb3a5>
0x000000ba6acffa00: <0x000000ba6acffa08 >0x00007ff70db07b40 <runtime.stopTheWorldWithSema+0xffffffffffffb3c0>
0x000000ba6acffa10:  0x00007ff70dbdee1d  0x0000000000000016
0x000000ba6acffa20:  0x000000ba6acffb40  0x00007ff70db1f0a5 <runtime.(*traceStackTable).put.func1+0xffffffffffffb1c5>
0x000000ba6acffa30:  0x00007ff70dbdee1d  0x000000ba6acffa58
0x000000ba6acffa40:  0x0000000000000000  0xfffffff100000000
0x000000ba6acffa50:  0x000000ba6acffaa0  0x000000ba6acffa90
0x000000ba6acffa60:  0x0000000000000489  0x00007ff70dad9ee0 <runtime.dumpregs+0xffffffffffffb680>
0x000000ba6acffa70:  0x00007ff70db39b2a  0x000000ba6acffa70
0x000000ba6acffa80:  0x000000ba6acffad0  0x00007ff70db2fcca <runtime.mstart0+0xffffffffffffb1ea>
0x000000ba6acffa90:  0x000000ba6acffaa8  0x00007ff70db05156 <runtime.hexdumpWords+0xffffffffffffb2f6>
0x000000ba6acffaa0:  0x0000000000000001  0x000000ba6acffb20
0x000000ba6acffab0:  0x00007ff70db39b2a  0x00007ff70dad5079 fatal error: unexpected signal during runtime execution
runtime: panic before malloc heap initialized
panic during panic
[signal 0x00000000c0000005 code=0x0000000000000000 addr=0x00007ff70eff6a40 pc=0x00007ff70db1f5db]

runtime stack:
runtime.stopTheWorldWithSema(0x0000000000000040)
        c:/git/goroot/src/runtime/proc.go:1321 fp=0x000000ba6acfefe0 sp=0x000000ba6acfefd8 pc=0x00007ff70db07b25
runtime: g 0: unexpected return pc for runtime.stopTheWorldWithSema called from 0x000000ba6acfefe0
stack: frame={sp:0x000000ba6acfefd8, fp:0x000000ba6acfefe0} stack=[0x000000ba6acefb90,0x000000ba6acffb90)
0x000000ba6acfeed8:  0x000000ba6acfef48  0x00007ff70db0837e <runtime.forEachP+0xffffffffffffb35e>
0x000000ba6acfeee8:  0x00007ff70dbddc9b  0x000000ba6acfef28
0x000000ba6acfeef8:  0x00007ff70dbde4dd  0x00007ff70dc78080
0x000000ba6acfef08:  0x00007ff70dbde4dd  0x0100000000000013
0x000000ba6acfef18:  0xc000000500000008  0x00007ff70db1f5db <runtime.traceGCSweepSpan+0xffffffffffffb25b>
0x000000ba6acfef28:  0x00007ff70eff6a40  0x0000000000000000
0x000000ba6acfef38:  0x00007ff70db07b25 <runtime.stopTheWorldWithSema+0xffffffffffffb3a5>  0x000000ba6acfefd8
0x000000ba6acfef48:  0x000000ba6acfef88  0x00007ff70db07ea5 <runtime.mstart1+0xffffffffffffb225>
0x000000ba6acfef58:  0x00007ff70dc78080  0x00007ff70dbf51e8
0x000000ba6acfef68:  0x0000000000000001  0x000000ba6acfefd8
0x000000ba6acfef78:  0x00007ff70db07b25 <runtime.stopTheWorldWithSema+0xffffffffffffb3a5>  0x00007ff70dc78080
0x000000ba6acfef88:  0x000000ba6acfefc8  0x00007ff70db07e25 <runtime.startTheWorldWithSema+0xffffffffffffb405>
0x000000ba6acfef98:  0x000000ba6acfefa8  0x00007ff70dc78080
0x000000ba6acfefa8:  0x00007ff70db07e60 <runtime.startTheWorldWithSema+0xffffffffffffb440>  0x00007ff70dc78080
0x000000ba6acfefb8:  0x00007ff70db07b25 <runtime.stopTheWorldWithSema+0xffffffffffffb3a5>  0x000000ba6acfefd8
0x000000ba6acfefc8:  0x000000ba6acfeff8  0x00007ff70db07b25 <runtime.stopTheWorldWithSema+0xffffffffffffb3a5>
0x000000ba6acfefd8: <0x000000ba6acfefe0 >0x00007ff70db07b40 <runtime.stopTheWorldWithSema+0xffffffffffffb3c0>
0x000000ba6acfefe8:  0x00007ff70dbe1cc5  0x000000000000002a
0x000000ba6acfeff8:  0x000000ba6acff040  0x00007ff70db19d85 <runtime.moduledataverify1+0xffffffffffffb225>
0x000000ba6acff008:  0x00007ff70dbe1cc5  0x0000000000000000
0x000000ba6acff018:  0x00007ff70dbdc319  0x00007ff70dc78080
0x000000ba6acff028:  0x00007ff70dbdc319  0x0000000000000001
0x000000ba6acff038:  0x0000000000000001  0x000000ba6acff060
0x000000ba6acff048:  0x00007ff70db1f5db <runtime.traceGCSweepSpan+0xffffffffffffb25b>  0x00007ff70dbdc319
0x000000ba6acff058:  0x3937303564616430  0x000000ba6acff0d0
0x000000ba6acff068:  0x00007ff70db09fa5 <runtime.gcstopm+0xffffffffffffb205>  0x000000ba6acff08f
0x000000ba6acff078:  0x0000000000000001  0x000000ba6acff0d0
0x000000ba6acff088:  0x20007ff70db09d54  0x0000000000000014
0x000000ba6acff098:  0x00007ff70db09e60 <runtime.stoplockedm+0xffffffffffffb400>  0x00007ff70dad5079 fatal error: unexpected signal during runtime execution
runtime: panic before malloc heap initialized
stack trace unavailable
exit status 4

[davy-ikv]

I've tried cross-compile from macOS to Windows with llvm-mingw (using ucrt) and it works well. Running the output binary on Windows successfully.

[voltagex]

I can confirm the patch in #54811 works against go 1.22.2.

Because I didn't document my last experiment properly

FROM debian:sid
WORKDIR /src
RUN apt update && apt full-upgrade -y
RUN apt install --no-install-recommends -y ca-certificates git golang-go
RUN git clone --single-branch --branch go1.22.2 https://github.com/golang/go

WORKDIR /src/go
COPY *.patch .
RUN git apply *.patch
WORKDIR /src/go/src
RUN GOOS=windows GOARCH=amd64 ./make.bash

• I could not work out how to pull a complete toolchain out of what I had built. Therefore I installed Go 1.22.2 on Windows, and replaced go.exe and link.exe from the patched source above.
• I installed a full Visual Studio 2022 with additional clang component from the installer. I suspect you'd be able to get away with the Build Tools and clang.
• You need to set CGO_ENABLED=1 and CC=clang
• You need to start the x64 Native Tools command prompt
• I was then able to build a simple binary with SQLite linked in.

HOWEVER:
It was then immediately detected as malicious by Windows Defender. This is of course, ridiculous.

https://www.microsoft.com/en-us/wdsi/submission/89af1110-6167-44d0-918d-d1c440a81b6f
https://www.microsoft.com/en-us/wdsi/submission/1b794e3a-f8df-4fbd-bf75-e799296f6a85
https://www.virustotal.com/gui/file/c97d9c5a02811bae97048042244a7cee0d44b0459b8f6687fb0cd7f924f7c60b
https://www.virustotal.com/gui/file/7b1e31ad7922c2b0573c11ce4157ab6f366d9d9d2c5687edac5219d212b8ffb9
(although my test.exe wasn't actually built with CGO_ENABLED)

If anyone is interested in getting this merged I am going to need help with the Gerrit process, and I'm going to need to find someone from Microsoft interested in fixing whatever's making Defender angry.

I believe link.exe built from source is also being removed!

[voltagex]

@dagood / @gdams sorry to ping you directly for this, but maybe this would be something in your wheelhouse

[dagood]

Assuming the ping is about Defender specifically, it looks like https://www.microsoft.com/en-us/wdsi/submission/89af1110-6167-44d0-918d-d1c440a81b6f and https://www.microsoft.com/en-us/wdsi/submission/1b794e3a-f8df-4fbd-bf75-e799296f6a85 have expired. If you upload those again, we may be able to dig deeper into this with the Defender team. (Sorry if it's just a permissions issue and they're still visible on your end, let me know if that's the case.)

Windows Defender is well known to have issues with Go programs in general (https://go.dev/doc/faq#virus, microsoft/go#1255), and it seems understandable to me if switching to a not-before-seen cgo compiler is making Defender act up more than usual. I'm a little confused about your comment about not having CGO_ENABLED enabled, though.

More broadly, I do think clang (and other) C compiler support for cgo on Windows is something that we'd be interested in helping with, but I can't promise we'll have time for it soon.

(For context on the thread: @gdams and I work on the Microsoft fork of Go and try to help out with OS-Windows issues.)

[voltagex]

@dagood thanks for the response!

If you upload those again, we may be able to dig deeper into this with the Defender team.
Sorry, I don't have these files any more, or the setup to build them again. I think you can get them from VirusTotal - I don't have that kind of account though.

I think DetectItEasy has a good hint as to what might be triggering Defender - repeating section names? Maybe something has gone wrong with the linker in that case.

To be honest, I'm less interested in solving the Defender alert right now, it'd just be good to get some additional help to get this running as it's currently beyond my ability.