-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hook for debugging shared libraries in Visual Studio #121
Changes from all commits
6cf9cc1
c355dfd
1cc8980
7374782
c1231cf
cc25825
da05d4c
a63049a
ca13d8d
907169b
d15cc2e
05c5a12
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
## PDBs hook | ||
This hook copies the PDBs from their original location in the build folder to the package folder. | ||
This is required for debugging libraries with Visual Studio when the original source files aren't present. | ||
For more information on how to debug using the hook check the [documentation](https://docs.conan.io/2/examples/dev_flow/debug/debugging_visual.html) | ||
|
||
### PDBs | ||
|
||
A PDB has the information to link the source code of a debuggable object to the Visual Studio debugger. Each PDB is linked to a | ||
specific file (executable or library) and contains the source file name and line numbers to display in the IDE. | ||
|
||
When compiling shared libraries in Debug mode the created binary will contain the information of where the PDB will be | ||
generated, which by default is the same path where the file is being compiled. The PDBs are created by the ``cl.exe`` | ||
compiler with the ``/Zi`` flag, or by the ``link.exe`` when linking a DLL or executable. | ||
|
||
PDBs are created when compiling a library or executable in Debug mode and are created by default in the same directory | ||
as the file it is associated with. This means that when using Conan they will be created in the build directory in the | ||
same path as the DLLs. | ||
|
||
When using the Visual Studio debugger, it will look for PDBs to load in the following paths: | ||
|
||
- The project folder. | ||
- The original path where the associated file was compiled. | ||
- The path where Visual is currently finding the compiled file, in our case the DLL in the package folder. | ||
|
||
### Locating PDBs | ||
|
||
To locate the PDB of a DLL we can use the ``dumpbin.exe`` tool, which comes with Visual by default and can be located | ||
using the ``vswhere`` tool. PDBs will usually have the same name as it's | ||
DLL, but it's not always the case, so checking with the ``dumpbin \PDBPATH`` command makes sure we are getting the PDB | ||
corresponding to each DLL. | ||
|
||
When a DLL is created it contains the information of the path where its corresponding PDB was generated. This can be | ||
manually checked by running the following commands: | ||
``` | ||
$ "%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -find "**\dumpbin.exe" | ||
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\dumpbin.exe | ||
|
||
# Use the path for the dumpbin.exe that you got from the previous command | ||
$ "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\dumpbin.exe" /PDBPATH <dll_path> | ||
... | ||
Dump of file .\bin\zlib1.dll | ||
|
||
File Type: DLL | ||
PDB file found at 'C:\Users\{user}\.conan2\p\b\zlib78326f0099328\p\bin\zlib1.pdb' | ||
... | ||
``` | ||
|
||
### Source files | ||
|
||
It is important to note that the PDB only contains the debug information, to debug through the actual file the source | ||
files are needed. These files have to be the exact same files as when the library was compiled and the PDB generated, | ||
as Visual Studio does a checksum check. In case where the build folder was deleted the source files from the source | ||
folder can be used instead by telling Visual where to find them, as it is explained in the documentation. | ||
|
||
### Static libraries | ||
|
||
PDBs can sometimes be generated for LIB files, but for now the feature only focuses on shared libraries and | ||
will only work with PDBs generated for DLLs. This is because the linking of PDBs and static libraries works differently | ||
than with shared libraries and the PDBs are generated differently, which doesn't allow us to get the name and path | ||
of a PDB through the ``dumpbin`` tool and will require different methods. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
from conan.tools.files import copy | ||
import glob | ||
import json | ||
import os | ||
import re | ||
from io import StringIO | ||
from conans.errors import ConanException | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. from conan.errors import ConanException will be more future-proof |
||
|
||
|
||
def post_package(conanfile): | ||
if conanfile.settings.get_safe("os") != "Windows" or conanfile.settings.get_safe("compiler") != "msvc": | ||
return | ||
conanfile.output.info("PDBs post package hook running") | ||
search_package_dll = os.path.join(conanfile.package_folder, "**/*.dll") | ||
package_dll = glob.glob(search_package_dll, recursive=True) | ||
if len(package_dll) == 0: | ||
return | ||
# Find dumpbin path | ||
output = StringIO() | ||
try: | ||
conanfile.run( | ||
r'"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -find "**\dumpbin.exe" -format json', | ||
stdout=output, scope="") | ||
except ConanException: | ||
raise ConanException( | ||
"Failed to locate dumpbin.exe which is needed to locate the PDBs and copy them to package folder.") | ||
dumpbin_path = json.loads(str(output.getvalue()))[0] | ||
|
||
for dll_path in package_dll: | ||
# Use dumpbin to get the pdb path from each dll | ||
dumpbin_output = StringIO() | ||
conanfile.run(rf'"{dumpbin_path}" /PDBPATH {dll_path}', stdout=dumpbin_output) | ||
dumpbin = str(dumpbin_output.getvalue()) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Usually, what paths are returned here? I just realized that for example most CCI recipes remove the pdbs in the package method, and if tyhis happens after the package, maybe the pdbs do not exist anymore for the interested recipes - I'm saying that maybe we shoukd have a way to avoid those pdb deletions in the first place? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you give me an example of a recipe that does this so I can check how it works? As I understand it dumpbin gets the path of were the PDB is originally located, this is usually (always?) where the library is created which should be the build folder not the package one. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Tested this and everything is working fine as expected! There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This should be fine. Having a look at https://learn.microsoft.com/en-us/cpp/build/reference/pdbpath?view=msvc-170, it should return the path embedded in the binary, that should be the path in the build folder where the binary was originated. As this is a post_package hook, it should work even if the package() method deletes the pdbs because the hook is executed after the method 😄 |
||
pdb_path = re.search(r"'.*\.pdb'", dumpbin) | ||
if pdb_path: | ||
pdb_path = pdb_path.group()[1:-1] | ||
# Copy the corresponding pdb file from the build to the package folder | ||
conanfile.output.info( | ||
f"copying {os.path.basename(pdb_path)} from {os.path.dirname(pdb_path)} to {os.path.dirname(dll_path)}") | ||
copy(conanfile, os.path.basename(pdb_path), os.path.dirname(pdb_path), os.path.dirname(dll_path)) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,46 @@ | ||
import os | ||
import tempfile | ||
import json | ||
import platform | ||
|
||
import pytest | ||
|
||
from tools import run | ||
|
||
|
||
@pytest.fixture(autouse=True) | ||
def conan_test(): | ||
old_env = dict(os.environ) | ||
env_vars = {'CONAN_HOME': tempfile.mkdtemp(suffix='conans')} | ||
os.environ.update(env_vars) | ||
current = tempfile.mkdtemp(suffix='conans') | ||
cwd = os.getcwd() | ||
os.chdir(current) | ||
try: | ||
yield | ||
finally: | ||
os.chdir(cwd) | ||
os.environ.clear() | ||
os.environ.update(old_env) | ||
|
||
|
||
@pytest.mark.win32 | ||
def test_copy_pdb_hook(): | ||
print("This is a placeholder for future tests") | ||
repo = os.path.join(os.path.dirname(__file__), "..") | ||
run(f'conan config install {repo}') | ||
conan_home = run('conan config home').strip() | ||
hooks_path = os.path.join(conan_home, 'extensions', 'hooks') | ||
old_file = os.path.join(hooks_path, '_hook_copy_pdbs_to_package.py') | ||
new_file = os.path.join(hooks_path, 'hook_copy_pdbs_to_package.py') | ||
os.rename(old_file, new_file) | ||
run('conan profile detect') | ||
run('conan new cmake_lib -d name=lib -d version=1.0') | ||
out = run('conan create . -s build_type=Debug -o "*:shared=True" -tf=""') | ||
assert "PDBs post package hook running" in out | ||
list_output = run('conan list lib/1.0:* --format=json') | ||
list_json = json.loads(list_output) | ||
revision = list_json['Local Cache']['lib/1.0']['revisions'].values() | ||
revision_info = next(iter(revision)) | ||
package_id = next(iter(revision_info['packages'])) | ||
path = run(fr'conan cache path lib/1.0:{package_id}').strip() | ||
assert os.path.isfile(os.path.join(path, 'bin', 'lib.pdb')) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
package - typo