Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trying to speed up type handling #1690

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Conversation

oxisto
Copy link
Member

@oxisto oxisto commented Sep 13, 2024

This PR should speed up type handling immensely. The list of first oder types is now a map of a list instead of a list. The issue is that we have a lot of "types" (remember, a "type" is a unique combination of a scope and a name), we potentially have many scopes and a simple "myType" can exist as multiple Type nodes before the type resolver harmonises this. In order to at least have unique types per scope, we iterate existing types and return one that matches the name/scope combination. This was previously done over the complete list and is now only restricted to those types whos name matches in the first place.

This is more a workaround than a complete solution to the problem of types but should at least be significantly faster now.

As a benchmark I parsed http://github.com/openstack/glance. This speeds up the parsing from 73s to 42s.

Before:

Metric Value
TranslationManager: Executing Language Frontend 16422 ms
CompressLLVMPass: Executing Pass 2 ms
ImportResolver: Executing Pass 528 ms
TypeResolver: Executing Pass 988 ms
TypeHierarchyResolver: Executing Pass 425 ms
CXXExtraPass: Executing Pass 215 ms
GoExtraPass: Executing Pass 327 ms
PythonAddDeclarationsPass: Executing Pass 1278 ms
ResolveCallExpressionAmbiguityPass: Executing Pass 4378 ms
JavaExternalTypeHierarchyResolver: Executing Pass 2 ms
EvaluationOrderGraphPass: Executing Pass 698 ms
JavaImportResolver: Executing Pass 0 ms
SymbolResolver: Executing Pass 33495 ms
ControlDependenceGraphPass: Executing Pass 6523 ms
DFGPass: Executing Pass 792 ms
ControlFlowSensitiveDFGPass: Executing Pass 3183 ms
DynamicInvokeResolver: Executing Pass 433 ms
ProgramDependenceGraphPass: Executing Pass 2295 ms
PrepareSerialization: Executing Pass 807 ms
FilenameMapper: Executing Pass 320 ms
TranslationManager: Translation into full graph 73128 ms

After:

Metric Value
TranslationManager: Executing Language Frontend 15854 ms
CompressLLVMPass: Executing Pass 6 ms
ImportResolver: Executing Pass 575 ms
TypeResolver: Executing Pass 856 ms
TypeHierarchyResolver: Executing Pass 198 ms
CXXExtraPass: Executing Pass 201 ms
GoExtraPass: Executing Pass 311 ms
PythonAddDeclarationsPass: Executing Pass 1360 ms
ResolveCallExpressionAmbiguityPass: Executing Pass 274 ms
JavaExternalTypeHierarchyResolver: Executing Pass 2 ms
EvaluationOrderGraphPass: Executing Pass 835 ms
JavaImportResolver: Executing Pass 1 ms
SymbolResolver: Executing Pass 7557 ms
ControlDependenceGraphPass: Executing Pass 7046 ms
DFGPass: Executing Pass 807 ms
ControlFlowSensitiveDFGPass: Executing Pass 3399 ms
DynamicInvokeResolver: Executing Pass 311 ms
ProgramDependenceGraphPass: Executing Pass 1667 ms
PrepareSerialization: Executing Pass 626 ms
FilenameMapper: Executing Pass 253 ms
TranslationManager: Translation into full graph 42154 ms

@maximiliankaul
Copy link
Contributor

maximiliankaul commented Sep 23, 2024

Speedup (start to end cpg parsing (frontend + passes)) against current main is about 4x with a larger Python test case 👍

@oxisto oxisto mentioned this pull request Sep 25, 2024
@oxisto oxisto marked this pull request as ready for review September 25, 2024 11:31
@oxisto oxisto force-pushed the speed-up-type-resolution branch 2 times, most recently from 6350b33 to 1975f92 Compare September 28, 2024 08:39
@oxisto
Copy link
Member Author

oxisto commented Sep 28, 2024

Unfortunately, while this speeds up python, this slows down the CXXExtraPass extremely :(

Copy link

sonarcloud bot commented Sep 28, 2024

Please retry analysis of this Pull-Request directly on SonarCloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants