Fix an issue with the last level cache values on Linux running on certain AMD Processors #108492
+98
−58
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mrsharm
commented
Oct 2, 2024
This was referenced Oct 3, 2024
3 tasks
mrsharm
changed the title
Maoni0
reviewed
Oct 5, 2024
| if (cacheSize == 0) | ||
| // | ||
| // Fallback to retrieve cachesize via /sys/.. if sysconf was not available | ||
| // for the platform. Currently musl and arm64 should be only cases to use |
There was a problem hiding this comment.
this comment needs to be updated to reflect what you are doing.
Maoni0
reviewed
Oct 5, 2024
|
|
||
| #if (defined(TARGET_LINUX) && !defined(TARGET_APPLE)) | ||
| { | ||
| // We expect to get the L3 cache size for Arm64 but currently expected to be missing that info |
There was a problem hiding this comment.
this comment needs to be updated to reflect what you are doing.
Maoni0
reviewed
Oct 5, 2024
|
|
||
| size_t size = trueSize ? s_maxTrueSize : s_maxSize; | ||
| if (size != 0) | ||
| return size; | ||
|
|
||
| size_t maxSize, maxTrueSize; | ||
| maxSize = maxTrueSize = GetLogicalProcessorCacheSizeFromOS(); // Returns the size of the highest level processor cache | ||
| maxSize = maxTrueSize = GetLogicalProcessorCacheSizeFromOS(useSysConf); // Returns the size of the highest level processor cache |
There was a problem hiding this comment.
do we really need to change the signature of GetLogicalProcessorCacheSizeFromOS? GetCacheSizePerLogicalCpu doesn't have any use of knowing this config value.
|
the rest looks okay to me.. would be great if @janvorli could take a look. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes: #76290
Problem Details
We recently discovered an issue that affects Unix based VMs where we are taking the host’s (as opposed to the VM’s) for certain AMD processor's last level cache size to be used in the GetLogicalProcessorCacheSizeFromOS call to discern the gen0 budget for both WKS and SVR. This is because
sysconf, the method we first try inGetLogicalProcessorCacheSizeFromOS, gets us the last level cache of the host machine as opposed to the fallback code path that reads the value of/sys/devices/system/cpu/cpu0/cache/index{LastLevelCache}/size. Consequently, the gen0 budgets are significantly different between Unix VMs and Windows using certain AMD processors on the same machine and the further implication of this is that we are probably setting much larger value than expected for the Gen0 budget for the GC running on Unix based VMs.The details from my v16 CPU based DevBox with an AMD EPYC 7763 64 Core Processor running Ubuntu 22.04.3 via WSL are as follows:
sysconfreturns the last cache size (L3) for the host machine (AMD EPYC™ 7763 – specs are here) as 256 MB.getconf -a | grep “LEVEL3_CACHE_SIZE”=>LEVEL3_CACHE_SIZE 268435456/sys/devices/system/cpu/cpu0/cache/index3/sizereturns 32 MiB, the same as the result from GetLogicalProcessorCacheSizeFromOS from Windows that calls GetLogicalProcessorInformation function (sysinfoapi.h) - Win32 apps | Microsoft Learn.L3: 32 MiB (1 instance)Solution
DOTNET_GCCacheSizeFromSysConfcan be set to 1 to revert to the current behavior i.e., using sysconf to obtain the last level cache.Performance Testing
Ran with the following GCPerfSim configurations for SVR:
-tc 28 -tagb 100 -tlgb 0 -lohar 0-pohar 0 -sohsr 100-4000 -lohsr 102400-204800 -pohsr 100-204800 -sohsi 0 -lohsi 0 -pohsi 0 -sohpi 0 -lohpi 0 -sohfi 0 -lohfi 0 -pohfi 0 -allocType reference -testKind time