[MINOR] improve(server): Add debug log when cacheShuffleData #2156
Conversation
| @@ -71,4 +77,8 @@ public ShufflePartitionedBlock[] getBlockList() { | |||
| public long getTotalBlockSize() { | |||
| return totalBlockSize; | |||
| } | |||
|
|
|||
| public long getTotalBlockLength() { | |||
There was a problem hiding this comment.
What's the difference of block lenght and block size?
There was a problem hiding this comment.
public int getSize() {
return length + 3 * 8 + 2 * 4;
}The block size is larger 3 * 8 + 2 * 4 than block length which instead of encoded length of this block.
|
Actually it's weird that we have size and length at the same time. Maybe we should have a better name for them. |
|
@jerqi Yes, do you think |
maobaolong
force-pushed
the
addDebugLogForCacheShuffleData
branch
from
071d5c5
to
8c4a4a3
Compare
Better than now. |
Encoded length is larger than data length, isn't it? |
|
@jerqi Yes, encoded length include data length and data structure size. |
| for (ShufflePartitionedBlock block : this.blockList) { | ||
| size += block.getSize(); | ||
| encodedLength += block.getSize(); |
There was a problem hiding this comment.
Could we modify the name in the ShufflePartitionedBlock, too?
| @@ -20,31 +20,37 @@ | |||
| import java.util.Arrays; | |||
|
|
|||
| import com.google.common.annotations.VisibleForTesting; | |||
| import org.apache.commons.lang3.tuple.Pair; | |||
| import org.apache.commons.lang3.tuple.Triple; | |||
There was a problem hiding this comment.
Could we extract a class instead of using Tripple?
There was a problem hiding this comment.
I abstract a class named ShufflePartitionedBlocksInfo
maobaolong
force-pushed
the
addDebugLogForCacheShuffleData
branch
from
0d96f95
to
723c602
Compare
|
@jerqi Thanks for your suggestion, addressed it by the newest commit, PTAL. |
…ce new data structure
|
@jerqi Thanks for the approved, i push a new commit to avoid introduce new data structure, hope this can be better? WDYT? |
…2156) ### What changes were proposed in this pull request? Add debug log when cacheShuffleData ### Why are the changes needed? Supply a way to know the cacheShuffleData detail, include the partitionId, blockCount, we can know well from this debug level log. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? No need.
What changes were proposed in this pull request?
Add debug log when cacheShuffleData
Why are the changes needed?
Supply a way to know the cacheShuffleData detail, include the partitionId, blockCount, we can know well from this debug level log.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
No need.