Add a script for generating the quick-tuning perfconfigs list #1689

djramic · 2024-10-28T13:27:15Z

This PR includes a script for finding quick-tuning perfconfigs in the following way:

For each tuning problem, it finds the top n perfcofnigs based on a given threshold
Using a linear programming model, it identifies the smallest subset of perfcofnigs that covers all problems.

Additionally, there are option to automatically generated the QuickTuningPerfconfigs.inc file that contains selected perfconfigs, and its integration into the codebase.

closes : ROCm/rocMLIR-internal#1641
closes : ROCm/rocMLIR-internal#1258
closes : ROCm/rocMLIR-internal#1518

…odebase

codecov · 2024-11-18T18:05:46Z

Codecov Report

Attention: Patch coverage is 74.00000% with 13 lines in your changes missing coverage. Please review.

Project coverage is 77.70%. Comparing base (5f51701) to head (738bbaf).
Report is 15 commits behind head on develop.

Files with missing lines	Patch %	Lines
...lir/lib/Dialect/Rock/Tuning/GridwiseGemmParams.cpp	74.00%	11 Missing and 2 partials ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #1689      +/-   ##
===========================================
- Coverage    77.76%   77.70%   -0.07%     
===========================================
  Files          100      100              
  Lines        27866    27897      +31     
  Branches      4063     4072       +9     
===========================================
+ Hits         21671    21678       +7     
- Misses        4540     4552      +12     
- Partials      1655     1667      +12

Flag	Coverage Δ
mfma	`77.70% <74.00%> (-0.07%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

dhernandez0 · 2024-11-18T10:44:27Z

mlir/utils/performance/analysis/quickTuningGen.py

+
+    def get_top_n_perfconfigs_per_problems(self, df, targetColumns):
+        """
+        Identifies the top perfcofnigs for each problem based on a threshold


perfcofnigs -> perfconfigs

dhernandez0 · 2024-11-18T10:46:03Z

mlir/utils/performance/analysis/quickTuningGen.py

+        """
+        Finds the minimal set of perfconfigs that cover all
+        problems using set cover optimizaiton.
+        Returns : A dictionary containing data types as keys and thier


thier -> their

dhernandez0 · 2024-11-19T08:16:48Z

mlir/test/Dialect/Rock/test_packed_arithmetic.mlir

why does this PR change this? and other tests?

Since affix-params gathers perfconfigs from the list, it is needed to adjust the tests after updating the list

dhernandez0 · 2024-11-19T08:22:13Z

mlir/lib/Dialect/Rock/Tuning/GridwiseGemmParams.cpp

-    params = {initParameters, nInitParameters};
+  if (opType == KernelType::Gemm) {
+    switch (dataTypeA.getIntOrFloatBitWidth()) {
+    case 8:


this would apply to both f8 and i8 here, is that the goal?

Here I 've just add select conv or gemm list based on opType. I think we're using non-accel configs for fp8

dhernandez0 · 2024-11-19T08:25:56Z

mlir/include/mlir/Dialect/Rock/Tuning/QuickTuningPerfconfigs.inc

+
+// BEGIN_GEMM_Wmma_i8_DECS
+static constexpr size_t nInitParametersForward8BitGemm = 15;
+static const InitParamsAccel initParametersForward8BitGemm[nInitParametersForward8BitGemm];


is this only for forward? otherwise we can keep names consistent.

Forward is relevant for conv, but for gemm it doesn't seem quite right to keep it there. I'll correct that.

dhernandez0 · 2024-11-19T08:46:10Z

mlir/utils/performance/analysis/quickTuningGen.py

+    parser.add_argument("--th",
+                        required=False,
+                        type=float,
+                        default=0.93)


is 0.93 the number used here? from what I heard 0.95 is what we consider noise, by using 0.93 we could be getting worse performance?

That's correct, but as I understood it, we needed faster quick-tuning, and it was acceptable to sacrifice a few percent of performance. So, I decided to extend on 0.93 since it noticeable reduced the number of configs.

dhernandez0 · 2024-11-19T08:48:20Z

mlir/utils/performance/analysis/quickTuningGen.py

+
+            # Create coverage matrix
+            A = np.zeros((n, m), dtype=int)
+            for problem, perfconfig_list in problems_to_perfconfigs.items():


nit: just enumerate here and don't use problem_to_index

dhernandez0 · 2024-11-19T08:48:26Z

mlir/utils/performance/analysis/quickTuningGen.py

+            A = np.zeros((n, m), dtype=int)
+            for problem, perfconfig_list in problems_to_perfconfigs.items():
+                i = problem_to_index[problem]
+                for perfconfig in perfconfig_list:


An indexing map is required here to provide a global index for the perfconfig, rather than using the index from the given problem

dhernandez0 · 2024-11-19T08:49:16Z

mlir/utils/performance/analysis/quickTuningGen.py

+            # Linear programming model to minimize the number of perfconfigs
+            prob = pulp.LpProblem("SetCoverProblems", pulp.LpMinimize)
+            x = pulp.LpVariable.dicts("x", range(m), cat='Binary')
+            prob += pulp.lpSum([x[j]] for j in range(m))


nice! could you add a comment here to explain, I think we are setting the objective function here, right?

dhernandez0 · 2024-11-19T08:49:28Z

mlir/utils/performance/analysis/quickTuningGen.py

+            x = pulp.LpVariable.dicts("x", range(m), cat='Binary')
+            prob += pulp.lpSum([x[j]] for j in range(m))
+            for i in range(n):
+                prob += pulp.lpSum([A[i][j] * x[j]


same here, I guess these are the constraints?

djramic requested review from jerryyin and sjw36 as code owners October 28, 2024 13:27

djramic requested review from stefankoncarevic, dhernandez0 and umangyadav November 15, 2024 15:24

djramic added 4 commits November 18, 2024 09:49

Add script for generate quick-tuning data

abbfac4

Added quick-tuning perfconfigs .inc file and integrated it into the c…

2601cae

…odebase

Corrected tests

06834a1

Correct clang-format

588592a

djramic force-pushed the quicktuning_gen_script branch from bf24e45 to 588592a Compare November 18, 2024 10:20

dhernandez0 reviewed Nov 19, 2024

View reviewed changes

djramic requested a review from causten as a code owner December 10, 2024 18:03

djramic force-pushed the quicktuning_gen_script branch from a478ee0 to 738bbaf Compare December 11, 2024 19:38

Addressing review comments - 1

8e5cf9c

djramic force-pushed the quicktuning_gen_script branch from bbefcf6 to 8e5cf9c Compare December 12, 2024 14:20

causten requested a review from dhernandez0 December 12, 2024 15:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a script for generating the quick-tuning perfconfigs list #1689

Add a script for generating the quick-tuning perfconfigs list #1689

djramic commented Oct 28, 2024 •

edited

Loading

codecov bot commented Nov 18, 2024 •

edited

Loading

dhernandez0 Nov 18, 2024

dhernandez0 Nov 18, 2024

dhernandez0 Nov 19, 2024

djramic Dec 10, 2024

dhernandez0 Nov 19, 2024

djramic Dec 10, 2024

dhernandez0 Nov 19, 2024

djramic Dec 10, 2024

dhernandez0 Nov 19, 2024

djramic Dec 10, 2024

dhernandez0 Nov 19, 2024

dhernandez0 Nov 19, 2024

djramic Dec 12, 2024

dhernandez0 Nov 19, 2024

dhernandez0 Nov 19, 2024

Add a script for generating the quick-tuning perfconfigs list #1689

Are you sure you want to change the base?

Add a script for generating the quick-tuning perfconfigs list #1689

Conversation

djramic commented Oct 28, 2024 • edited Loading

codecov bot commented Nov 18, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

djramic commented Oct 28, 2024 •

edited

Loading

codecov bot commented Nov 18, 2024 •

edited

Loading