Sorted all invocations of alloca

[AlexandreEichenberger]

Alloca was shown to be an issue with large benchmarks, where too much stack alloc ended up crashing the process. Alloca are typically used for temporaries needed within the computations of an onnx operation.

Went through all instances of alloca, and left alone the scalars (allocating a few bytes per operations is fine). For the larger data, I migrated it to alloc.

Because alloc results in a more expensive call to allocate and free, when not using parallelism, I moved such calls outside the loops. But when using parallelism, such call must be inside the parallel region as each parallel thread must have its version of temporaries, so in such cases the alloc stayed inside the parallel loop.

I ensured that the buffer hoisting did not bring the alloc outside of the scope_alloca, which seems to also catch the alloc operations. I ran individual checks on several key operations, as at this time the verifying to parallel operations in our CIs are still very sketchy.

  CheckONNXModel.py -m gemm.mlir -r="-O3 -march=arm64" -a="-parallel"
  CheckONNXModel.py -m reduceMin1.mlir -r="-O3 -march=arm64 -shapeInformation=0:1024x1024x1024" -a="-parallel"
  CheckONNXModel.py -m layernorm4d.mlir -r="-O3 -march=arm64 " -a="-parallel"

[tungld]

LGTM.

I remember we encounted segfaults because of alloca in the past and added the hoisting pass to move it outside loops. It's a pity to hear that that pass does not work with scope_alloca.

I see this pass in MLIR to promote small alloc to alloca, not sure if it works or not.

[tungld]

I often see in MLIR they use alloca to create a LLVM struct.

[AlexandreEichenberger]

Make sense, it's still small. Want me to remove the comment? @tungld

[tungld]

Yes, it's ok to remove it. Thanks!

[AlexandreEichenberger]

Changed it with an explanation of why it is good to have alloca here

[tungld]

If it is a good practice to use alignedAlloca for scalar type, should we add a verification in create.mem.alignedAlloca that checks type would be scalar. By that way, we can avoid the situation where users call create.mem.alignedAlloca for non-scalar type.

[AlexandreEichenberger]

Left for another PR.

[AlexandreEichenberger]

I remember we encounted segfaults because of alloca in the past and added the hoisting pass to move it outside loops. It's a pity to hear that that pass does not work with scope_alloca.

scope_alloca stop the migration of alloc and alloca out of that scope. It is important for parallel regions, as having it stay in the parallel region means that each thread will get its own. So that works fine.

Now I also manually moved the case with no parallel out of the loops, just to make sure that is fine. Would you like me to check if it's really needed? That way we could have the call always inside, and if parallel is there, its blocked and otherwise it is not?

In one case, I really need it outside manually, because it was inside of an inner-loop if-then-else. So by having it outside of the if-then-else (allocating in all case), it allows it to migrate to the outside.

[AlexandreEichenberger]

Responded to comments, and removed the placement of most "alloc" before/inside the main loop depending on parallel. Placing it inside works in both cases. If its parallel, it remains inside because of the inserted scope_alloca; if its sequential, then there is no scope and it naturally migrate outside the main loop.

[tungld]

If its parallel, it remains inside because of the inserted scope_alloca; if its sequential, then there is no scope and it naturally migrate outside the main loop.

Great, thanks! It was what I expected.

[jenkins-droid]

Jenkins Linux amd64 Build #15879 [push] Sorted all invocations o... started at 22:37

[jenkins-droid]

Jenkins Linux s390x Build #15882 [push] Sorted all invocations o... started at 23:37

[jenkins-droid]

Jenkins Linux ppc64le Build #14909 [push] Sorted all invocations o... started at 23:50

[jenkins-droid]

Jenkins Linux amd64 Build #15879 [push] Sorted all invocations o... passed after 1 hr 14 min

[jenkins-droid]

Jenkins Linux s390x Build #15882 [push] Sorted all invocations o... passed after 1 hr 34 min

[jenkins-droid]

Jenkins Linux ppc64le Build #14909 [push] Sorted all invocations o... passed after 2 hr 21 min