Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Some benchmarks #38

Open
certik opened this issue Dec 21, 2023 · 0 comments
Open

Some benchmarks #38

certik opened this issue Dec 21, 2023 · 0 comments

Comments

@certik
Copy link
Contributor

certik commented Dec 21, 2023 •
edited
Loading

Here are some benchmark results.

array_09.cpp

Source code: https://github.com/lcompilers/lc/blob/f6548855009827d57398f5335ca629b712b1a8be/integration_tests/array_09.cpp

An equivalent Fortran code: https://github.com/lfortran/lfortran/blob/0425cf2c0b988762db49033bfa9696483c345c69/integration_tests/arrays_op_17.f90

Using Apple M1 Max.

Compilation Speed

$ time clang++ -I$CONDA_PREFIX/include integration_tests/array_09.cpp -o x.clang
clang++ -I$CONDA_PREFIX/include integration_tests/array_09.cpp -o x.clang  0.99s user 0.07s system 100% cpu 1.048 total
$ time lc --extra-arg-before="-I$CONDA_PREFIX/include" integration_tests/array_09.cpp -o x.lc
Error while trying to load a compilation database:
Could not auto-detect compilation database for file "integration_tests/array_09.cpp"
No compilation database found in /Users/ondrej/repos/lcompilers/lc/integration_tests or any parent directory
fixed-compilation-database: Error while opening fixed database: No such file or directory
json-compilation-database: Error while opening JSON database: No such file or directory
Running without flags.
lc --extra-arg-before="-I$CONDA_PREFIX/include" integration_tests/array_09.cp  0.71s user 0.05s system 98% cpu 0.772 total

LC is 1.36x faster.

For a comparison, LFortran compiles the equivalent code in:

$ time lfortran arrays_op_17.f90 -o x.lf
lfortran arrays_op_17.f90 -o x.lf  0.06s user 0.01s system 95% cpu 0.076 total

So another 10x speedup is possible by using our own custom parser for a subset of C++.

Further speedup is possible by not using LLVM.

Binary Size

$ ll -h x.clang 
-rwxr-xr-x  1 ondrej  staff   1.2M Dec 21 16:32 x.clang
$ strip x.clang 
$ ll -h x.clang
-rwxr-xr-x  1 ondrej  staff   1.0M Dec 21 16:32 x.clang
$ ll -h x.lc
-rwxr-xr-x  1 ondrej  staff    33K Dec 21 16:29 x.lc

LC generates 32.3x smaller binary.

Runtime Speed

$ time ./x.clang
{2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2}{{4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4}}{8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8}
{{34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34}}
./x.clang  0.01s user 0.00s system 78% cpu 0.014 total
$ time ./x.lc
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 

8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 

./x.lc  0.00s user 0.00s system 62% cpu 0.008 total

LC binary runs 1.75x faster.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@certik