OSS Energy Efficiency of Programming Languages

[mkorbi]

The results of the https://github.com/greensoftwarelab/Energy-Languages , which is based on scientific research, are often used in various discussions and presentations. A minor update was provided by 2020. The testing is based partially on Benchmarkgames.

The proposal would be to fork the repository (as the activity is very low and PRs are not managed | or ask if they would like to "donate" it) and initially update the given programming languages. If we believe something is missing we can give this as a good first issue to the community.

Why this is relevant?
In the context of the energy efficiency of programming languages, these results are very often presented. The problem or critics are that the language versions are outdated, younger languages especially improve their effectiveness from release to release etc. Also, the results are often wrongly used and taken out of context.

Caveat
Does it provide actually any "high" value to the community?

Our Chance
Not only provide some nice tables and the results hidden behind a science paywall but gives a proper categorization. So people can take the results plus a qualified description for their communication.

[wojtekidd]

Also what we could do is make sure that we maintain and update this data with every new release of the language that comes up - especially the most widely used 5-10 languages.

[nikhil-vytla]

I'm all for keeping benchmarks as current as possible! Do we know how much time/effort/compute power it would take to fully benchmark each language? If the overhead is very intensive, should we consider staggering data updates (ex. update data once every N new releases)?

[catblade]

And likely these benchmarks will deviate according to what hardware they are running on too, so need full down-to-the-hardware plan, along with a place to publish these and contribute to them. Equinix has a nice platform where we have access to mulitple hardwares, and they have an interest in this space, so negotiating with them for testing may do well.

These discussions I'd like to see in the mailing list, because I don't see easy ways to tag the Equinix folks, but I do see them in the calls.

[vielmetti]

Registering interest here from an Equinix point of view, @catblade you can tag me if there's a plan. (More comments to the nature of that plan, below).

[mkorbi]

I reached out to the owner of the Energy-Language repo to understand their plans and future approaches. Hopefully will get some feedback and we can collab here 🥇

[mkorbi]

Update: One of the main contributors doesn't work on this anymore and doesn't have responsibilities. But he guided me to his professor. Dropped him a message too.

[leonardpahlke]

I would question this effort if this is one of our main concerns/goals. It is definitely an interesting project, but if this is a project for the wg, I would put it back, given the other ideas seem more feasible in the near future as well.

Dynamic analysis of energy consumption per application is feasible (static analysis is not, I suspect). We need to understand why some languages consume more or less energy in certain situations, so we need deep analysis (like an energy debugger), which complicates this effort enormously. Now, if we consider more or less all common programming languages, this becomes even more difficult and increases the effort considerably.

You may have something else in mind, but should we do what I outlined above, I think it will take a long time (and we may get stuck). If we want to make it happen, we could think about collaborating with a university, but even then there is not really an advantage for the university to collaborate with us :D

I think it's a good idea, but shouldn't we focus on something else first?

[vielmetti]

To reanimate this a little -

There is near-universal interest among programming language developers in having their tools and tooling go faster. Sometimes you through more kilowatts at the problem and provide more hardware to do your compute (compile and test), other times you get efficiencies from better use of the processor architecture by using all of the features or implementing better/faster/more clever algorithms.

I don't want to get into a battle of "which programming language is best" - horses for courses, there are lots of reasons to pick one over the other - but within any given programming language I think there's room for targeted benchmarking, regression testing, and general picking-apart of how energy usage changes between releases, and if there are regressions have some useful way to catch and capture and give feedback on that.

[catblade]

@vielmetti Hello! And Happy New Year!

I agree with not getting into a "which language is best" battle, BUT I would like us (at some point, even not us, but someone) to put together a library, similar to that of CVEs, of efficiencies for certain types of functions, and then play games like that of this project here: https://gitbom.dev/ , treating efficiency changes like bugs (or security deficiencies).

I have no idea where this work should take place, but would love it to assist.