The sustainability/health of Lightbend config is in question

[mdedetrich]

Pinging @pjfanning , @johanandren and @jrudolph

Upstream issue is here lightbend/config#787 and my response is here lightbend/config#787 (comment). We should probably discuss a gameplan here for a worst case scenario, i.e. should we end up spending our capacity maintaining config considering its a pretty critical dependency or should we try and move to something like https://github.com/ekrich/sconfig (which would also make it easier to open up Pekko for Scala.js/scala-native)

[JD557]

Just to clarify, whatever the decision, that would be for 1.1.0, right? It would be pretty nice to be able to migrate one thing at a time.

(I'm probably just stating the obvious, but it's better to play it safe and ask anyway)

[mdedetrich]

We are definitely not changing this for 1.0.x, thats out of the question.

[pjfanning]

We have no evidence that Pekko or Akka users are hitting major problems. I would be a -1 on doing a Pekko fork of config and -0.9 on changing Pekko away from using config jar in the foreseeable future.

[mdedetrich]

That may be fair, however there are definite concerns regarding the sustainability and maintenance of the Lightbend specific libraries falling under the Akka umbrella, there are even pull requests for known bugs which are being ignored. It isn't really good to have critical dependencies in such a state.

To be clear, no such change will happen any time soon and the best course of action is to see and wait. Of course we aren't hitting major problems, but if we do its essentially a library out of our control. From the response here lightbend/config#787 (comment) its kind of evident that that library cares about Akka's concerns, not anyone elses and over time this become and increasing issue as Pekko deviates further from Akka (and especially the case if we want to integrate things like Akka.js).

To me its pretty clear its not really an open source library in spirit

[pjfanning]

One thing to bare in mind, is that Pekko is the result of the license change. Taking a code base into Apache that is still Apache licensed and not officially retired will have different complications.

If Lightbend could be persuaded to donate it to the ASF then maybe a new Incubator podling could be formed around it.

[mdedetrich]

I am also not that eager to pull in config into ASF, this to me is one of our last resorts. Ideally it would become a community effort with some maintainers from our committer team.

https://github.com/ekrich/sconfig does look like an attractive option, in the sense its also being used for Scala critical software (metals uses it) and it solves other problems as well. They even provide a migration to go from typesafe config to sconfig (see https://github.com/ekrich/sconfig#migrating-an-existing-lightbendconfig-project-to-sconfig)

[pjfanning]

There is further evidence that typesafe/config is not being maintained. PRs from outside contributors are not being merged. Example - lightbend/config#797

If we were to create a Pekko version, I suspect we could have issues because the license has not changed and we are unlikely to be able to persuade Lightbend to donate it to the ASF (but we can ask, I guess).

We don't tend to expose typesafe/config classes in APIs so it should be feasible (in theory) to fork the code and change package names and just use the new libs from our fork - in a way that should have very limited impact on our users.

Not something that we would uptake in any 1.0.x patches but possibly in 1.1.x minor releases.

Is there an appetite to pursue this?

[mdedetrich]

I would much prefer to use sconfig then the proposed solutions, its actually being used in Scala projects as a drop in replacement for typesafe config, has a migration tool to convert code from typesafe config to sconfig and to boot it also supports other platforms (not relevant now, may be relevant in the future).

The most critical thing is that its properly maintained.

Now there may be some cases where there is divergent behaviour and/or missing features, but considering that sconfig is maintained I am sure they will accept contributions upstream and in any case its likely we will do this gradually rather than all at once (i.e. have both sconfig and typesafe config as we iteratively migrate to sconfig).

A primary motivation for this reasoning is to also stop the NIH syndrome, we really don't have the capacity to maintain forks

[JD557]

Another argument for sconfig is the cross compilation aspect. I believe going with that would make it much easier to have pekko.js/pekko native.

[jrudolph]

sconfig is looking very nice but I would still be very wary to make such a large change right now (or maybe ever). I can understand the desire to have more actively maintained upstream dependencies but I cannot see the scenario where typesafe/config suddenly becomes unsuitable. I find Havoc's comments in the linked thread quite level-headed. There is always an economic aspect to software development and especially with the scarce resources in OS development, so having upstream dependencies becoming mature to the level of only providing the minimal amount of updates is expected and actually a good thing for stability.

On the Pekko side, despite the compatibility of sconfig with the actual config files this might still be quite a big chunk of work because configuration matters are so spread over all the codebases. Also consider, that the config API is part of Pekko's public API, so we would either have to opt for just breaking compatibility or have a massive and potentially complicated transition period where we deprecate but still support the public APIs based on the existing config API.

On a more general note, can we try to keep the discussion a bit less extreme? We are not the military and have to worry about potential "worst case scenarios". But playing with this, the worst case would be that the library is deleted from Maven Central, which just won't happen. A critical security issue or serious incompatibility with an upcoming Java version is also not to expect, so we are likely good for the next decade (we can also probably go back to the config version published a decade ago, and that would also work).

[mdedetrich]

To put my stance down on the table, I also think its premeture to do such a drastic change to sconfig right now. As you said the current config is part of the public facing API which means that any such change would have to be tabled for Pekko 2.x.x. My view on the matter is making a fork doesn't really achieve anything, in the sense that it creates as many problems as it solves.

That being said, I don't entirely agree with the premise that "we should keep things as they are". In my view I really would like Pekko to shift away from the NIH syndrome that it inherited from Lightbend, after all its a community run project and not being headed by a single company. Moving to sconfig helps facilitate this goal even if it is a bigger undertaking.

In summary, any effort that lets us offload parts of Pekko to the scala community (which sconfig is inside of, arguably its one of the best examples of a well maintained Scala OS project) should be taken seriously, and its also in the spirit of community driven OS.