Moving to a pure-maven build

[chrisdutz]

Hi all,

as I spoke with several people, I would like to suggest to update the current build to a more standard maven build.
My main goal here would be, to allow providing integration modules for Apache PLC4X (https://plc4x.apache.org) which would bring a large number of protocol adapters for communicating with industrial hardware.

However I see here that several people would also like support of other IDEs for developing on Kura.

I would be willing to help with this effort, as on the Apache Sofware Foundation side, helping projects fix build issues with maven is sort of my speciality.

Happy to work with you guys.

Chris

[mattdibi]

Hi there @chrisdutz,

thanks for chiming in! We're extremely interested in the proposed changes. Welcome aboard! Feel free to ping us in this discussion for any support you might need!

[chrisdutz]

Could someone please explain to me this whole "target-platform" part of the build?
Also did I start migrating (in a branch in a fork of mine) ... this is going to be quite an undertaking, as I need to model the entire dependency structure via maven.
I did notice some things however, that I could need a bit of explaining:

• Why is there a separate test project for many modules? Then some of the test-modules have both src/main/java and src/test/java classes in them ... why is that?
• There are multiple mixtures of logging frameworks being used (I've spotted Log4J and SLF4J)
• Why are the artifacts(Submodules) of Kura all released via a mixture of version numbers ... this makes it super hard to maintain for users

[MMaiero]

The target platform is a concept that is needed by the OSGi / Tycho build to define the base environment. It contains the OSGi foundation where then the Kura applications will live. Not sure how this can be ported in a pure maven build.

Regarding this: "Why is there a separate test project for many modules? Then some of the test-modules have both src/main/java and src/test/java classes in them ... why is that?"
The choice to create a separate test folder is kind of inherited from the decision done by the original contributors. The separation between "src/main/java" and "src/test/java" in the tests is related to the fact that in "src/test/java" you have the unit tests while in "src/main/java" you have the integration tests that are run by Tyco plugins and spawn an OSGi environment where the OSGi components can perform service discovery and all the other OSGi capabilities to test that not only the class works but works in an OSGi environment.

There are multiple mixtures of logging frameworks being used (I've spotted Log4J and SLF4J)

To be checked. For what I remember we are using SLF4j to interface with log4j

Why are the artifacts(Submodules) of Kura all released via a mixture of version numbers ... this makes it super hard to maintain for users

Also here original choice by the original contributors. The idea, for what I was told, was that it could be possible to have a different lifecycle for the different components. So, if a bug is found, a patch can be issued and distributed in the form of a single jar instead of redistributing the whole framework. An upgrade path can then harmonize things

Hope this helps

[chrisdutz]

Regarding the decision to release individual parts separately ... We had several such projects over at the ASF. Mostly initiated in the really early days of Maven. But as far as I know all of them dropped that concept. It makes releasing things a lot more complicated but most of all for users it was super intransperrant to which versions work with which versions. In the end most decided to simply release all modules with each release, as it didn't really add more complexity to the release itself and it made consuming things a lot simpler.

[gnunzi]

Hi @chrisdutz,
maybe you don't need to migrate the entire build at once.
The heater example is using the maven-bundle-plugin already, no tycho compiler. You could use the same approach. In my opinion you should be able to use vscode right away, see picture below.

A small improvement could be to create a general parent pom (e.g. "kura-bundle-packaging" child of kura-pom) similar to the heater's pom and put it in a sub-directory (e.g. "/bundle-packaging"). This would ease working with vscode, since you would open the "bundle-packaging" directory with its sub-modules only.
Does it make sense? This could be a shortcut.

[chrisdutz]

Well my plan was to add the dependencies to the poms in a first step in order to get things to at least to compile and then to work on the osgi bundling, getting the tests to work step by step.

But the first step must be the dependencies.

[mattdibi]

@chrisdutz @gnunzi FYI we explored VSCode support for a bit and it works (for a broad definition of "works") with our current build system with a few quirks/adjustements. Take a look at #4573 if you're curious.

You can test it with these instructions. The patch is somewhat old but IIRC it still works.

[chrisdutz]

Yeah ... so my goal would be to make the build work without an IDE ... so the build should work with a simple "mvn package". So I've been adding hundreds of dependencies and I guess it will still take some time. Also have I been noticing some dependency cycles. So there's one maven module depending on artifacts that itself somehow depend on it. In OSGI-world this might work, but it's a show-stopper in maven-world. It would require splitting up artifacts into multiple modules.

I hope that I'll find the time to finish adding at least the dependencies first ... then in a next step I'll look into the cycles.

[mattdibi]

@chrisdutz ...uh... the build works without an IDE. Did you look at the build instructions? No IDE installation is required to perform the maven build alone (the only requirements are listed here). Did we miss something?

[chrisdutz]

Well I think I tried building it without IDE and it couldn't get it working ... mabe I did something wrong? I mean ... that was actually my intent in proposing the maven update.

[mattdibi]

I'm sorry to hear that. There must have been some miscommunication on our end. In case you want to give it another spin the instruction are what I already linked you:

• Requirements
• Build instructions
• Apple M1 caveats

Our team uses multiple OSes (currently Mac, Linux and Windows) and architectures (x86, arm64) and we're all able to build it without issues. Let us know if you encounter any issue with the build process.

[chrisdutz]

#5536

So this is where I'm currently working on things ... however i have no idea how many more hours I'll be adding dependencies ;-)

A bit unfortunate, that OCX brought me closer to you folks now any not a year or two earlier, as I'm currently totally loosing my open-source drive after 8 years of the industry constantly taking and never giving anything back ... so I might prioritize things down a bit on my end seeing how huge this project is and how long it would probably take to bring it into the form I usually bring other projects to.

[marcellorinaldo]

Hi Christofer! I don't think that this PR is actually needed. All the dependencies are specified in the target platform, there is no need to duplicate them in the projects. Moreover, some internal package exports will not be found on maven central. This is how OSGi works and this is the reason why you should compile with maven plugins like Eclipse Tycho and bndtools.

You were mentioning that the build is not working properly on you side, which of steps here (README) are failing at your end?

[chrisdutz]

No worries ... I doubt I will continue this endeavour as I have already invested many hours into updating the Maven dependencies. And it feels like I'm not reaching the end ... also has my initial work shown many cycles, which would be a nightmare to resolve.

As I mentioned in another thread. Over the last 8 years of putting all my energy into industrial automation, I am growing tired and am thinking of giving up.

I will definitely not participate in any project that requires me to use a development environment and doesn't let me choose the one I'm familiar with, so also don't expect a PR for adding PLC4X from my side.

[marcellorinaldo]

An integration with PLC4X could be accomplished with what @gnunzi was proposing here #5521 (comment). It should be fairly simple using the maven-bundle-plugin

[mattdibi]

I will definitely not participate in any project that requires me to use a development environment and doesn't let me choose the one I'm familiar with

@chrisdutz @gnunzi I think you guys might be interested in #5563. The setup is still experimental but I achieved some promising results. Take it for a spin if you're interested (instruction are available in the PR description).