Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collector 3.x and Enrich 4.x #145

Closed
wants to merge 13 commits into from
Closed

Collector 3.x and Enrich 4.x #145

wants to merge 13 commits into from

Conversation

pondzix
Copy link
Contributor

@pondzix pondzix commented Feb 15, 2024

New collector and enrich versions mean:

  • Cats Effect 3
  • Http4s as web server, so no more Akka HTTP
  • Using IO monad instead of Id

To be consistent with other Snowplow applications, this commit also brings small improvements to configuration loading. Just like collector configuration, iglu resolver and all enrichments support HOCON configuration format.

As collector 3.x is on scala 2.13 and enrich 2.12, it was easier to cross compile collector to 2.12 than upgrade enrich to 2.13. That's why micro uses collector from this branch -> https://github.com/snowplow/stream-collector/tree/3.1.0_cross_scala_2.12 and is based on scala 2.12

} yield ()
}

private def setupSSLContext(): IO[Unit] = IO {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've always found java keystores a bit pain to generate and manage. There is nothing wrong with this approach but I wonder if one day we should add pem support to ease on the user-side of SSL support. It shouldn't be big of a deal. Given you're AFK I can have a stab at it with the subsequent try work.


object MicroAdapterRegistry {

private val adaptersSchemas = AdaptersSchemas(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this not be a problem if we update this inside Enrich (which can be done with configuration now) and it does not match?

Copy link
Contributor Author

@pondzix pondzix Feb 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it to load schemas from defaults provided by Enrich instead of hardcoded values in micro 0d64df1

@pondzix
Copy link
Contributor Author

pondzix commented Feb 16, 2024

Part of #147

@pondzix pondzix closed this Feb 16, 2024
@pondzix pondzix deleted the micro_http4s branch April 9, 2024 07:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants