Add timestamps to built packages enabling filtering integrations by update/release date

[kpollich]

From @jamiehynds

At the rate we ship new integrations, and update existing ones, it's a real challenge for users to discover new integrations (and for us to raise awareness of new integrations). Users have to track PR's in the integrations repo in order to figure out which integrations have just shipped.

I'm sure it's been discussed in the past, but is there any open issue to add values to packages such as release_date and last_updated ? We could then use these dates in the integrations UI (and possibly website) to allow users to sort by newest integration and/or a New category which auto-updates with recently released integrations.

We also see questions like "what integrations are on our roadmap?" or "for what services do we have upcoming integrations" somewhat frequently in Slack or discuss posts, and I think building some kind of timestamp metadata into integrations could help us answer these questions, e.g.

• Render a list of prerelease integrations sorted by newest first
• Render a list of prerelease integrations sorted by "nearest version to 1.0.0" first

As far as implementation goes, I think we'd be able to derive these timestamps based on the last commit for each integration and include it in a yml artifact generated by the integration build process, or simply in the changelog.yml file. Right now, the integrations UI in Kibana doesn't "parse" the changelog at all, but we do render it directly in a modal + code block.

[jsoriano]

We have an issue about adding build metadata in packages (elastic/package-spec#446). This metadata could also include the build time. In the case of published packages the build and publication time would not be the same but would be pretty close.

For getting the latest published packages, I could imagine generating a RSS file on the publishing pipelines. This file could be consumed by kibana or directly by users. We could generate files with or without prerelease packages.

Render a list of prerelease integrations sorted by "nearest version to 1.0.0" first

I don't think it is possible to automate this, as this would require an understanding of the roadmap and its status.