Bump plugins to get latest documentation changes

[jordansissel]

We've made some documentation changes in most plugins, and in order for those docs changes to ship with 5.6.0, we need to release every plugin that's had a docs change.

We didn't notice this in time for the 5.5.0 release, and on release day, we (mostly @dedemorton's hard work) do a lot of contortions in order to get the docs updated. It was manual and difficult work.

This is a blocker for 5.6.0 because I'm not going through that pain again :)

[jordansissel]

The tooling we have right now (thanks, @ph!) has certain expectations about how the plugin version is written, and there are a handful of plugins that use a different versioning system than the rest of the plugins

I'm going to try and fix that before moving forward on this. I'll keep this updated as I progress.

[jordansissel]

% egrep "version += +[^'\"][^0-9]" tmp/*/*gemspec  | fex /2 ':{2:}'
logstash-filter-dissect   s.version = DISSECT_VERSION
logstash-filter-useragent   s.version         = :File.read('version').split("\n").first
logstash-input-beats   s.version         = BEATS_VERSION
logstash-input-kinesis   spec.version       = Logstash:Input:Kinesis:VERSION
logstash-input-tcp   s.version       = :File.read('version').split("\n").first

The rest use s.version = 'some version string'.

[jordansissel]

https://github.com/jordansissel/logstash-mass_effect/tree/version-bumper

[jordansissel]

Patches pushed to branches, here's a sample:

• version bump for some docs fixes logstash-plugins/logstash-input-beats#254
• version bump for some docs fixes logstash-plugins/logstash-codec-cef#47
• version bump for some docs fixes logstash-plugins/logstash-input-file#160

[jordansissel]

@jsvd Can you take a look at some of these branches in the above comment?

I'm going to spot check a few others. All plugins should now have a patch-test branch that has the same changes applied (version bump + changelog).

Next steps for me to do after the reviews are completed:

1. Get these patches merged in plugins
2. Publish all plugins
3. Update Logstash 5.6 branch's gem lock to target new plugins

[jsvd]

@jordansissel the test PR's look good, same for your version-bumper branch.

One thing to note is that several plugins had their major versions bumped, and these will only be included in Logstash 6.0

For all of these it's necessary to branch off of the latest release tag and bump that branch.

We can see the major versions of plugins used by 5.5.1 by running the command below in
bin/logstash -i irb (we should be able to use bin/ruby but I couldn't get Bundler to set up correctly:

Bundler.environment.specs.select {|spec| spec.metadata["logstash_plugin"] == "true" }.map {|spec| [spec.name, spec.version.segments[0] ].join(",")}

From here we should be able to fetch the latest patch or minor release tag in each plugin repo, branch from there, and execute your bump_version

[jordansissel]

@jsvd yeah I had that thought; thank you for verifying! I'm expecting this to be a bit of a battle given the docs changes may have landed only in the new majors, but I will find out soon.

[jordansissel]

I'm pushing all the current changes to plugins now.

Next step will be to do the same patch bumps on all plugin minor versions targeted for Logstash 5.6

[jordansissel]

Unresolved

logstash-output-hipchat v4.0.3

master is v4.0.3 and needs to be published.

❌ bundle install fails: Gem::InstallError: httparty requires Ruby version >= 2.0.0.

❗️ I am going to skip this plugin. We should remove it from the docs until we resolve this?

logstash-output-jms v3.0.1

master is v3.0.2 and needs to be published

❌ bundle install fails: Gem::InstallError: semantic_logger requires Ruby version >= 2.1.

Resolved

logstash-output-rabbitmq v4.0.10

asciidoc typo fix on message_properties setting <<Hash,Hash>> to be <<hash,hash>>

• Fix asciidoc problem and do a version bump logstash-plugins/logstash-output-rabbitmq#69
• Publish plugin after Fix asciidoc problem and do a version bump logstash-plugins/logstash-output-rabbitmq#69 is merged

✅ Published logstash-output-rabbitmq-4.0.11-java.gem

logstash-input-graphite v3.0.3

This should be v3.0.4 but somehow it didn't get published in my last run. Jarvis won't publish it because the tests are failing on what seems like a flaky test - https://travis-ci.org/logstash-plugins/logstash-input-graphite/jobs/266105322

I'm having Jarvis ignore the test failure.

✅ published https://rubygems.org/gems/logstash-input-graphite/versions/3.0.4

logstash-output-datadog_metrics v3.0.1

master is v3.0.2 and needs to be published.

✅ Published logstash-output-datadog_metrics-3.0.2.gem

logstash-output-exec v3.1.1

master is v3.1.2 and needs to be published.

✅ Published logstash-output-exec-3.1.2.gem

logstash-output-lumberjack v3.1.3

master is 3.1.5 and needs to be published

✅ Published logstash-output-lumberjack-3.1.5.gem

logstash-filter-hashid v0.1.2

logstash-mass_effect lists this as a plugin that is manually removed. I think this is correct and this plugin should not show up in the documentation.

The hashid filter appears (to me) to have the same functionality as the existing fingerprint filter.

Anyway, we don't have time really to decide and execute on removing hashid, so here are the next steps:

• Docs fix and version bump logstash-plugins/logstash-filter-hashid#5
• Publish after Docs fix and version bump logstash-plugins/logstash-filter-hashid#5 is merged.

✅ Published logstash-filter-hashid-0.1.3.gem

logstash-output-kafka v5.1.9

needs an include_path fix, version bump, and release.

• Fix asciidoc include_path + version bump logstash-plugins/logstash-output-kafka#146
• Publish plugin after Fix asciidoc include_path + version bump logstash-plugins/logstash-output-kafka#146 is merged.

✅ Published logstash-output-kafka-5.1.10.gem

logstash-input-kafka v5.1.10

This needs an include_path fix and a corresponding version bump and release

• Review + Merge PR: Fix asciidoc include_path. Version bump.  logstash-plugins/logstash-input-kafka#226
• Publish plugin once the pr is good.

✅ Published logstash-input-kafka-5.1.11.gem

[jordansissel]

Next step is to remove the jms and hipchat outputs from the docs because these two plugins latest version's do not work in Logstash 5.x due to their depedencies requiring Ruby 2.x syntax.

[jordansissel]

Currently blocked on:

• Retries cherry pick 7.x logstash-plugins/logstash-output-elasticsearch#647
• and Regenerate lock file for 5.6 #8031 being updated with the latest plugin versions.

[jordansissel]

Blocked on logstash-plugins/logstash-codec-rubydebug#1

[jordansissel]

Next steps:

• ✅ (@suyograo) Update Regenerate lock file for 5.6 #8031 with latest gemfile lock
• (@jordansissel) Remove jms output from docs (for this reason) -- Or, someone fix the jms output plugin. Either works. One must be done before the 5.6 release.
• (@jordansissel) Remove hipchat output from docs (for this reason) -- Or, someone fix the hipchat output plugin. Either works. One must be done before the 5.6 release.

[jordansissel]

PR to remove jms and hipchat outputs: elastic/logstash-docs#408

[jordansissel]

I think this is resolved fully.

Some related work pending getting the docs published, but all the preparation work (version bumps, etc) in this issue is complete.