Examples for how to use this cookbook?

[luck02]

It would help allot if there were an example on how to setup and use this recipe.

[martinb3]

Hi there --

The best examples are in the test suites with test-kitchen:

suites:
  - name: default # server
    run_list:
      - recipe[java::default]
      - recipe[elkstack::default]

  - name: agent # java agent with server
    run_list:
      - recipe[java::default]
      - recipe[elkstack::agent]

  - name: forwarder # alternative golang agent with server
    data_bags_path: "test/integration/forwarder/data_bags"
    encrypted_data_bag_secret_key_path: "test/integration/forwarder/encrypted_data_bag_secret"
    run_list:
      - recipe[java::default]
      - recipe[elkstack::default]
      - recipe[elkstack::forwarder]

It shouldn't require any special setup for the most common use case. We'll work on a blog post or something similar that could elaborate on it. Thanks!

[Joseph-R]

Hey @martinb3 , thanks for the feedback!

So I'm in the same boat as @luck02 a bit, in that I am coming at this cookbook with completely fresh eyes, trying to figure out how to use it, and coming up with some questions. I'm going to pile on here, if that's cool, in the hopes that some of this stuff will be bundled up to the README.md or a blog post. I am happy to submit a PR for the former, btw, if that is desirable.

Anyways, I am trying to get this up in Test Kitchen to poke around a bit. I've got two VMs converged with the following kitchen.yml, adapted (very slightly) from the cookbook's kitchen.yml:

---
driver:
  name: vagrant
  network:
  - ["forwarded_port", {guest: 9200, host: 9200, auto_correct: true}]  # elastic
  - ["forwarded_port", {guest: 80,   host: 8080, auto_correct: true}]  # kibana
  - ["forwarded_port", {guest: 443,   host: 8443, auto_correct: true}] # ssl

driver_config:
  require_chef_omnibus: true

provisioner:
  name: chef_zero
  data_bags_path: '../../data_bags'
  environments_path: '../../environments'
  nodes_path: '../../nodes'
  roles_path: '../../repo/roles'
  client_rb:
    environment: development
  attributes:
    openssh:
      server:
        password_authentication: 'yes'
        permit_root_login: 'yes'
    cloud:
      public_ipv4: '127.0.0.1'
    authorization:
      sudo:
        users: ['vagrant']
        passwordless: true
    elasticsearch:
      network:
        host: '127.0.0.1'
      index:
        number_of_replicas: 0
        # so the kitchen node doesn't have unallocated replica shards
        # and comes up green (healthy) instead of yellow
      discovery:
        zen:
          minimum_master_nodes: 1 # since search returns more than one, but they are fake
    elkstack:
      config:
        iptables:
          enabled: true
        kibana:
          password: 'kibana'

platforms:
  - name: centos-6.5
    run_list:
    - recipe[yum]
    driver_config:
      box: centos_6.5_packer
      box_url: https://s3.amazonaws.com/o2o-public/centos_6.5_packer_provisionerless.box


suites:
  - name: elkserver
    driver:
      customize:
        memory: 3072
    run_list:
      - recipe[O2O-elk::single-node]
  - name: logtest
    driver:
    run_list:
      - recipe[O2O-elk::agent]

Where my wrapper cookbook recipes are just doing what you suggested above (installing java::default + elkstack::default on the server box, and java::default + elkstack::agent on the agent).

The deploys complete successfully, and I am able to access kibana at https://127.0.0.1:8443. But it doesn't look like my agent box is correctly configured to send data over to the server.

How can I best configure that for a local environment?

Versions:

OS: CentOS 6.5
Chef Development Kit Version: 0.6.2
chef-client version: 12.3.0
berks version: 3.2.4
kitchen version: 1.4.0

[martinb3]

Hi @Joseph-R -- test kitchen converges completely independent machines. Each test case on elkstack is a completely self-contained scenario. Test-kitchen shares no data between your agent and server test cases, so they'd never be sending log lines from the agent to the server. Each test case is completely independent and we'd never send logs from the agent test case to the server test case (they're separate tests, and just the way TK works).

For a local environment, elkstack uses chef search to find servers from agents, or it allows you to set a specific master server. If you have an example of an issue in that scenario, I'd be glad to take a look, but test-kitchen won't be a good scenario. It never has supported multi-server tests that can see/reference each other.