This gem contains code for sending hourly updates on certain key tables to our AWS based Open Search cluster
gem 'business_atomic_events', git: '[email protected]:eLocal/business_atomic_events.git
gem 'flipper'
You may also want to constrain usage of the gem to production environment only.
Add config/initializers/business_atomic_events.rb
file containing the following code to make sure
your application has access to the credentials required to communicate with Open Search:
# frozen_string_literal: true
if Rails.env.production?
require 'ssm_to_env'
SsmToEnv.load!(
'/service/business_atomic_events/open_search/production/password' => 'OPEN_SEARCH_PASSWORD',
'/service/business_atomic_events/open_search/production/user_name' => 'OPEN_SEARCH_USER'
)
end
- Ensure existence of Open Search indices that will receive data fed by this gem. Please refer to https://aws.amazon.com/opensearch-service/getting-started/ for details. Our existing Open Search cluster is here: https://search-john-test-duj2npqk5k27ffd2uwgb52rm7i.us-east-1.es.amazonaws.com`. Index names should start with index prefixes which need to be the same for all indices of an application. For our major applications. Open search index name's structure is
<index_prefix>_<index>
, where bothindex_prefix
andindex
are reflected in the YAML configuration file described below in (2). - Create a configuration file
config/business_atomic_events.yml
which will define queries to populate Open Search indices (one query per index) and specify a target index for each query. A good example is the affiliates application's configuration file you can find here:https://github.com/eLocal/affiliates/blob/master/config/business_atomic_events.yml
. - Ensure that a periodic (cron-based) job named
BusinessAtomicEvents::FeedGenerator
runs a few minutes past each hour.
Flipper gem is a required dependency.
- Feature
bae_current_feeds
needs to be enabled for current hourly feeds generation. - Two or more features are required to be enabled for generation of historic feeds. One of them is
bae_history_feeds
. Additionally, for each index that needs historic feeds generation, enable a feature namedbae_history_<index-name>
. E.g, if an index inbusiness_atomic_events.yml
configuration file is listed asoutbound_call_pings
, then the feature to enable isbae_history_outbound_call_pings
.
While current feeds will be generated automatically via BusinessAtomicEvents::FeedGenerator
cron-based job , historic feeds need to be triggered manually. Log into one of the production servers via ssh and start rails console from the application directory via bin/rails c
command. Then, at the console prompt, start the historic feeds job for the required date interval, e.g.:
BusinessAtomicEvents::SpawnHistoricFeeds.perform_async('2022-01-01', '2022-10-19')