sherlock-1.12

• generalized and expanded Sherlock with a new anomaly detection pipeline, which uses Meta's Prophet for prediction
• added query function for the Prophet Service to support Batch Query feature
• added DetectorException to handle HTTP exceptions
• added Prophet-related parameters/CLI arguments
• refactored EgadsConfig to Detector Config; refactored EgadsResult to DetectorResult
• optimized the original Egads detection pipeline (~12% speed up)
• fixed original pipeline bug such that Sherlock generates an array of empty lists instead of nulls when the input source is empty
• fixed original pipeline bug such that Sherlock runs user selected Time Series/Anomaly Detection models for cronJob/backfillJob
• fixed original pipeline bug such that NaiveModel is working
• modified web UI to correspond with backend changes
• added Jupyter Notebooks to compare Egads/Prophet pipeline results ('forecaster_comparison' folder)
• unit tests and comments; fix typos

sherlock-1.11

• Moved packaging to Maven Central at https://repo1.maven.org/maven2/com/yahoo/sherlock/sherlock/
• Added Create Job button on Flash query page to add it as a Job that runs periodically

sherlock-1.10

• Added job level SLA. Job level SLA defaults to the SLA of the selected cluster but user can now change it as per job's datasource availability
• Druid cluster is now modifiable for any existing job
• Bugfix: proper updates for clusterId, jobId and emailId while doing CRUD operations
• Added Index rebuild button under meta manager tab to build the redis indexes for clusterId, jobId and emailId on an ad hoc basis

sherlock-1.9

Adding mTLS authentication for druid query/status REST APIs

• added new CLI settings params for mTLS connectivity.
  • truststore-path, truststore-type, truststore-password, keystore-path, keystore-type, keystore-password, key-dir, cert-dir, https-hostname-verification, custom-ssl-context-provider-class, custom-secret-provider-class. Refer README.md for usage of each of the params.

sherlock-1.7

add settings.xml

sherlock-1.6

Previously, the anomaly task, email sender task and redis data backup task were part of a single thread. New implementation uses custom implementation of ScheduledThreadPoolExecutor to manage above tasks in separate threads running periodically. New implementation also reschedules the scheduler thread in the case of ExecutionException thrown from the task during runtime rather than dying silently. It uses default implementation of ScheduledExceptionHandler which always reschedules upon exception. One can use custom version of this ScheduledExceptionHandler interface to allow limited resubmissions of a thread.

sherlock-1.0.4

Issue #18 : updated maven surefire version to 3.0.0-M3

Description of changes:

1. Previously all the job reports are set instantly after the job completion. Now, per job email metadata is added in redis to keep track of email -> job & email -> reports relationship to trigger emails, about available job reports in redis, at user specified time. User can go to meta-manager UI and manage the email trigger time per email.

2. Added support for redis back up in two ways: 1) is backing up redis dump in local redis dump file periodically by executing bgsave explicitly every day midnight. 2) user can use --backup-redis-db-path CLI config(refer README) to specify redis json dump location to store redis data in json formated file. In the case of redis data loss user can use /Debug/Restore endpoint to submit the json data file location to restore redis data.

3. For security, user can specify allowed druid broker urls to connect using a whitelist file. --druid-brokers-list-file CLI setting(refer README) can be used to specify the location to whitelist file.

4. Bugfix in the case where druid query grouping dimension can be a json object(javascript function). Now query parser checks for json object and allows it as a group by dimension.

I confirm that this contribution is made under the terms of your choice and that I have the authority necessary to make this contribution on behalf of its copyright owner.

sherlock-1.0.3

• added multi-select for backfill jobs
• bug-fix for monthly job scheduling
• added detection window config on UI for flash query
• enhanced filter for invalid time-series
• added custom granularity feature for instant anomaly jobs, scheduled jobs and backfill jobs
• 1 minute granularity is now available to monitor nearly realtime
• user can select time-series models and anomaly detection models from UI

Added rerun job option on UI, Job backfill bugfix

• User can now rerun jobs(which missed druid data when it was scheduled) from the UI by clicking rerun timestamp from the report timeline after the data is available
• This PR also fixes backfill window start-time error and also allows to select window end optionally in backfill form

Added job cloning feature

User can clone the existing job using clone option. New cloned job will appear in the job list with _cloned suffix in job name and all same configurations from old job.