Skip to content

jfrog/log-analytics-newrelic

BranchesTags

Repository files navigation

NEWRELIC

This integration is last tested with Artifactory 7.71.4 and Xray 3.85.5 versions.

Table of Contents

Note! You must follow the order of the steps throughout NewRelic Configuration

  1. NewRelic Setup
  2. JFrog Metrics Setup
  3. Fluentd Installation
  4. Dashboards
  5. Generating data for Testing
  6. References

NewRelic Setup

New Relic setup can be done by going through the onboarding steps below or by using license key directly if one exists. If a license key exists, use the New Relic Fluentd plugin to forward logs, violations and metrics directly to your New Relic account.

  • Create an account in New Relic
  • From the account dropdown, click API keys
  • Copy the license key which is also referenced in the UI as ingest - license

JFrog Metrics Setup

To enable metrics in Artifactory, make the following configuration changes to the Artifactory System YAML

artifactory:
    metrics:
        enabled: true
    openMetrics:
        enabled: true

Once this configuration is done and the application is restarted, metrics will be available in Open Metrics Format

Metrics are enabled by default in Xray. For kubernetes based installs, openMetrics are enabled in the helm install commands listed below

Fluentd Installation

OS / Virtual Machine

Ensure you have access to the Internet from VM. Recommended install is through fluentd's native OS based package installs:

OS Package Manager Link
CentOS/RHEL Linux - RPM (YUM) https://docs.fluentd.org/installation/install-by-rpm
Debian/Ubuntu Linux - APT https://docs.fluentd.org/installation/install-by-deb
MacOS/Darwin MacOS - DMG https://docs.fluentd.org/installation/install-by-dmg
Windows Windows - MSI https://docs.fluentd.org/installation/install-by-msi
Gem Install** MacOS & Linux - Gem https://docs.fluentd.org/installation/install-by-gem 
** For Gem based install, Ruby Interpreter has to be setup first, following is the recommended process to install Ruby

1. Install Ruby Version Manager (RVM) as described in https://rvm.io/rvm/install#installation-explained, ensure to follow all the onscreen instructions provided to complete the rvm installation
	* For installation across users a SUDO based install is recommended, the installation is as described in https://rvm.io/support/troubleshooting#sudo

2. Once rvm installation is complete, verify the RVM installation executing the command 'rvm -v'

3. Now install ruby v2.7.0 or above executing the command 'rvm install <ver_num>', ex: 'rvm install 2.7.5'

4. Verify the ruby installation, execute 'ruby -v', gem installation 'gem -v' and 'bundler -v' to ensure all the components are intact

5. Post completion of Ruby, Gems installation, the environment is ready to further install new gems, execute the following gem install commands one after other to setup the needed ecosystem

	'gem install fluentd'

After FluentD is successfully installed, the below plugins are required to be installed

gem install fluent-plugin-concat
gem install fluent-plugin-newrelic
gem install fluent-plugin-jfrog-siem
gem install fluent-plugin-jfrog-metrics
gem install fluent-plugin-jfrog-send-metrics

Configure Fluentd

We rely heavily on environment variables so that the correct log files are streamed to your observability dashboards. Ensure that you fill in the .env file with correct values. Download the jfrog.env file from here

  • JF_PRODUCT_DATA_INTERNAL: The environment variable JF_PRODUCT_DATA_INTERNAL must be defined to the correct location. For each JFrog service you will find its active log files in the $JFROG_HOME/<product>/var/log directory
  • NEWRELIC_LICENSE_KEY: License Key from NewRelic
  • JPD_URL: Artifactory JPD URL of the format http://<ip_address>
  • JPD_ADMIN_USERNAME: Artifactory username for authentication
  • JPD_ADMIN_TOKEN: Artifactory Access Token for authentication
  • COMMON_JPD: This flag should be set as true only for non-kubernetes installations or installations where JPD base URL is same to access both Artifactory and Xray (ex: https://sample_base_url/artifactory or https://sample_base_url/xray)
  • NEWRELIC_LOGS_URI: This New Relic logs endpoint needs to be set if your New Relic instance is in the EU region (or if any other custom configuration is needed). It defaults to https://log-api.newrelic.com/log/v1
  • NEWRELIC_METRICS_URI: This New Relic metrics endpoint needs to be set if your New Relic instance is in the EU region (or if any other custom configuration is needed). It defaults to https://metric-api.newrelic.com/metric/v1

Apply the .env files and then run the fluentd wrapper with one argument pointed to the fluent.conf.* file configured.

source jfrog.env
./fluentd $JF_PRODUCT_DATA_INTERNAL/fluent.conf.<product_name>

Docker

Note! These steps were not tested to work out of the box on MAC In order to run fluentd as a docker image to send the log, siem and metrics data to Newrelic, the following commands needs to be executed on the host that runs the docker.

  1. Check the docker installation is functional, execute command 'docker version' and 'docker ps'.

  2. Once the version and process are listed successfully, build the intended docker image for the observability platform using the docker file,

    • Download Dockerfile from here to any directory which has write permissions.

  3. Download the docker.env file needed to run Jfrog/FluentD Docker Images for the intended observability platform,

    • Download docker.env from here to the directory where the docker file was downloaded.
* **NEWRELIC_LOGS_URI**: This New Relic logs endpoint needs to be set if your New Relic instanceif your New Relic instance is in the EU region (or if any other custom configuration is needed). It defaults to https://log-api.newrelic.com/log/v1 if isn't set.
* **NEWRELIC_METRICS_URI**: This New Relic metrics endpoint needs to be set if your New Relic instance is in the EU region (or if any other custom configuration is needed). It defaults to https://metric-api.newrelic.com/metric/v1 if isn't set.
For NewRelic as the observability platform, execute these commands to setup the docker container running the fluentd installation

1. Execute 'docker build --build-arg SOURCE="JFRT" --build-arg TARGET="NEWRELIC" -t <image_name> .'

    Command example

    'docker build --build-arg SOURCE="JFRT" --build-arg TARGET="NEWRELIC" -t jfrog/fluentd-newrelic-rt .'

    The above command will build the docker image.

2. Fill the necessary information in the docker.env file

    JF_PRODUCT_DATA_INTERNAL: The environment variable JF_PRODUCT_DATA_INTERNAL must be defined to the correct location. It will be the directory where logs are mounted ex: /var/opt/jfrog/artifactory
    NEWRELIC_LICENSE_KEY: License Key from [NewRelic](https://one.newrelic.com/launcher/api-keys-ui.api-keys-launcher)
    JPD_URL: Artifactory JPD URL of the format `http://<ip_address>`
    JPD_ADMIN_USERNAME: Artifactory username for authentication
    JPD_ADMIN_TOKEN: Artifactory [Access Token](https://jfrog.com/help/r/how-to-generate-an-access-token-video/artifactory-creating-access-tokens-in-artifactory) for authentication
    COMMON_JPD: This flag should be set as true only for non-kubernetes installations or installations where JPD base URL is same to access both Artifactory and Xray (ex: https://sample_base_url/artifactory or https://sample_base_url/xray)
    NEWRELIC_LOGS_URI: This New Relic logs endpoint needs to be set if your New Relic instance is in the EU region (or if any other custom configuration is needed). It defaults to https://log-api.newrelic.com/log/v1
    NEWRELIC_METRICS_URI: This New Relic metrics endpoint needs to be set if your New Relic instance is in the EU region (or if any other custom configuration is needed). It defaults to https://metric-api.newrelic.com/metric/v1

3. Execute 'docker run -it --name jfrog-fluentd-newrelic-rt -v <path_to_logs>:/var/opt/jfrog/artifactory --env-file docker.env <image_name>'

    The <path_to_logs> should be an absolute path where the Jfrog Artifactory Logs folder resides, i.e for an Docker based Artifactory Installation,  ex: /var/opt/jfrog/artifactory/var/logs on the docker host.

    Command example

    'docker run -it --name jfrog-fluentd-newrelic-rt -v $JFROG_HOME/artifactory/var/:/var/opt/jfrog/artifactory --env-file docker.env jfrog/fluentd-newrelic-rt'

Kubernetes Deployment with Helm

Recommended installation for Kubernetes is to utilize the helm chart with the associated values.yaml in this repo.

Product Example Values File
Artifactory helm/artifactory-values.yaml
Artifactory HA helm/artifactory-ha-values.yaml
Xray helm/xray-values.yaml

Warning

The old docker registry partnership-pts-observability.jfrog.io, which contains older versions of this integration is now deprecated. We'll keep the existing docker images on this old registry until August 1st, 2024. After that date, this registry will no longer be available. Please helm upgrade your JFrog kubernetes deployment in order to pull images as specified on the above helm value files, from the new releases-pts-observability-fluentd.jfrog.io registry. Please do so in order to avoid ImagePullBackOff errors in your deployment once this registry is gone.

Add JFrog Helm repository

helm repo add jfrog https://charts.jfrog.io
helm repo update

Throughout the exampled helm installations we'll use jfrog-nr as an example namespace. That said, you can use a different or existing namespace instead by setting the following environment variable

export INST_NAMESPACE=jfrog-nr

If you don't have an existing namespace for the deployment, create it and set the kubectl context to use this namespace

kubectl create namespace $INST_NAMESPACE
kubectl config set-context --current --namespace=$INST_NAMESPACE

Replace placeholders with your masterKey and joinKey. To generate each of them, use the command openssl rand -hex 32

export JOIN_KEY=$(openssl rand -hex 32)
export MASTER_KEY=$(openssl rand -hex 32)

Artifactory ⎈:

  1. Skip this step if you already have Artifactory installed. Else, install Artifactory using the following commands below:

    Generate a secret that contains the Artifactory license key (optional, user can enter it manually later)

    kubectl create secret generic artifactory-license --from-file=<path_to_license_file>

  2. Install Artifactory using the above join/master keys and secret

helm upgrade --install artifactory  jfrog/artifactory \
       --set artifactory.masterKey=$MASTER_KEY \
       --set artifactory.joinKey=$JOIN_KEY \
       --set artifactory.license.secret=artifactory-license \
       --set artifactory.license.dataKey=artifactory.cluster.license \
       --set artifactory.metrics.enabled=true \
       -n $INST_NAMESPACE

💡Note: Metrics collection is disabled by default in Artifactory. Please make sure that you are following the above helm upgrade command to enable them in Artifactory by setting artifactory.metrics.enabled=true. For Artifactory versions <=7.86.x, please enable metrics by setting the flag artifactory.openMetrics.enabled=true

  1. Follow the instructions how to get your new Artifactory URL from the helm install output
   export SERVICE_IP=$(kubectl get svc --namespace $INST_NAMESPACE artifactory-artifactory-nginx -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
   echo http://$SERVICE_IP/
  1. Using the Artifactory UI generate JFrog's admin Access Token. Using that fetched token, create a kubernetes generic secret for JFrog's admin token - using any of the following methods
kubectl create secret generic jfrog-admin-token --from-file=token=<path_to_token_file>

OR

kubectl create secret generic jfrog-admin-token --from-literal=token=<JFROG_ADMN_TOKEN>
  1. For Artifactory installation, download the .env file from here. Fill in the jfrog_helm.env file with correct values
  • NEWRELIC_LICENSE_KEY: License Key from NewRelic
  • JPD_URL: Artifactory JPD URL of the format http://<ip_address>
  • JPD_ADMIN_USERNAME: Artifactory username for authentication
  • COMMON_JPD: This flag should be set as true only for non-kubernetes installations or installations where JPD base URL is same to access both Artifactory and Xray (ex: https://sample_base_url/artifactory or https://sample_base_url/xray)
  • NEWRELIC_LOGS_URI: This New Relic logs endpoint needs to be set if your New Relic instance is in the EU region (or if any other custom configuration is needed). It defaults to https://log-api.newrelic.com/log/v1 if isn't set
  • NEWRELIC_METRICS_URI: This New Relic metrics endpoint needs to be set if your New Relic instance is in the EU region (or if any other custom configuration is needed). It defaults to https://metric-api.newrelic.com/metric/v1 if isn't set

Apply the .env files using the helm command below

source jfrog_helm.env
  1. Postgres password is required to upgrade Artifactory. Run the following command to get the current Postgres password
POSTGRES_PASSWORD=$(kubectl get secret artifactory-postgresql -o jsonpath="{.data.postgresql-password}" | base64 --decode)
  1. Upgrade Artifactory installation using the command below
helm upgrade --install artifactory jfrog/artifactory \
       --set artifactory.joinKey=$JOIN_KEY \
       --set databaseUpgradeReady=true --set postgresql.postgresqlPassword=$POSTGRES_PASSWORD \
       --set newrelic.license_key=$NEWRELIC_LICENSE_KEY \
       --set jfrog.observability.jpd_url=$JPD_URL \
       --set jfrog.observability.username=$JPD_ADMIN_USERNAME \
       --set jfrog.observability.common_jpd=$COMMON_JPD \
       --set newrelic.logs_uri=$NEWRELIC_LOGS_URI \
       --set newrelic.metrics_uri=$NEWRELIC_METRICS_URI \
       -f helm/artifactory-values.yaml \
       -n $INST_NAMESPACE

💡Note: Setting newrelic.logs_uri and newrelic.metrics_uri values in the above command is optional and only required if your New Relic endpoints isn't the default. For example, if working with New Relic EU servers, make sure to set these env variables

Artifactory-HA ⎈:

  1. Skip this step if you already have Artifactory or Artifactory-HA installed
  2. For HA installation, please create a license secret on your cluster prior to installation
kubectl create secret generic artifactory-license --from-file=<path_to_license_file>artifactory.cluster.license
  1. install Artifactory-HA using the commands below
helm upgrade --install artifactory-ha  jfrog/artifactory-ha \
       --set artifactory.masterKey=$MASTER_KEY \
       --set artifactory.joinKey=$JOIN_KEY \
       --set artifactory.license.secret=artifactory-license \
       --set artifactory.license.dataKey=artifactory.cluster.license \
       --set artifactory.metrics.enabled=true \
       -n $INST_NAMESPACE

💡Note: Metrics collection is disabled by default in Artifactory-HA. Please make sure that you are following the above helm upgrade command to enable them in Artifactory by setting artifactory.metrics.enabled=true. For Artifactory versions <=7.86.x, please enable metrics by setting the flag artifactory.openMetrics.enabled=true

  1. Follow the instructions how to get your new Artifactory URL from the helm install output
   export SERVICE_IP=$(kubectl get svc --namespace $INST_NAMESPACE artifactory-artifactory-nginx -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
   echo http://$SERVICE_IP/
  1. Using the Artifactory UI generate JFrog's admin Access Token. Using that fetched token, create a kubernetes generic secret for JFrog's admin token - using any of the following methods
kubectl create secret generic jfrog-admin-token --from-file=token=<path_to_token_file>

OR

kubectl create secret generic jfrog-admin-token --from-literal=token=<JFROG_ADMN_TOKEN>
  1. Download the .env file from here. Fill in the jfrog_helm.env file with correct values:
  • NEWRELIC_LICENSE_KEY: License Key from NewRelic
  • JPD_URL: Artifactory JPD URL of the format http://<ip_address>
  • JPD_ADMIN_USERNAME: Artifactory username for authentication
  • COMMON_JPD: This flag should be set as true only for non-kubernetes installations or installations where JPD base URL is same to access both Artifactory and Xray (ex: https://sample_base_url/artifactory or https://sample_base_url/xray)
  • NEWRELIC_LOGS_URI: This New Relic logs endpoint needs to be set if your New Relic instance is in the EU region (or if any other custom configuration is needed). It defaults to https://log-api.newrelic.com/log/v1 if isn't set
  • NEWRELIC_METRICS_URI: This New Relic metrics endpoint needs to be set if your New Relic instance is in the EU region (or if any other custom configuration is needed). It defaults to https://metric-api.newrelic.com/metric/v1 if isn't set

Apply the .env files and then run the helm command below

source jfrog_helm.env
  1. Postgres password is required to upgrade Artifactory. Run the following command to get the current Postgres password
POSTGRES_PASSWORD=$(kubectl get secret artifactory-ha-postgresql -o jsonpath="{.data.postgresql-password}" | base64 --decode)
  1. Upgrade Artifactory HA installation using the command below
helm upgrade --install artifactory-ha  jfrog/artifactory-ha \
    --set artifactory.joinKey=$JOIN_KEY \
    --set databaseUpgradeReady=true --set postgresql.postgresqlPassword=$POSTGRES_PASSWORD \
    --set newrelic.license_key=$NEWRELIC_LICENSE_KEY \
    --set jfrog.observability.jpd_url=$JPD_URL \
    --set jfrog.observability.username=$JPD_ADMIN_USERNAME \
    --set jfrog.observability.common_jpd=$COMMON_JPD \
    --set newrelic.logs_uri=$NEWRELIC_LOGS_URI \
    --set newrelic.metrics_uri=$NEWRELIC_METRICS_URI \
    -f helm/artifactory-ha-values.yaml \
    -n $INST_NAMESPACE

💡Note: Setting newrelic.logs_uri and newrelic.metrics_uri values in the above command is optional and only required if your New Relic endpoints isn't the default. For example, if working with New Relic EU servers, make sure to set these env variables

Xray ⎈:

  1. If wasn't created during Artifactory's installation, create a secret for JFrog's admin token - Access Token using any of the following methods
kubectl create secret generic jfrog-admin-token --from-file=token=<path_to_token_file>

OR

kubectl create secret generic jfrog-admin-token --from-literal=token=<JFROG_ADMN_TOKEN>

  1. If wasn't created during Artifactory's installation, download the .env file from here. Fill in the jfrog_helm.env file with correct values:

    • NEWRELIC_LICENSE_KEY: License Key from NewRelic
    • JPD_URL: Artifactory JPD URL of the format http://<ip_address>
    • JPD_ADMIN_USERNAME: Artifactory username for authentication
    • COMMON_JPD: This flag should be set as true only for non-kubernetes installations or installations where JPD base URL is same to access both Artifactory and Xray (ex: https://sample_base_url/artifactory or https://sample_base_url/xray)
    • NEWRELIC_LOGS_URI: This New Relic logs endpoint needs to be set if your New Relic instance is in the EU region (or if any other custom configuration is needed). It defaults to https://log-api.newrelic.com/log/v1 if isn't set
    • NEWRELIC_METRICS_URI: This New Relic metrics endpoint needs to be set if your New Relic instance is in the EU region (or if any other custom configuration is needed). It defaults to https://metric-api.newrelic.com/metric/v1 if isn't set

    Apply the .env files and then run the helm command below

    source jfrog_helm.env

  2. Generate a master key for xray

    export XRAY_MASTER_KEY=$(openssl rand -hex 32)

    Use the same joinKey as you used in Artifactory installation $JOIN_KEY to allow Xray node to successfully connect to Artifactory and use the command below to install/upgrade Xray

    helm upgrade --install xray jfrog/xray --set xray.jfrogUrl=$JPD_URL \
       --set xray.masterKey=$XRAY_MASTER_KEY \
       --set xray.joinKey=$JOIN_KEY \
       --set newrelic.license_key=$NEWRELIC_LICENSE_KEY \
       --set jfrog.observability.jpd_url=$JPD_URL \
       --set jfrog.observability.username=$JPD_ADMIN_USERNAME \
       --set jfrog.observability.common_jpd=$COMMON_JPD \
       --set newrelic.logs_uri=$NEWRELIC_LOGS_URI \
       --set newrelic.metrics_uri=$NEWRELIC_METRICS_URI \
       -f helm/xray-values.yaml

    💡Note: Setting newrelic.logs_uri and newrelic.metrics_uri values in the above command is optional and only required if your New Relic endpoints isn't the default. For example, if working with New Relic EU servers, make sure to set these env variables

Dashboards

Adding JFrog dashboards

In the New Relic UI, on the left menu go to Add Data -> All -> Search Bar and search for jfrog. From the results choose Dashboards and add the JFrog dashboards

Artifactory dashboard

JFrog Artifactory Dashboard is divided into three sections Application, Audit, Requests and Docker

  • Application - This section tracks Log Volume (information about different log sources) and Artifactory Errors over time (bursts of application errors that may otherwise go undetected)
  • Audit - This section tracks audit logs help you determine who is accessing your Artifactory instance and from where. These can help you track potentially malicious requests or processes (such as CI jobs) using expired credentials
  • Requests - This section tracks HTTP response codes, top 10 IP addresses for uploads and downloads
  • Docker - To monitor Dockerhub pull requests users should have a Dockerhub account either paid or free. Free accounts allow up to 200 pull requests per 6 hour window. Various widgets have been added in the new Docker tab under Artifactory to help monitor your Dockerhub pull requests. An alert is also available to enable if desired that will allow you to send emails or add outbound webhooks through configuration to be notified when you exceed the configurable threshold
  • Metrics - To gain insights into the system performance, storage consumption, and connection statistics associated with JFrog Artifactory

Xray dashboard

JFrog Xray Dashboard is divided into two sections Logs and Violations

  • Logs - This dashboard provides a summary of access, service and traffic log volumes associated with Xray. Additionally, customers are also able to track various HTTP response codes, HTTP 500 errors, and log errors for greater operational insight
  • Violations - This dashboard provides an aggregated summary of all the license violations and security vulnerabilities found by Xray. Information is segment by watch policies and rules. Trending information is provided on the type and severity of violations over time, as well as, insights on most frequently occurring CVEs, top impacted artifacts and components
  • Metrics - To gain insights into the system performance, storage consumption, connection statistics, count and type of artifacts and components scanned by JFrog Xray

Demo Requirements

  • Kubernetes Cluster
  • Artifactory and/or Xray installed via JFrog Helm Charts
  • Helm 3
  • New Relic account setup with license key

Generating Data for Testing

Partner Integration Test Framework can be used to generate data for metrics

References

About

No description, website, or topics provided.

Resources

Readme

License

Apache-2.0 license

Security policy

Security policy
Activity
Custom properties

Stars

1 star

Watchers

4 watching

Forks

2 forks
Report repository

Releases

6 tags

Packages

No packages published

Contributors 2

  •  
  •  

Languages