New Relic Reports

New Relic Reports is a full scheduled report solution for use with the New Relic One platform.

Table of Contents

• Overview
• Using This Document
• Quickstart
• Prerequisites
• Introduction
• Concepts
  • Report Generation
  • Report Delivery
  • Running Reports
• Usage
  • Reports
    • Template Reports
    • Dashboard Reports
    • Query Reports
    • Report Execution Context
    • Report Output
    • Publish Configurations
    • Channels
    • Manifest Files
  • Using the CLI
  • Using the CLI image
  • Using the AWS Lambda function
  • Using the AWS Scheduler Lambda function
  • Using the Java Scheduler
  • Using the Reports Builder
  • Using the New Relic Reports AWS Stack

Overview

New Relic Reports is a scheduled reporting solution for use with New Relic. It consists of several components.

• A reporting engine providing capabilities for generating different types of reports and delivering reports via different channels that can be run from the command line as a Node.js application, a Docker image, or an AWS Lambda function. See the section "Running Reports" for more details.
• The Reports Builder, a New Relic application available on the New Relic catalog that provides a simple user interface for building reports from within the New Relic platform.
• The Java Scheduler, a Java application that works with the CLI and the Reports Builder to provide a full end-to-end scheduled reporting solution that can be run as a Docker image or anywhere that Java and Node.js can be run.
• The AWS Scheduler Lambda function, an AWS Lambda function that works with AWS Lambda function to run reports defined in the Reports Builder on the specified schedules.
• The New Relic Reports AWS Stack, an AWS CloudFormation Stack that can be used to deploy the AWS Lambda function, the AWS Scheduler Lambda function, and the supporting resources to provide a full end-to-end scheduled reporting solution within AWS utilizing AWS Lambda and AWS EventBridge.

Using This Document

This document is organized as follows.

• Quickstart - The fastest way to get up and running with scheduled reports.
• Introduction - A brief tutorial to familiarize you with the report types that can be generated and the outputs of each report type
• Concepts - A high-level overview of the core concepts of the New Relic Reports reporting engine
• Usage - Comprehensive, detailed information about every component of the solution including core engine concepts such as the types of reports, report output, publish configurations, channels, and manifest files followed by details on using the CLI, using the Java Scheduler, using the Lambda fuction, using the Scheduler Lambda function, using the Reports Builder, and using the AWS Reports Stack.

Quickstart

The quickest way to get started with New Relic Reports is to enable the New Relic Reports Builder and deploy the New Relic Reports AWS Stack or the Java Scheduler.

This will provide you with the UI to build scheduled reports and a runtime to schedule and run reports.

Prerequisites

To run reports locally using the CLI, you will need the following.

• Node >= 20.0.0
• A terminal application that supports Bash scripts

To build and deploy the CLI image you will need the following.

• Docker
• A Docker repository

To build and deploy the Java Scheduler image, you will need the following.

• Docker
• A Docker repository

To run the Java Scheduler as a standalone Java application, you will need the following.

• Node >= 20.0.0
• Java >= 21.0.0
• A terminal application that supports Bash scripts

To build and deploy the New Relic Reports AWS Stack or the Lambda based images, you will need the following.

• Docker
• An ECS repository
• The AWS CLI
• AWS credentials

In addition, each channel type has it's own prerequisites as follows.

• For the email channel, SMTP server settings (for testing locally try Mailhog)
• For the S3 channel, AWS credentials and an S3 bucket
• For the Slack channel, an Incoming Webhook URL.
• For the webhook channel, the Webhook URL, any required custom header details, and the payload format.

Introduction

This tutorial is meant to introduce you to the types of reports that can be generated and the output that is produced for each report type. For this simple tutorial, we will be generating reports manually from the command line and the results will be saved to the local file system so that the results are easy to view. While this is convenient for development and testing, it is not the recommended way to generate and deliver reports. It is used below for edification purposes only.

NOTE: Scheduling reports and report delivery are not addressed in this simple tutorial. For information on building scheduled reports, refer to the section on using the Reports Builder. For information on deploying a runtime that supports scheduling and running reports, see the section on using the Java Scheduler or the section on using the New Relic Reports AWS Stack. For information on available mechanisms for report delivery see the section on channels.

Before you begin

If you haven't already, make sure you have checked the prerequisites for running reports locally. You will also need a New Relic User key to test with.

To install New Relic Reports locally, perform the following commands.

git clone [email protected]:newrelic/nr-reports.git
cd nr-reports
npm install

Then open a terminal that supports Bash scripts and execute the following commands, making sure to replace path/to/nr-reports with the path to the directory where you cloned the nr-reports repository AND YOUR_USER_KEY with your New Relic User key.

cd path/to/nr-reports
export NEW_RELIC_API_KEY="[YOUR USER KEY]"

NOTE: To use an account located in the EU datacenter, additionally execute the following command.

export NEW_RELIC_REGION="EU"

Run a dashboard report

Dashboard reports are generated by specifying one or more dashboard GUIDs. The reporting engine will use Nerdgraph to download a dashboard snapshot PDF for each dashboard and optionally combine multiple snapshots in a single PDF.

Here's how you run a dashboard report.

Find your dashboard GUID(s)

The easiest way to find the GUID for a single-page dashboard is via the NR1 UI.

1. Navigate to your dashboard
2. Locate the dashboard name above the filter bar
3. On one side of the dashboard name, locate the box that labeled "Metadata". Click anywhere in the box.
4. In the "drawer" that slides out from the side of the screen, locate the label "Entity guid" followed by a long string of numbers and letters (this is the dashboard GUID).
5. Hover over the string of numbers and letters and click on the clipboard icon that appears. The dashboard GUID will now be copied in the clipboard.

To find the GUID of a page within a multi-page dashboard, it is necessary to use the Nerdgraph API Explorer.

1. Navigate to the Nerdgraph API Explorer.
2. In the text box labeled "User key", enter your New Relic User key.
3. Paste the following text into the GraphQL source editor box. Make sure to replace the text "My dashboard" with the full name of the dashboard that contains the page you want to use (not the name of the page itself).

{
  actor {
    entitySearch(queryBuilder: {type: DASHBOARD, name: "My dashboard"}) {
      results {
        entities {
          guid
          name
        }
      }
    }
  }
}

4. Hit the "Play" button or type Ctrl-Enter to execute the query.
5. In the results text box, locate the entity that contains the name of the page you want to use in the name property.
6. Copy the value or the guid property shown just above the name property.

Run the dashboard report

Now run the report using the following command, replacing the string ABCDEF123456 with your dashboard GUID.

./nr-reports-cli/bin/nr-reports.sh -d ABCDEF123456

Now there should be a new PDF file in the current directory called dashboard-[DASHBOARD_GUID].pdf where [DASHBOARD_GUID] is the GUID of your dashboard. Open it up and you should see a snapshot of your dashboard.

Run a query report

Now let's see how to run a query report. Query reports are generated by specifying a single NRQL query. The reporting engine will automatically run the query and, by default, write the results to a CSV file.

Let's see how that works by running a simple query report to show the average latency of all APM services in your account grouped by application name and URL using the following NRQL.

SELECT average(duration) as 'Duration' FROM Transaction FACET appName as 'Application Name', request.uri AS 'URL'

To do that, run the following command, replacing the string 1234567 with your account ID.

./nr-reports-cli/bin/nr-reports.sh -a 1234567 -q "SELECT average(duration) as 'Duration' FROM Transaction FACET appName as 'Application Name', request.uri AS 'URL'"

Now there should be a new CSV file in the current directory called query-report.csv. Open it up in a text editor and you should see something like the following.

Application Name,URL,Duration
Shop Service,/api/v1/checkout,1.5191369267857142
Shop Service,/api/v1/products,1.5092493357575756
Shop Service,/api/v1/products/1234,1.4948035056074764

Summary

Here's what we just did.

1. Used the CLI script to run a report at the command line using a dashboard entity GUID. This generated a PDF file in the current directory.
2. Used the CLI script to run a query report at the command line using a simple NRQL query. This generated a CSV file in the current directory.

Though useful for experimenting, in most cases, you won't be generating reports by running the CLI directly. Instead, you will use one of the provided mechanisms for automating the generation and delivery of reports. See the usage section for more details.

Concepts

Report Generation

The New Relic Reports engine provides support for generating reports in several ways.

Template Reports

NOTE: This functionality has been removed as of v4.0.0 and no replacement is planned.

Dashboard Reports

Dashboard reports use Nerdgraph to collect snapshot URLs from one or more user specified dashboard GUIDs. Snapshot URLs are downloaded as PDFs. When more than one dashboard is specified, the PDFs can optionally be concatenated into a single PDF.

NOTE: Dashboard reports can only be generated for single page dashboards or for a single page of a multi-page dashboard.

Query Reports

Query reports provide a mechanism to export the results of running a NRQL query by simply specifying a query and one or more account IDs to run the query against. No additional configuration is required. By default, query results are exported to CSV but can also be exported as HTML or JSON.

Report Delivery

A variety of mechanisms are supported for delivering report output. These mechanisms are referred to as channels. The following types of channels are supported.

• File: Report output is saved to a file and copied to a destination directory on the local filesystem. Mostly meant for development and testing purposes.
• Email: Report output is included inline or as attachments to an email using a user defined email message and sent via SMTP.
• S3: Report output is saved to a file and uploaded to an S3 bucket.
• Slack: Report output is posted to a Slack channel via a Slack webhook.
• Webhook: Report output is posted to a custom Webhook.

Running Reports

There are several ways to run reports using the reporting engine.

1. Packaged as a Docker image

   Dockerfiles are provided to package the reporting engine, along with your manifest files as a Docker image with a CLI based ENTRYPOINT that can be run via external scheduled task mechanisms such as AWS ECS Scheduled Tasks. Both of these can be used in conjuction with the appropriate environment variables to run reports built using the New Relic Reports builder to provide a full scheduled reporting solution.

2. Packaged as an AWS Lambda function

   A Dockerfile is provided to package the reporting engine, along with your manifest files as an AWS Lambda function. The Lambda can be deployed with the provided CloudFormation template and the provided helper scripts.

3. Using the command line interface (CLI)

   Reports can be run directly from a command line after cloning or downloading the repository and installing dependencies. This is useful primarily for testing and debugging reports.

NOTE: The mechanisms listed are used purely to execute reports using the reporting engine. They do not interact with the Reports Builder and do not include any scheduling functionality. Reports must be manually defined using manifest files, CLI options, engine options, or environment variables.

Usage

Reports

The New Relic Reports reporting engine generates reports using report generators. There are two types of report generators: the dashboard report generator and the query report generator.

Each report generator interacts with one or more New Relic APIs to generate report output. Report output is distributed to destinations by selecting a publish configuration which specifies a set of channels that are used to route the report output to the destination.

The entire process is governed by a set of configuration parameters that can be specified at the command line, in environment variables, or in a manifest file.

Dashboard Reports

Dashboard reports provide a way to easily capture PDFs of one or more dashboards. When more than one dashboard GUID is specified, each dashboard is captured as a separate PDF file. These files can optionally be combined into a single file.

The following YAML shows an example of a basic dashboard report definition in a manifest file.

reports:
- id: example-dashboard-report
  name: Example dashboard report
  dashboards:
  - DASHBOARD_GUID_1

When the reporting engine runs this report, it will execute the dashboardCreateSnapshotUrl Nerdgraph mutation with the GUID DASHBOARD_GUID_1 in order to create a URL to download a PDF snapshot of the dashboard with the given GUID. It will then download the PDF to a file named dashboard-DASHBOARD_GUID_1.pdf since no outputFileName is specified. The PDF file will be copied to the current working directory since the default channel is the file channel and no destDir channel parameter is set.

See the section Dashboard Report Properties for more information on the available dashboard report properties.

Query Reports

Query reports provide a simple way to run a NRQL query and export the results of the query to a file. By default, the results are exported to a CSV file.

The following YAML shows an example of a query report definition in a manifest file that will run the example query from the section Run a query report in the Getting Started tutorial.

reports:
- id: example-query-report
  query: "SELECT average(duration) as 'Duration' FROM Transaction FACET appName as 'Application Name', request.uri"
  accountIds:
  - 1234567
  timeout: 10

When the reporting engine runs this report, it will execute the following Nerdgraph query using the given NRQL query for the $query argument, the given account ID (in a 1 member array) for the $accountIds argument, and the given timeout for the $timeout argument.

{
   actor {
      nrql(accounts: $accountIds, query: $query, timeout: $timeout) {
         results
         metadata {
            facets
            eventTypes
         }
      }
   }
}

The results will be tabulated into a CSV file where the rows in the CSV file correspond to the each item in the returned results array and the columns in the CSV file correspond to the facets in the query (in this case Application Name and URL) followed by the fields selected by the query (in this case Duration), just like you'd see in a TABLE widget on a dashboard. Following is an example CSV that might be generated from this query.

Application Name,URL,Duration
Shop Service,/api/v1/checkout,1.5191369267857142
Shop Service,/api/v1/products,1.5092493357575756
Shop Service,/api/v1/products/1234,1.4948035056074764

NOTE: Queries for query reports are always run synchronously. Asynchronous queries are not supported.

Multi-account Queries

Multiple account IDs can be specified for a query report. The reporting engine supports three different modes for executing queries against multiple accounts. The query mode is specified using the multiAccountMode option in a query report definition in the manifest. Specifying the multi-account mode at the CLI is not supported and therefore will always use the default multi-account mode cross-account.

Cross-account Queries

By default, a multi-account query will be run using a cross-account query. With cross-account queries, a query is run against each of the accounts (up to a maximum of 5) using a query like the following.

{
  actor {
    nrql(
      accounts: [ACCOUNT_ID_1, ACCOUNT_ID_2, ACCOUNT_ID_3]
      options: {}
      query: "NRQL_QUERY"
      timeout: 70
    ) {
      results
    }
  }
}

The results are aggregated and returned as a single set of results.

To specify that the reporting engine should run a multi-account query using a cross-account query, either set the multiAccountMode option in the manifest to cross-account or leave it out entirely, as cross-account is the default.

Per-account Queries

Sometimes you may want to run a query individually against multiple accounts and get each set of results individually rather than aggregating the set of results across all accounts. For example, if you want to find the top 5 transactions of each of 5 different accounts and export them to a CSV file with each row including the account ID, using a cross-account query won't suffice. To account for this, the reporting engine provides two multi-account modes that execute queries separately against each account: per-account and per-account-concurrent.

In the per-account case, a single GraphQL query is run that utilizes GraphQL aliases to run multiple GraphQL queries in a single GraphQL call, as in the following example.

{
   NrqlQuery1: {
      actor {
         nrql(
            accounts: [ACCOUNT_ID_1]
            options: {}
            query: "NRQL_QUERY"
            timeout: 70
         ) {
            results
         }
      }
   }
   NrqlQuery2: {
      actor {
         nrql(
            accounts: [ACCOUNT_ID_2]
            options: {}
            query: "NRQL_QUERY"
            timeout: 70
         ) {
            results
         }
      }
   }
}

In the per-account-concurrent case, multiple GraphQL queries are run concurrently, one query per account.

NOTE: The per-account and per-account-concurrent modes are not "native" query types like cross-account. Rather, they are implemented in the reporting engine.

Report Execution Context

The reporting engine creates an "execution context" each time it runs a report. The execution context is a collection of properties that can be used by the components that generate the report. When the execution context is created, the reporting engine populates it as follows.

• If a manifest file is specified:
  • Add all properties from the variables section at the top-level
  • Add properties from the report definition
• If no manifest file is specified:
  • Add the report id and the outputFileName if one is specified
  • If the report is a dashboard report, add an array property named dashboards containing all dashboard GUIDs
  • If the report is a query report add a property named query containing the specified NRQL query

In addition, prior to being passed to any channel implementation, the publish configuration ID and the channel ID are added to the execution context. If a manifest file is specified, all properties from the selected publish configuration and all channel configuration parameters are also added.

NOTE: When populating the context, if two properties with the same name are added by different steps, the later property will overwrite the earlier property.

Report Output

Every report produces output. The output can either be a file stored in the temporary work directory created when the reporting engine starts or data that can be rendered to produce text-based content.

The type of output depends on the type of report.

Report Type	Output Type
Dashboard Report	File
Query Report	query results

The report output, whether file or renderable data, is passed to the channel implementations for delivery.

NOTE: The type of output generated for a given report type is not always the same as the format used to send the output via a particular channel. For example, the default output of a query report is a set of NRQL row data. But this data can be sent via the email channel as an HTML table in the body of the email or in a file attached to the email.

Dashboard Report Output

Dashboard reports always produce one or more PDF files.

Query Report Output

By default, a query report produces a set of rows, one for each row returned in the NRQL query result. The results are passed along to the channel implementations along with a renderer that, by default, generates CSV content from the rows.

The output format may be changed using the queryResultsFormat channel configuration parameter or the QUERY_RESULTS_FORMAT environment variable. When set to html, the query renderer will generate an HTML <table> element instead of CSV. When set to json, the query renderer will generate a JSON string for an array of objects, where each row of the query results is mapped to an object which contains a property for each column in the query results where the value of the property is the value of the corresponding row and column in the query results.

Additionally, when the output format is set to html, the queryResultsHtmlMaxRows channel configuration parameter or the QUERY_RESULTS_HTML_MAX_ROWS environment variable can be used to control the number of rows to include in the generated HTML. By default, 25 rows will be included.

The following shows an example of the same set of results formatted using each output format.

CSV:

Application Name,URL,Duration
Shop Service,/api/v1/checkout,1.5191369267857142
Shop Service,/api/v1/products,1.5092493357575756

HTML (formatted for display purposes only):

<table border="1">
  <thead>
    <tr>
      <th>Application Name</th>
      <th>URL</th>
      <th>Duration</th>
    </tr>
    <tr>
      <td>Shop Service</td>
      <td>/api/v1/checkout</td>
      <td>1.5191369267857142</td>
    </tr>
    <tr>
      <td>Shop Service</td>
      <td>/api/v1/products</td>
      <td>1.5092493357575756</td>
    </tr>
</table>

JSON (formatted for display purposes only):

[
  {
    "Application Name": "Shop Service",
    "URL": "/api/v1/checkout",
    "Duration": 1.5191369267857142,
  },
  {
    "Application Name": "Shop Service",
    "URL": "/api/v1/products",
    "Duration": 1.5092493357575756,
  }
]

Publish Configurations

Each report can have one or more publish configurations. Publish configurations tie a report to a set of channels that are used to distribute report output. After a report has been run, the reporting engine selects a publish configuration to use and distributes the report output via the channels specified in the selected publish configuration.

Publish configurations are defined in a manifest file and consist of a required id, a name, an enabled flag, a set of channels and an optional schedule expression. For example, the following report definition defines three publish configurations. The default publish configuration uses a file channel to copy the report output to the /tmp directory on the local machine. The copy-to-s3 publish configuration uses an s3 channel to copy the report output to an S3 bucket named Daily Report. The send-email publish configuration uses an email channel to send the report output to the set of recipients specified in the to and cc channel parameters.

reports:
- id: example-dashboard-report
  name: Example dashboard report
  dashboards:
  - DASHBOARD_GUID_1
  publishConfigs:
  - id: default
    name: Copy file
    channels:
    - id: copy-file-to-tmp
      name: Copy file to /tmp
      type: file
      destDir: "/tmp"
  - id: copy-pdf-to-s3
    name: Copy dashboard PDF to S3 bucket
    channels:
    - id: copy-file
      name: Copy to S3
      type: s3
      bucket: "Daily Report"
  - id: email-pdf
    name: Email dashboard PDF
    channels:
    - id: send-email
      name: Email PDF report to leadership
      type: email
      subject: Average transaction duration by application and URI
      from: [email protected]
      to: [email protected], [email protected]
      cc: [email protected], [email protected]

NOTE: In the absence of a manifest file or if the current report definition does not include any publish configurations, a default publish configuration with the id default is always created with the list of channels constructed by the engine or with the default channel. For example, if no channel IDs are specified at all when using the CLI, a publish configuration named default will automatically be added with the default file channel. Similarly, if no channel IDs are specified when using the AWS Lambda, a publish configuration named default will automatically be added with the default s3 channel.

Publish configuration selection

The reporting engine chooses the publish configuration to use via the following process.

1. Inspect the following sources in the following order for a comma-delimited string specifying a list of candidate publish configuration IDs.
2. When using the CLI, the -u option.
3. When using the AWS Lambda function, the publishConfigIds parameter specified in the options element of the AWS Lambda event.
4. The PUBLISH_CONFIG_IDS environment variable
5. Add the default publish configuration ID default to the end of the list of candidate publish configuration IDs.
6. For each candidate publish configuration ID, search the report definition for a publish configuration with a matching ID. The first publish configuration with a matching ID will be used.

Publish configuration schedule expression

Each publish configuration can have an optional schedule expression. The schedule expression is a CRON expression. The schedule expression is meant for use by components which can manage (create, update, delete) schedules in an external scheduling system like AWS EventBridge. Currently, the New Relic Reports Scheduler Lambda provided with the New Relic Reports AWS Stack is the only such component. To develop custom components which integrate with other scheduling mechanisms, see the New Relic Reports Scheduler Lambda source as an example.

NOTE: The schedule expression is ignored by the engine itself. That is, the engine has no built-in scheduling capabilities on it's own.

Channels

Channels are used to specify how report output should be distributed. Each publish configuration can specify one or more channels to use when the publish configuration is selected. Channels are defined together with the publish configuration in a manifest file and consist of a required id, a name, a type, and a set of channel parameters that are specific to the channel type.

The following channel types are supported:

• File (the default when running from the CLI)
• Email
• S3 (the default when running from the AWS Lambda)
• Slack
• Webhook

NOTE: In the absence of a manifest file or if a publish configuration does not include any channels, a default channel with the id ${report.id}.${publishConfig.id}.${channelType} is always created using the default channel type appropriate to the execution environment used to invoke the engine. For example, when using the CLI, if no channel is specified for a given publish configuration, the default file channel will be used automatically. Similarly, when using the AWS Lambda, if no channel is specified for a given publish configuration the default s3 channel will be used automatically.

Channel parameters

All channel types support configuration parameters that govern how reports are distributed when using that channel type. For example, the file channel type supports a destDir configuration parameter that specifies the destination directory that the report outputs should be copied into. The email channel type supports configuration parameters that specify the subject of the email, the list of recipients, and so on.

Channel configuration parameters are specified via a manifest file, or using environment variables. Most parameters can be specified using either method. For example, when using the email channel type, the subject can be specified using the EMAIL_SUBJECT environment variable or using the subject channel configuration parameter in the channel definition of a manifest file as shown below.

reports:
- id: example-dashboard-report
  name: Example dashboard report
  dashboards:
  - DASHBOARD_GUID_1
  publishConfigs:
  - id: default
    name: Email dashboard PDF
    channels:
    - id: send-email
      name: Email PDF report to leadership
      type: email
      subject: Average transaction duration by application and URI
      from: [email protected]
      to: [email protected], [email protected]
      cc: [email protected], [email protected]

Except for a few cases, channel configuration parameters defined in a manifest file can be specified directly at the channel level or at the enclosing publishConfiguration level or report level or in the global variables element. This is possible because the channel implementations lookup most channel configuration parameters via the report execution context. This behavior makes it possible to "share" channel configuration parameters between channels. For instance, in the example below, the email subject configuration parameter is defined at the report level and the email from configuration parameter is defined in the global variables element. In this case, both the send-email-1 channel and the send-email-2 channel will use the same subject value and the same from value.

variables:
  from: [email protected]
reports:
- id: example-dashboard-report
  name: Example dashboard report
  dashboards:
  - DASHBOARD_GUID_1
  subject: Average transaction duration by application and URI
  publishConfigs:
  - id: default
    name: Email dashboard PDF
    channels:
    - id: send-email-1
      name: Email PDF report to leadership
      type: email
      to: [email protected]
    - id: send-email-2
      name: Email PDF report to team
      type: email
      to: [email protected]

Some channel configuration parameters can only be specified via environment variables as they hold sensitive information. For example, to specify the SMTP user password when using the email channel type, the environment variable EMAIL_SMTP_PASS must be used.

Channel parameter scoping

When using a manifest file to specify channel parameters, parameters are implicitly scoped to the channel where they are defined. In contrast, most environment variables apply "globally" to all channels. Except in a few cases, they can not be scoped to a specific channel. For example, there is no way to specify different email subjects for different channels using the EMAIL_SUBJECT environment variable. For this reason, it is recommended to use a manifest file for those channel parameters that support it.

For channel configuration parameters that can only be specified via environment variables, a "namespacing" mechanism is supported for scoping these parameters to a specific channel. Using this mechanism, a channel configuration parameter can be scoped to a specific channel, publish configuration or report by prefixing the configuration parameter environment variable name with the ID of the corresponding channel, publish configuration, or report, in upper-case, with all non-alphanumeric characters replaced with the _ character.

For example, to scope the EMAIL_SMTP_PASS to a channel with the ID send-mail, use an environment variable with the name SEND_MAIL_EMAIL_SMTP_PASS.

Channel parameter interpolation

Some channel parameters can contain substitution variable references of the form {{ NAME }}. At publish time, these references will be replaced by substituting the reference with the value of the substitution variable named NAME (this is called interpolation). If no variable exists with the name NAME, the reference will be replaced with the name of the variable itself. For example, to include the report name in a channel parameter that supports interpolation, use the string {{ REPORT_NAME }} in the value for the channel parameter.

The supported substitution variables are listed below.

Name	Description
REPORT_ID	The ID of the report being run
REPORT_NAME	The name of the report being run
PUBLISH_CONFIG_ID	The ID of the publish configuration containing the channel where this reference occurs
PUBLISH_CONFIG_NAME	The name of the publish configuration containing the channel where this reference occurs
CHANNEL_ID	The ID of the channel where this reference occurs
CHANNEL_NAME	The name of the channel where this reference occurs
TIMESTAMP	The current time as the number of milliseconds since the epoch
DATETIME	A string containing the current date and time in the form ${year}-${month}-${day}_${hour}${minutes}${seconds}
RESULTS	The report output. NOTE: For the webhook channel, the output is passed through JSON.stringify() to convert it to a JSON string.
RESULTS_CSV_FORMATTED	The report output interpreted as CSV content and formatted as a simple text-based table. NOTE: This token should only be used for query reports. Otherwise report delivery will fail.

In addition, The subject and body channel parameters of the email channel support the following tokens.

Name	Description
EMAIL_FROM	The email address of the sender
EMAIL_TO	The email addresses of the recipients. Multiple email addresses will be separated by commas.
EMAIL_CC	The email addresses of copy recipients. Multiple email addresses will be separated by commas.
EMAIL_SUBJECT	The email subject. NOTE: If this parameter is used in the subject channel parameter, it will be replaced with the empty string.

Substitution variables may also reference any channel parameters or report properties in a report definition that are primitive types (strings, numbers, booleans).

Finally, for all channel types that support channel parameters that are subject to interpolation, the contextVars and envVars channel parameters may be used to expose substitution variables from the current report execution context or from environment variables. For example, in the following webhook channel definition, the property named foo from the report execution context and the environment variable named BEEP are exposed as substitution variables.

    - id: post-webhook
      name: Post query results to webhook
      type: webhook
      contextVars:
      - foo
      envVars:
      - BEEP
      payload: "{ ... }"

The additional substitution variables would be referenced as {{ foo }} and {{ BEEP }}.

Specifying channels

When the reporting engine is run using a manifest file, the channels to use for a given report are part of the channel definition of the selected publish configuration for the report as specified in the manifest file.

If the reporting engine is run without a manifest file, the channels to use are determined as follows.

• If the reporting engine was run from the CLI, the value of the -c option will be used.
• If the reporting engine was run using the AWS Lambda, the value of the channelIds parameter specified in the options element of the AWS Lambda event will be used.
• In either case, if no channel IDs are found, the reporting engine will use the value of the CHANNEL_IDS environment variable.

In all cases, if a non-empty value is found, it is interpreted as a comma-separated list of channel IDs. For example, the value s3,email specifies that the report output(s) should be published to the S3 and Email channels.

If no channel is specified using any of the above mechanisms, report output(s) will be published to the file channel when running from the CLI or the s3 channel when running using the AWS Lambda

As an example, the CLI command used in the run a query report section could have explicitly specified the file channel as follows.

./nr-reports-cli/bin/nr-reports.sh -a 1234567 -q "SELECT average(duration) as 'Duration' FROM Transaction FACET appName as 'Application Name', request.uri AS 'URL'" -c file

This is not necessary since the file channel is the default. However, to use the email channel instead, it would be specified as follows.

./nr-reports-cli/bin/nr-reports.sh -a 1234567 -q "SELECT average(duration) as 'Duration' FROM Transaction FACET appName as 'Application Name', request.uri AS 'URL'" -c email

NOTE: Channel parameters can not be specified when using the CHANNEL_IDS environment variable, the -c CLI option, or the channelIds option to the AWS Lambda function, or when using the default file or s3 channels. In these cases, the respective channel implementation will attempt to locate the configuration parameters it needs in the environment (where supported) and will default any optional parameters.

Output File Name

For report types that produce file output, file names are calculated as follows.

• For dashboard reports where the combinePdfs is not set or set to false, the snapshot for each dashboard GUID specified in the report definition will be saved to a file named dashboard-<GUID>.pdf.
• For dashboard reports where the combinePdfs is set to true, the snapshots for all dashboards will be saved in a file named consolidated_dashboards.pdf.

For report types that produce text output, most of the channel implementations support the option to save the output to a file and to use the file in place of or in conjuction with the assets published by the channel. For example, in addition to being able to copy file(s) from report types that produce file(s), the file channel supports the ability to save text data to a file from report types that produce text.

In such cases, the name of the file to which the text data is written is determined as follows.

• If a property named outputFileName exists in the report execution context, it will be used as the file name.
• Otherwise, the file name will be set to <REPORTID>.<EXT>, calculated as follows:
  • If a manifest file is used to run the report, <REPORTID> will be the value of the id attribute of the report definition. Otherwise, for query reports, <REPORTID> will be query-report.
  • <EXT> will be set to the value of the property named fileExtension in the report execution context. If no such property exists, it will be set to the value of the FILE_EXTENSION environment variable. If no value is specified for the FILE_EXTENSION environment variable, the extension csv will be used for query reports without the passThrough option set on the report definition and the extension txt will be used in all other cases.

File Channel

The file channel type writes generated report output to a destination directory on the local filesystem. It is mostly meant for development and testing although it could be used to copy reports to volumes locally attached to a Docker container.

For report types that produce file output, all generated files are moved from the temporary work directory created when the reporting engine starts into the destination directory.

For report types that produce text, the text will be written to a file. The file name will be calculated as specified in the section Output File Name. If the calculated file name is absolute, the file will be written at the specified location. Otherwise, the file will be written to the destination directory. Any intermediate directories along the path to the destination directory will be automatically created if they do not exist.

The following channel parameters are supported for the file channel type.

Name	Environment Variable	Description	Required	Default
destDir	FILE_DEST_DIR	Output destination directory	N	. (current working directory)

Here is an example of specifying a file channel configuration in a manifest file.

reports:
- id: example-dashboard-report
  name: Example dashboard report
  dashboards:
  - DASHBOARD_GUID_1
  publishConfigs:
  - id: default
    name: Copy file
    channels:
    - id: copy-file-to-tmp
      name: Copy file to /tmp
      type: file
      destDir: "/tmp"

NOTE: The file channel type is the default channel when running from the CLI.

Email Channel

The email channel type delivers report output via email.

Messages are sent over SMTP using the Nodemailer module.

Messages are constructed based on the report output type and channel configuration parameters using the following process.

• A new message is created using the Recipient(s), Sender, Cc, and Subject values specified in the current report execution context or the corresponding environment variables. Prior to being set, channel parameter interpolation is applied to the Subject using the current report execution context.
• If the report type produces file output:
  • All generated files are added to the message as attachments.
  • The body of the message is generated by performing channel parameter interpolation on the message body found in the body parameter in the channel configuration. If no body was specified, a simple default message body is used.
• If the report type produces text:
  • If the attachOutput channel configuration parameter is set to true, the text will be rendered using the format determined by the report type. For example, by default the text output from a query report will be rendered as CSV. The resulting output is written to a file. The file name will be calculated as specified in the section Output File Name. The file name must NOT be absolute. Message construction will continue as if the report type produces file output with the generated file as the output, with one exception. If the passThrough parameter in the channel configuration is set to true, the message body will be generated by re-rendering the text as HTML regardless of the format determined by the report type.
  • If the passThrough parameter in the channel configuration is set to true, the message body will be generated by rendering the text as HTML regardless of the format determined by the report type.
  • Otherwise, the text will be rendered using the format determined by the report type and added to the report execution context with the property name result. The body of the message is then generated by performing channel parameter interpolation on the message body found in the body parameter in the channel configuration. If no body was specified, a simple default message body is used.
• The content type for the message is set using the format parameter in the channel configuration. If no format was specified, the content type is set to html by default. If the format parameter is set but is anything other than html or text, an error is raised.

The following channel parameters are supported for the email channel type.

Name	Environment Variable	Description	Required	Default
to	EMAIL_TO	Recipient emails; Multiple email addresses can be specified separated by commas.	Y
cc	EMAIL_CC	CC recipient emails; Multiple email addresses can be specified separated by commas.	N
from	EMAIL_FROM	Sender email	Y
subject	EMAIL_SUBJECT	Subject line (supports channel parameter interpolation)	N	''
body	N/a	The message body (supports channel parameter interpolation)	N	see the channel implementation
emailSmtpServer	EMAIL_SMTP_SERVER	SMTP server hostname	Y
emailSmtpPort	EMAIL_SMTP_PORT	SMTP server port	N	587
emailSmtpSecure	EMAIL_SMTP_SECURE	SMTP TLS option; true/yes/on/1 forces TLS, anything else defaults to no TLS unless the server upgrades with STARTTLS	N	true
N/a	EMAIL_SMTP_USER	Username for SMTP authentication	N
N/a	EMAIL_SMTP_PASS	Password for SMTP authentication; only used if EMAIL_SMTP_USER is also specified	N
format	N/a	Email format; html/text	N	html
passThrough	N/a	Flag to enable rendering text report output as HTML (see above for more details)	N
attachOutput	N/a	Flag to enable attaching rendered text report output as a file	N
contextVars	N/a	List of context variable names to expose as substitution variables	N
envVars	N/a	List of environment variable names to expose as substitution variables	N

NOTE: When using a manifest file, the body, format, passThrough, attachOutput, contextVars, and envVars parameters can only be specified in the channel parameters. They are not looked up via the report execution context.

NOTE: All EMAIL_SMTP_* environment variables support channel parameter scoping.

Here is an example of specifying an email channel configuration in a manifest file.

reports:
- id: example-dashboard-report
  name: Example dashboard report
  dashboards:
  - DASHBOARD_GUID_1
  publishConfigs:
  - id: default
    name: Email dashboard PDF
    channels:
    - id: send-email
      name: Email PDF report to leadership
      type: email
      subject: Average transaction duration by application and URI
      from: [email protected]
      to: [email protected], [email protected]
      cc: [email protected], [email protected]
      body: |
        <!doctype html>
        <html class="no-js" lang="">

        <head>
          <meta charset="utf-8">
          <meta name="viewport" content="width=device-width, initial-scale=1">
        </head>

        <body>
          <main>
            <h1>{{ REPORT_NAME }}</h1>
            <p>Please find the attached report which shows average transaction duration
            by application.</p>
          </main>
        </body>

        </html>

Because why not? Everyone needs more email.

S3 Channel

The s3 channel type uploads generated report output to an S3 bucket.

For report types that produce file output, all generated files are uploaded from the temporary work directory created when the reporting engine starts into the S3 bucket.

For report types that produce text, the text will be rendered using the format determined by the report type. The resulting output will be uploaded to the S3 bucket as an S3 object. The key for the object will be calculated as specified in the section Output File Name. NOTE: The calculated file name must be a valid S3 key name.

The following channel parameters are supported for the s3 channel type.

Name	Environment Variable	Description	Required	Default
bucket	S3_DEST_BUCKET	Destination S3 bucket	Y

NOTE: The S3_DEST_BUCKET environment variable supports channel parameter scoping.

Here is an example of specifying an s3 channel configuration in a manifest file.

reports:
- id: example-dashboard-report
  name: Example dashboard report
  dashboards:
  - DASHBOARD_GUID_1
  publishConfigs:
  - id: default
    name: Copy dashboard PDF to S3 bucket
    channels:
    - id: copy-file
      name: Copy to S3
      type: s3
      bucket: "Daily Report"

The s3 channel type is the default channel when running from a Lambda.

Slack Channel

The slack channel type posts report output to a Slack channel via a Slack webhook. A Slack webhook URL must be specified using the SLACK_WEBHOOK_URL environment variable.

Because Slack webhooks do not support transferring files, the slack channel type only supports report types that produce text. Specifying a slack channel type for a report type that produces file output will cause a warning message to be logged.

Messages are constructed using the following process.

• If the passThrough parameter in the channel configuration is set to true, the body of the message is set to the raw report output. In this case, the report output must be a JSON string that conforms to the Incoming Webhook JSON payload format. This method can be used to send messages containing BlockKit visual components.
• Otherwise, the body of the message is set to a JSON object with a single property named text with the text output as it's value after applying channel parameter interpolation. The text output is automatically escaped so that it may safely include all mrkdwn formatting options.

The following channel parameters are supported for the slack channel type.

Name	Environment Variable	Description	Required	Default
N/a	SLACK_WEBHOOK_URL	Slack Webhook URL	Y
message	N/a	The message text (supports channel parameter interpolation)	N	see the channel implementation
passThrough	N/a	Flag to enable sending raw report output	N	false
contextVars	N/a	List of context variable names to expose as substitution variables	N
envVars	N/a	List of environment variable names to expose as substitution variables	N

NOTE: When using a manifest file the passThrough parameter can only be specified in the channel element.

NOTE: When using a manifest file, the message, passThrough, contextVars, and envVars parameters can only be specified in the channel parameters. They are not looked up via the report execution context.

NOTE: The SLACK_WEBHOOK_URL environment variable supports channel parameter scoping.

Here is an example of specifying a Slack channel configuration in a manifest file.

reports:
- id: example-query-report
  query: "SELECT average(duration) as 'Duration' FROM Transaction FACET appName as 'Application Name', request.uri"
  accountIds:
  - 1234567
  timeout: 10
  publishConfigs:
  - id: slack
    name: Post to Slack
    channels:
    - id: post-slack
      name: Post query results to Slack
      type: slack
      message: |
        {{ REPORT_NAME }}

        Results from the average transaction duration query.

        {{ RESULTS_CSV_FORMATTED }}

Webhook Channel

The webhook channel type sends report output to any Webhook endpoint by including the report output in a Webhook payload or using the report output as the payload itself. A webhookURL must be specified using the WEBHOOK_URL environment variable. The webhook channel type supports specifying the HTTP method, custom HTTP headers, and HTTP basic user-password authentication.

The webhook channel type only supports report types that produce text. Specifying a webhook channel type for a report type that produces file output will cause a warning message to be logged.

Webhook payloads are constructed using the following process.

• If the passThrough parameter in the channel configuration is set to true, the body of the message is set to the raw report output The report output must be in a valid format that the Webhook endpoint will accept (often JSON).
• Otherwise, the body of the message is set to the value of the payload channel parameter after applying channel parameter interpolation. The value of the payload channel parameter with the applied substitutions must be in a valid format that the Webhook endpoint will accept (often JSON).

Webhook payloads are sent using the following process.

• Build the webhook payload per the process above.
• Build the request headers using the following process.
  • If an HTTP Basic username and password are set, compute the HTTP Basic credentials string and add an Authorization header with the computed value.
  • Given the parameter names webhookHeaderN or environment variable names WEBHOOK_HEADER_N where N is the number 1-5, if a parameter or environment environment variable exists with the given name with a value of the form header-name: value, add a header with the name header-name and the value value after applying channel parameter interpolation.
• Make an HTTP request with the HTTP method specified in the parameter webhookHttpMethod, the environment variable WEBHOOK_HTTP_METHOD, or using the value POST to the URL specified in the parameter webhookUrl or the environment variable WEBHOOK_URL using the set of headers and payload built in the previous steps.

The following channel parameters are supported for the webhook channel type.

Name	Environment Variable	Description	Required	Default
webhookUrl	WEBHOOK_URL	Webhook URL endpoint	Y
webhookHttpMethod	WEBHOOK_HTTP_METHOD	Webhook HTTP Method; GET/POST/PUT	N	POST
N/a	WEBHOOK_HTTP_BASIC_USER	Webhook HTTP Basic authentication username	N
N/a	WEBHOOK_HTTP_BASIC_PASS	Webhook HTTP Basic authentication password	N
webhookHeaderN	WEBHOOK_HEADER_N	Custom webhook HTTP header where N is the number 1-5 and the value is of the form header-name: value (supports channel parameter interpolation)	N
payload	N/a	Webhook payload (supports channel parameter interpolation)	N
passThrough	N/a	Flag to enable sending raw report output	N	false
contextVars	N/a	List of context variable names to expose as substitution variables	N
envVars	N/a	List of environment variable names to expose as substitution variables	N

NOTE: When using a manifest file the payload, passThrough, contextVars, and envVars parameters can only be specified in the channel parameters. They are not looked up via the report execution context.

NOTE: All WEBHOOK_* environment variables support channel parameter scoping.

Here is an example of specifying a webhook channel configuration in a manifest file.

reports:
- id: example-query-report
  query: "SELECT average(duration) as 'Duration' FROM Transaction FACET appName as 'Application Name', request.uri"
  accountIds:
  - 1234567
  timeout: 10
  publishConfigs:
  - id: external-webhook
    name: Post to external webhook
    channels:
    - id: post-webhook
      name: Post query results to webhook
      type: webhook
      webhookHeader1: "X-Foo: bar"
      payload: |
        {
          "reportId": "{{ REPORT_ID }}",
          "reportName": "{{ REPORT_NAME }}",
          "publishConfigId": "{{ PUBLISH_CONFIG_ID }}",
          "publishConfigName": "{{ PUBLISH_CONFIG_NAME }}",
          "channelId": "{{ CHANNEL_ID }}",
          "channelName": "{{ CHANNEL_NAME }}",
          "results": {{ RESULTS }}
        }

Custom webhook HTTP headers

The webhook channel type supports sending custom HTTP headers on the Webhook HTTP request. Up to 5 custom HTTP headers may be specified. Headers may be specified as channel parameters in the manifest file or as environment variables. To specify the custom HTTP headers in the manifest file, use the parameter name webhookHeaderN, where the N is the number 1-5. For example, webhookHeader1, webhookHeader2, and so on. To specify custom HTTP headers as environment variables, use the environment variable name WEBHOOK_HEADER_N, where N is the number 1-5. For example, WEBHOOK_HEADER_1, WEBHOOK_HEADER_2, and so on.

The value of each custom header must be specified using the format header-name: value where header-name is the name of the custom HTTP header to add and value is the value to send for the custom HTTP header. For example, to specify a header named X-Foo with the value bar, the value for the channel parameter would be X-Foo: bar.

Manifest File

A manifest file is a JSON or YAML file with the following format.

Manifest File Format

A manifest file contains a single top-level object with the following properties.

Property Name	Description	Type	Required	Default
variables	A map of "global" properties that are added to the report execution context of all reports.	object	N	{}
reports	An array of report definitions.	array	N	[]

A full YAML example is shown below.

variables:
  subject: Average transaction duration by application and URI
reports:
- id: example-dashboard-report
  name: Example dashboard report
  dashboards:
  - DASHBOARD_GUID_1
  publishConfigs:
  - id: default
    name: Copy file
    channels:
    - id: copy-file-to-tmp
      name: Copy file to /tmp
      type: file
      destDir: "/tmp"
  - id: copy-pdf-to-s3
    name: Copy dashboard PDF to S3 bucket
    channels:
    - id: copy-file
      name: Copy to S3
      type: s3
      bucket: "Daily Report"
  - id: email-pdf
    name: Email dashboard PDF
    channels:
    - id: send-email
      name: Email PDF report to leadership
      type: email
      subject: Average transaction duration by application and URI
      from: [email protected]
      to: [email protected], [email protected]
      cc: [email protected], [email protected]
- id: example-query-report
  query: "SELECT average(duration) as 'Duration' FROM Transaction FACET appName as 'Application Name', request.uri"
  accountIds:
  - 1234567
  timeout: 10
  publishConfigs:
  - id: slack
    name: Post to Slack
    channels:
    - id: post-slack
      name: Post query results to Slack
      type: slack
  - id: external-webhook
    name: Post to external webhook
    channels:
    - id: post-webhook
      name: Post query results to webhook
      type: webhook
      webhookHeader1: "X-Foo: bar"
      payload: |
        {
          "reportId": "{{ REPORT_ID }}",
          "reportName": "{{ REPORT_NAME }}",
          "publishConfigId": "{{ PUBLISH_CONFIG_ID }}",
          "publishConfigName": "{{ PUBLISH_CONFIG_NAME }}",
          "channelId": "{{ CHANNEL_ID }}",
          "channelName": "{{ CHANNEL_NAME }}",
          "results": {{ RESULTS }}
        }

The variables map can contain any properties. These properties are added to the report execution context for all reports.

The reports array contains a list of report definitions.

Report Definitions

A report definition is an object with a set of common properties and one or more additional properties that are particular to the report type. The following sections show the supported common properties and the properties supported by each report type.

An example manifest file is provided in the examples directory in both JSON and YAML format.

Common Properties

The following properties are common to all report types.

Property Name	Description	Type	Required	Default
id	The report identifier	string	Y
name	The report name	string	N
publishConfigs	The list of publish configurations for the report	array	N	(see publish configurations)

Dashboard Report Properties

Property Name	Description	Type	Required	Default
dashboards	An array of dashboard entity GUIDs	array	Y
combinePdfs	true to combine all PDFs whan more than one dashboard is specified or false to use separate PDFs.	boolean	N	false

Query Report Properties

Property Name	Description	Type	Required	Default
accountId	An account ID to run the query with. One of the this property or the accountIds property must be specified.	number	Y if accountIds is not specified
accountIds	A list of account IDs to run the query with. A maximum of 5 account IDs is allowed if multiAccountMode is set to cross-account. One of the this property or the accountId property must be specified.	array	Y if accountId is not specified
query	The NRQL query to run.	string	Y
multiAccountMode	The method used to query multiple accounts when multiple account IDs are specified. Valid values are cross-account, per-account, and per-account-concurrent.	string	N	cross-account
timeout	The query timeout in seconds. Must be between 5 and 120 seconds.	number	N	5

Engine Options

The reporting engine supports several options which control various aspects of it's behavior. When running from the CLI, these options can be specified using the CLI options. When running from a Lambda, these options can be specified in the options object in the event object (or event.body object) passed to the handler function. In both cases, these options can also be specified via environment variables. Options specified via CLI options or the event payload take precedence over environment variables.

The following options are supported. For more information on the CLI options, see the section Using the CLI. For more information on the Lambda options, see the section Using the AWS Lambda Function.

Option	Description	CLI Option	Lambda Option	Environment Variable
Manifest file	Path to a manifest file	-f	manifestFilePath	MANIFEST_FILE_PATH
Report IDs	List of report IDs to run. Multiple report IDs can be specified separated by commas. Ignored if a manifest file is not specified.	-r	reportIds	REPORT_IDS
Publish Configuration IDs	List of publish configuration IDs used during publish configuration selection. Ignored if a manifest file is not specified.	-u	publishConfigIds	PUBLISH_CONFIG_IDS
Dashboard IDs	List of dashboard entity GUIDs. Multiple dashboard entity GUIDs can be specified separated by commas. Ignored if a manifest file is specified.	-d	dashboardIds	DASHBOARD_IDS
NRQL Query	An NRQL query. Ignored if a manifest file or dashboard IDs are specified.	-q	nrqlQuery	NRQL_QUERY
Account IDs	List of account IDs to use with a query report. Multiple account IDs can be specified separated by commas (see note below). Required if a NRQL query is specified.	-a	accountId	NEW_RELIC_ACCOUNT_ID
Channel IDs	List of channel IDs. Multiple channel IDs can be specified separated by commas. Ignored if a manifest file is specified.	-c	channelIds	CHANNEL_IDS
Output file name	Output file name to use for the output file when saving output to a file when running a query report with the file or s3 channels. Ignored if a manifest file or dashboard IDs are specified.	-o	outputFileName	n/a
Verbose mode	Enable verbose logging mode	--verbose	n/a	Set LOG_LEVEL to verbose
Debug mode	Enable debug (very verbose) logging mode	--debug	n/a	Set LOG_LEVEL to debug
S3 Source Bucket	Name of S3 bucket to read manifest file from.	Not supported	sourceBucket	SOURCE_BUCKET

NOTE: As mentioned above, multiple account IDs can be specified via the -a CLI option and the accountId Lambda option by separating each ID with a ,. You should not do this with the NEW_RELIC_ACCOUNT_ID environment variable when running using the AWS Lambda function as this value is also used by the New Relic Lambda extension.

Additionally, the engine uses the following environment variables.

| Environment Variable | Description | Required | Default | | --- | --- | | NEW_RELIC_API_KEY | A New Relic User API key | Y | N/a | | NEW_RELIC_REGION | A New Relic datacenter region identifier. One of US or EU. | N | US |

Using the CLI

The New Relic Reports CLI runs reports using the New Relic Reports engine. It is used by the CLI image. Reports can also be run directly from the command line using the provided wrapper script. However, usage at the command line is mostly meant to be used locally for development and testing purposes.

The reports to run can be specified via the CLI options or environment variables. When the engine starts, it resolves the set of reports to process in the following order of precedence.

• The -f option or MANIFEST_FILE_PATH environment variable
• The -d option or DASHBOARD_IDS environment variable
• The -q option or NRQL_QUERY environment variable

If none of the options or environment variables are specified, the engine will attempt to load a manifest file at the path include/manifest.json.

Refer to the Options section or for additional options and details.

CLI Usage

nr-reports-cli/bin/nr-reports.sh -f <manifest-file>
nr-reports-cli/bin/nr-reports.sh -d <dashboard-ids> [-c <channel-ids>]
nr-reports-cli/bin/nr-reports.sh -q <nrql-query> -a <account-id> [-c <channel-ids>] [-o <output-file>]

CLI Options

• --help

  Show help

• --version

  Show version number

• -f, --manifest

  Run all reports defined in the manifest file <manifest-file>. Takes precedence over -n, -d, and -q and their corresponding environment variables.

  The MANIFEST_FILE_PATH environment variable may also be used to specify a manifest file. If both are specified, the -f option takes precedence.

• -r, --report-name

  Run only the reports with report IDs listed in <report-ids>. Report IDs are separated by commas. Ignored if a manifest file is not specified.

  The REPORT_IDS environment variable may also be used to specify report IDs. If both are specified, the -r option takes precedence.

  NOTE: This option refers to report names for historical reasons. However, as of v3.0 this option takes report IDs.

• -u, --publish-config-ids

  Publish report outputs using the first publish configuration with an ID that matches an ID in the list <publish-config-ids> for each report. Publish configuration IDs are separated by commas. If no match is found the publish configuration with the ID default is used. Ignored if a manifest file is not specified.

  The PUBLISH_CONFIG_IDS environment variable may also be used to specify publish configuration IDs. If both are specified, the -u option takes precedence.

• -d, --dashboard-ids

  Run a dashboard report with the dashboard GUIDs listed in <dashboard-ids>. Dashboard GUIDs are separated by commas. Takes precedence over -q. Ignored if a manifest file is specified.

  The DASHBOARD_IDS environment variable may also be used to specify the dashboard GUIDs. If both are specified, the -d option takes precedence.

• -q, --nrql-query

  Run a query report with the NRQL query <nrql-query>. Requires -a. Ignored if a manifest file or a dashboard GUID string is specified.

  The NRQL_QUERY environment variable may also be used to specify the a NRQL query. If both are specified, the -q option takes precedence.

• -a, --account-id

  Use the account <account-id> when running a query report with -q. Multiple account IDs are separated by commas. Required with -q.

• -c, --channel-ids

  Publish report output to the channels listed in <channel-ids>. Channel IDs are separated by commas. Ignored if a manifest file is specified.

• -o, --output-file

  Use <output-file> as the name of the file when saving output to a file when running a query report with the file or s3 channels. Ignored if a manifest file or dashboard GUID string is specified.

• --verbose

  Enable verbose mode.

• --debug

  Enable debug mode (be very verbose).

Preparing to use the CLI

Prior to working with the CLI you will need to perform the following steps to configure the terminal session where the CLI will be run.

1. Set the environment variable named NEW_RELIC_API_KEY to your New Relic User API key by executing the following command.

   export NEW_RELIC_API_KEY="[YOUR USER KEY]"

2. Optionally to use the EU datacenter execute the following command.

   export NEW_RELIC_REGION="EU"

CLI Examples

The examples shown below use the ./nr-reports-cli/bin/nr-reports.sh wrapper.

• Run a dashboard report to export a snapshot of the dashboard with the GUID ABCDEF123456 and save it to a file.

  ./nr-reports-cli/bin/nr-reports.sh -d ABCDEF123456

  In this example, the reporting engine will export the dashboard snapshot as a PDF and publish the PDF to the default file channel. The file channel will copy the PDF to the current working directory as a file named dashboard-ABCDEF123456.pdf.

• Run a query report that executes the specified NRQL query against account 12345 and saves the query result in a CSV file.

  ./nr-reports-cli/bin/nr-reports.sh \
    -a 12345 \
    -q "SELECT average(duration) as 'Duration' FROM Transaction FACET appName as 'Application Name', request.uri"

  In this example, the reporting engine will run the given NRQL query against account 12345, convert the query result to CSV data and publish the CSV data to the default file channel. The file channel will copy the CSV file to the current working directory as a file named query-report.csv.

• Run all reports specified in the given manifest file.

  ./nr-reports-cli/bin/nr-reports.sh -f /tmp/manifest.yaml

  In this example, the reporting engine will read all report definitions from the specified manifest file and run each report in turn.

Using the CLI image

A Dockerfile is provided to build a Docker image that provides an ENTRYPOINT that runs the CLI with no arguments. Arguments can be passed to the the CLI via arguments to the docker run command. Engine options can also be specified as environment variables. This image is meant to be used in conjuction with external scheduled task mechanisms such as AWS ECS Scheduled Tasks to run reports on a schedule. It can also be used as a base image. It can also be used as a way to test and debug reports locally without needing to have everything required to run the CLI available on the local machine. This can be more inconvenient than running the CLI directly on the local machine but has the benefit that it will produce reports in the exact environment they will be run when the image is deployed.

Building the CLI image

In addition to the CLI Dockerfile, the build.sh script is provided to simplify building the CLI image. It supports the following options.

Option	Description	Example
--image-repo image-repository	The repository to use when tagging the image. Defaults to nr-reports.	--image-repo nr-reports
--image-tag image-tag	The tag to use when tagging the image. Defaults to latest.	--image-tag 1.0

You can either run the script directly or use the npm run build command while in the ./nr-reports-cli directory.

Here are a few examples.

• Build an image using all the defaults. The image will be tagged with nr-reports:latest in the local Docker registry.

  cd ./nr-reports-cli
  npm run build

• Build an image with a custom image name. The image will be tagged with my-great-reports:1.1 in the local Docker registry.

  cd ./nr-reports-cli
  npm run build -- --image-repo my-great-reports --image-tag 1.1

Running the CLI image

The following examples show how you can run reports using the CLI image. Though the image is intended to be used in conjuction with a scheduled task mechanism, it can be helpful for testing and debugging reports in the exact environment they will be run when the image is deployed rather than running in a local environment which may not be consistent with the deployed image.

NOTE: In the examples below, the AWS configuration and credential files in the local .aws directory are mounted into the home directory of the pptruser in the container so that the AWS SDK for Node.js has access to the AWS configuration and credentials without having to pass those via arguments on the command line. This is only done for example purposes and in general should only be used locally when testing and developing reports.

Running a report using the default manifest file with the CLI image

The example below uses the default manifest file located at include/manifest.json. The channels and the channel parameters are specified in the manifest file, except for those specified with the -e option.

docker run --rm -e NEW_RELIC_API_KEY='[YOUR_USER_API_KEY]' \
    --cap-add=SYS_ADMIN \
    --name nr-reports \
    -e EMAIL_SMTP_SERVER='[YOUR_SMTP_SERVER]' \
    -e EMAIL_SMTP_PORT=[YOUR_SMTP_SERVER_PORT] \
    -e EMAIL_SMTP_SECURE='true or false' \
    -v /path/to/.aws:/home/pptruser/.aws \
    nr-reports

NOTE: To use an account located in the EU datacenter, additionally add the following option.

    -e NEW_RELIC_REGION=EU

Running a report using a custom manifest file with the CLI image

The example below uses a custom manifest file located at include/custom-manifest.json. The channels and the channel parameters are specified in the manifest file, except for those specified with the -e option.

docker run --rm -e NEW_RELIC_API_KEY='[YOUR_USER_API_KEY]' \
    --cap-add=SYS_ADMIN \
    --name nr-reports \
    -e EMAIL_SMTP_SERVER='[YOUR_SMTP_SERVER]' \
    -e EMAIL_SMTP_PORT=[YOUR_SMTP_SERVER_PORT] \
    -e EMAIL_SMTP_SECURE='true or false' \
    -v /path/to/.aws:/home/pptruser/.aws \
    nr-reports -f include/custom-manifest.json

NOTE: To use an account located in the EU datacenter, additionally add the following option.

    -e NEW_RELIC_REGION=EU

Using the AWS Lambda function

The reporting engine can be also be deployed as an AWS Lambda function. The Lambda function can be combined with other AWS services to trigger report generation in a variety of ways. For example, an AWS EventBridge trigger can be used to run reports on a schedule. Or, an Application Load Balancer trigger can be used to expose an HTTP endpoint for generating reports on demand by making a request to the endpoint.

NOTE: While the reporting engine can be deployed as a Lambda function individually, the recommended way to deploy and run the Lambda function is using the New Relic Reports AWS Stack along with the New Relic Reports Builder for a complete end-to-end scheduled reporting solution.

The AWS Lambda function and S3

The Lambda function supports reading manifest files from Amazon S3 if the sourceBucket engine option is set. For example, if the sourceBucket option is set to my-in-bucket and the manifestFile option is set to my-manifest.json, the AWS Lambda function will load the object with the key my-manifest.json in the S3 bucket my-in-bucket.

In addition, if a source bucket is specified and no channel is specified for a report, the default being s3, or s3 is specified as a channel without a destination bucket, the AWS Lambda function will default the destination bucket to the source bucket.

Preparing to work with the AWS Lambda function

Prior to working with the Lambda function you will need to perform the following steps.

1. Create an Amazon ECR repository to host the Lambda container image.
2. Install the AWS CLI on the build machine.
3. Configure the AWS region used by the AWS CLI in the AWS CLI configuration file or using the AWS_REGION environment variable.
4. Configure AWS credentials for programmatic access by the AWS CLI appropriately.
5. Copy the file cf-params-sample.json to a file named cf-params.deploy.json in the ./nr-reports-lambda/deploy directory.
6. Replace the ParameterValue for the RunnerECRImageRepo parameter with the URI of the Amazon ECR repository.
7. Replace the ParameterValue for the RunnerECRImageTag parameter with a value to use to tag the image in the Amazon ECR repository.

Build the Lambda container image

When deploying the reporting engine as a Lambda function, the engine is designed to be packaged as a Lambda container image and pushed to a repository hosted in Amazon ECR. Whether deploying the Lambda function individually or deploying it as part of the New Relic Reports AWS Stack, the process to build and push the Lambda container image is the same.

Building and pushing the Lambda container image

A Dockerfile Dockerfile is provided to build a Docker image that can be deployed as a Lambda container image. The image is built from an AWS base image for Node.js. By default, version 20 is used but this can be customized by specifying the AWS_LAMBDA_VER argument when building the image. The image automatically includes the New Relic AWS Lambda Extension Layer corresponding to the version of the base image that is specified.

The build.sh script is provided to simplify building the Lambda container image. It supports the following options.

Option	Description	Example
-t image-name	The image name used to tag the image in your local Docker registry. Defaults to nr-reports-lambda when the build.sh is run directly or nr-reports-lambda:$package_json_version when run using npm run build.	-t nr-reports-lambda-test:3.2.0
--full	Download and include the latest New Relic AWS Lambda Extension Layer. Defaults to false.	--full
--push	Push the image to the ECR repository specified in the RunnerECRImageRepo parameter defined in the deploy/cf-params.deploy.json file. Defaults to false.	--push
-p prefix	The prefix to use when looking up environment variables. This option should always be specified with the value RUNNER when using the build.sh script directly.	-p RUNNER

You can either run the script directly or use the npm run build command while in the ./nr-reports-lambda directory. For example, once you have met the prerequisites to build and push the Lambda container image, to perform a full build of the image using the default image name and version number and automatically push it to the ECR repository specified in the ./deploy/cf-params.deploy.json using the NPM script, run the following command.

npm run build -- --full --push

Upon successful completion, output similar to the following will be displayed.

Pushing image...

The push refers to repository [12345.dkr.ecr.aws_region.amazonaws.com/example-repo-name/nr-reports-lambda]
************: Pushed
************: Pushed
************: Pushed
************: Pushed
************: Pushed
************: Pushed
************: Layer already exists
************: Layer already exists
************: Layer already exists
************: Layer already exists
************: Layer already exists
************: Layer already exists
X.Y.Z: digest: sha256:**************************************************************** size: 2840
Done.

NOTE: While the build step can be run individually, it can automatically be run when the Lambda function is deployed or updated.

Deploy the Lambda function

The Lambda function is deployed and managed as a CloudFormation stack using the provided CloudFormation template and the CloudFormation parameters file ./nr-reports-lambda/deploy/cf-params.deploy.yml created when preparing to work with the AWS Lambda function.

Preparing to deploy or update the AWS Lambda function

Prior to deploying or updating the Lambda function, you will need to perform the following steps.

1. Perform all steps required to work with the AWS Lambda function.

2. Identify or create a function execution role that the Lambda service will assume to invoke the function.

3. Optionally create an AWS Secrets Manager secret in which to store the New Relic license key used by the New Relic AWS Lambda Extension.

   This secret should have a single key-value pair with the key LicenseKey and your New Relic License Key as the value. Replace the ParameterValue for the NRLicenseKeySecret parameter in the cf-params.deploy.json file with the ARN of the secret.

   NOTE: You may also choose to specify the New Relic License Key as a Lambda environment variable by replacing the ParameterValue for the NRLicenseKey parameter in the cf-params.deploy.json file with your New Relic License Key. However, this is not recommended for security purposes.

4. Optionally create an AWS Secrets Manager secret in which to store the New Relic User API key used by the Lambda function.

   If you do not intend to use the Lambda function with the New Relic AWS Scheduler Lambda, this secret should have a single key-value pair with the key UserApiKey and your New Relic User API key as the value. Replace the ParameterValue for the UserApiKeySecret parameter in the cf-params.deploy.json file with the ARN of the secret.

   If you intend to use the Lambda function with the New Relic AWS Scheduler Lambda, see the scheduler Lambda Secret section for details needed to create the appropriate secret and replace the ParameterValue for the SecretName parameter in the cf-params.deploy.json file with the ARN of the secret.

   NOTE: If you do not intend to use the Lambda function with the New Relic AWS Scheduler Lambda, you may also choose to specify the New Relic User API key as a Lambda environment variable by replacing the ParameterValue for the UserApiKey parameter in the cf-params.deploy.json file with your New Relic User API key. However, this is not recommended for security purposes.

Additionally, update the CloudFormation parameters file ./nr-reports-lambda/deploy/cf-params.deploy.yml created when preparing to work with the AWS Lambda function. Documentation for each parameter in the CloudFormation parameters file is provided inline in the CloudFormation template as comments. For example, below is the documentation for the UserApiKeySecret parameter.

  #
  # Name or ARN of an AWS Secrets Manager secret containing a New Relic User API
  # key used for GraphQL Nerdstorage queries and mutations. By default, the
  # secret must contain a 'UserApiKey' property for the API key. A different key
  # can be specified by the UserApiKeySecretKey parameter.
  #
  UserApiKeySecret:
    Type: String
    Description: ID (name or ARN) of a Secret containing the New Relic User API key.
    AllowedPattern: '(^$)|(^arn:(aws[a-zA-Z-]*)?:secretsmanager:.+:\d{12}:secret:[a-zA-Z0-9/_+=\.@\-]+$)'
    Default: ''

Deploying the Lambda function

The deploy.sh script is used to deploy the CloudFormation stack for the Lambda function. It supports the following options.

Option	Description	Example
-n stack-name	The name to use for the CloudFormation stack. Defaults to nr-reports-lambda.	-n nr-reports-lambda-stack
-t image-name	The image name used to tag the image in your local Docker registry. Defaults to nr-reports-lambda when the deploy.sh is run directly or nr-reports-lambda:$package_json_version when run using npm run deploy.	-t nr-reports-lambda-test:3.2.0
-p prefix	The prefix to use when looking up environment variables. This option should always be specified with the value RUNNER when using the deploy.sh script directly.	-p RUNNER
--no-build	Do not invoke the build.sh script script before deploying. Defaults to false.	--no-build

The deploy.sh script will first invoke the build.sh script to build and push the Lambda container image as outlined in the section Building and pushing the Lambda container image. The build.sh script will be invoked with the --full and --push flags. The -t option specified when the deploy.sh script was invoked will be passed through as well as the -p option.

The deploy.sh script will then use the aws cloudformation deploy command to create the stack using the CloudFormation template and the CloudFormation parameters file ./nr-reports-lambda/deploy/cf-params.deploy.yml.

You can either run the script directly or use the npm run deploy command while in the ./nr-reports-lambda directory. For example, to build, push, and deploy the Lambda function using the NPM script using the default image name and version number and the default stack name, run the following command.

npm run deploy

Upon successful completion, output similar to the following will be displayed.

Deploying stack nr-reports-lambda...

Waiting for changeset to be created..
Waiting for stack create/update to complete
Successfully created/updated stack - nr-reports-lambda
Done.

Update the Lambda function

The update.sh script is used to update the Lambda function to a new version of the container image. It supports the following options.

Option	Description	Example
-n function-name	The name of the function that was specified for the RunnerFunctionName parameter in the CloudFormation parameters file when the Lambda function was deployed.	-n RunNewRelicReport
-t image-name	The image name used to tag the image in your local Docker registry. Defaults to nr-reports-lambda when the update.sh is run directly or nr-reports-lambda:$package_json_version when run using npm run update. Only used then the --build option is specified.	-t nr-reports-lambda-test:3.2.0
-p prefix	The prefix to use when looking up environment variables. This option should always be specified with the value RUNNER when using the update.sh directly.	-p RUNNER
--build	Invoke the build.sh script script before updating. Defaults to false.	--build

The update.sh script uses the aws lambda update-function-code command to notify the Lambda function with the name specified using the -n option that its image has been updated and that it needs to be refreshed. The function-name specified using the -n option should be the name of the function that was specified for the RunnerFunctionName parameter in the CloudFormation parameters file when the Lambda function was deployed.

Additionally if the --build option is specified, the update.sh script will first invoke the build.sh script to build and push the Lambda container image as outlined in the section Building and pushing the Lambda container image. The build.sh script will be invoked with the --full and --push flags. The -t option specified when the update.sh script was invoked will be passed through as well as the -p option.

You can either run the script directly or use the npm run update command while in the ./nr-reports-lambda directory. For example, if the name of the function specified in the RunnerFunctionName parameter when the Lambda function was deployed was RunNewRelicReport, to build, push, and update the Lambda function using the NPM script using the default image name and version number, run the following command.

npm run update -- -n RunNewRelicReport --build

NOTE: As some of the environment variables used by the Lambda function may contain secrets (license key, user API key, and/or other sensitive data) and since the output from the aws lambda update-function-code command displays the value of all environment variables, no output will be displayed after the UPDATE command header.

NOTE: When updating the Lambda function, you may need to increment the value of the RunnerECRImageTag parameter specified in the ./nr-reports-lambda/deploy/cf-params.deploy.json file.

Delete the Lambda function

The delete.sh script is used to delete the CloudFormation stack for the Lambda function. It supports the following options.

Option	Description	Example
-n stack-name	The name of the CloudFormation stack. Defaults to nr-reports-lambda.	-n nr-reports-lambda-stack

You can either run the script directly or use the npm run delete command while in the ./nr-reports-lambda directory. For example, to delete the Lambda function using the NPM script using the default stack name, run the following command.

npm run delete

Upon successful completion, output similar to the following will be displayed.

Deleting stack nr-reports-lambda...
Waiting for stack delete to complete...
Done.

Using the AWS Scheduler Lambda Function

The New Relic Reports Scheduler AWS Lambda function is a key component of the end-to-end scheduled reporting solution provided by the New Relic Reports AWS Stack. The scheduler monitors the reports defined in the New Relic Reports Builder and automatically creates, updates, and deletes AWS EventBridge Cron-based schedules to trigger the AWS Lambda function to run each report based on the schedule defined for it in the New Relic Reports Builder.

NOTE: While the scheduler can be deployed as a Lambda function individually, the recommended way to deploy and run the scheduler is using the New Relic Reports AWS Stack along with the New Relic Reports Builder for a complete end-to-end scheduled reporting solution.

Preparing to work with the AWS Scheduler Lambda function

Prior to working with the scheduler Lambda function you will need to perform the following steps.

1. Create an Amazon ECR repository to host the scheduler Lambda container image.
2. Install the AWS CLI on the build machine.
3. Configure the AWS region used by the AWS CLI in the AWS CLI configuration file or using the AWS_REGION environment variable.
4. Configure AWS credentials for programmatic access by the AWS CLI appropriately.
5. Copy the file cf-params-sample.json to a file named cf-params.deploy.json in the ./nr-reports-scheduler/deploy directory.
6. Replace the ParameterValue for the SchedulerECRImageRepo parameter with the URI of the Amazon ECR repository.
7. Replace the ParameterValue for the SchedulerECRImageTag parameter with a value to use to tag the image in the Amazon ECR repository.

Build the Scheduler Lambda container image

When deploying the scheduler Lambda function, the scheduler is designed to be packaged as a Lambda container image and pushed to a repository hosted in Amazon ECR. Whether deploying the scheduler Lambda function individually or deploying it as part of the New Relic Reports AWS Stack, the process to build and push the scheduler Lambda container image is the same.

Building and pushing the scheduler Lambda container image

A Dockerfile Dockerfile is provided to build a Docker image that can be deployed as a Lambda container image. The image is built from an AWS base image for Node.js. By default, version 20 is used but this can be customized by specifying the AWS_LAMBDA_VER argument when building the image. The image automatically includes the New Relic AWS Lambda Extension Layer corresponding to the version of the base image that is specified.

The build.sh script is provided to simplify building the scheduler Lambda container image. It supports the following options.

Option	Description	Example
-t image-name	The image name used to tag the image in your local Docker registry. Defaults to nr-reports-scheduler when the build.sh is run directly or nr-reports-scheduler:$package_json_version when run using npm run build.	-t nr-reports-scheduler-test:3.2.0
--full	Download and include the latest New Relic AWS Lambda Extension Layer. Defaults to false.	--full
--push	Push the image to the ECR repository specified in the SchedulerECRImageRepo parameter defined in the deploy/cf-params.deploy.json file. Defaults to false.	--push
-p prefix	The prefix to use when looking up environment variables. This option should always be specified with the value SCHEDULER when using the build.sh script directly.	-p SCHEDULER

You can either run the script directly or use the npm run build command while in the ./nr-reports-scheduler directory. For example, once you have met the prerequisites to build and push the scheduler Lambda container image, to perform a full build of the image using the default image name and version number and automatically push it to the ECR repository specified in the ./deploy/cf-params.deploy.json using the NPM script, run the following command.

npm run build -- --full --push

Upon successful completion, output similar to the following will be displayed.

Pushing image...

The push refers to repository [12345.dkr.ecr.aws_region.amazonaws.com/example-repo-name/nr-reports-scheduler]
************: Pushed
************: Pushed
************: Pushed
************: Pushed
************: Pushed
************: Pushed
************: Layer already exists
************: Layer already exists
************: Layer already exists
************: Layer already exists
************: Layer already exists
************: Layer already exists
X.Y.Z: digest: sha256:**************************************************************** size: 2840
Done.

NOTE: While the build step can be run individually, it can automatically be run when the scheduler Lambda function is deployed or updated.

Deploy the Scheduler Lambda function

The scheduler Lambda function is deployed and managed as a CloudFormation stack using the provided CloudFormation template and the CloudFormation parameters file ./nr-reports-scheduler/deploy/cf-params.deploy.yml created when preparing to work with the AWS Scheduler Lambda function.

Preparing to deploy or update the AWS scheduler Lambda function

Prior to deploying or updating the scheduler Lambda function, you will need to perform the following steps.

1. Perform all steps required to work with the AWS scheduler Lambda function.

2. Identify or create a function execution role that the Lambda service will assume to invoke the function.

3. Create the scheduler Lambda secret with the required key-value pairs.

4. Optionally create an AWS Secrets Manager secret in which to store the New Relic license key used by the New Relic AWS Lambda Extension.

   This secret should have a single key-value pair with the key LicenseKey and your New Relic License Key as the value. Replace the ParameterValue for the NRLicenseKeySecret parameter in the cf-params.deploy.json file with the ARN of the secret.

   NOTE: You may also choose to specify the New Relic License Key as a Lambda environment variable by replacing the ParameterValue for the NRLicenseKey parameter in the cf-params.deploy.json file with your New Relic License Key. However, this is not recommended for security purposes.

Additionally, update the CloudFormation parameters file ./nr-reports-scheduler/deploy/cf-params.deploy.yml created when preparing to work with the AWS scheduler Lambda function. Documentation for each parameter in the CloudFormation parameters file is provided inline in the CloudFormation template as comments. For example, below is the documentation for the ScheduleGroupName parameter.

  #
  # The schedule group name of the schedule group the scheduler will use to
  # create report schedules.
  #
  ScheduleGroupName:
    Type: String
    Description: The schedule group name of the schedule group the scheduler will use to create report schedules.
    Default: ScheduleGroup

The scheduler Lambda secret

The scheduler Lambda uses an AWS Secrets Manager secret to store the sensitive information that it needs to query the New Relic Reports Builder. The secret must include the following key-value pairs.

Key	Value
apiKey	Your New Relic User API key
accountId	The New Relic account ID where schedules will be stored. Multiple accounts IDs may be specified separated by commas.

NOTE:

• As of v4.1.0, customers should use the New Relic Reports Builder app available in the New Relic catalog. Customers preferring to manually deploy their own instance of the app will need to include a key-value pair with the key sourceNerdletId and a value that specifies the Nerdpack ID of their instance.
• The accountId value may contain multiple New Relic account IDs separated by commas. The scheduler Lambda will query for schedules in each account when it polls the associated instance of the New Relic Reports Builder. However, the specified user associated with the New Relic User API key must have access to each account in order for scheduler Lambda to query each account successfully.
• The New Relic Reports AWS Stack will automatically create the scheduler Lambda secret using the values specified in the nr-reports-stack/deploy/cf-params.deploy.json when it is deployed. When deploying the scheduler Lambda separately, the scheduler Lambda secret must be created manually with the above key-value pairs.

Deploying the scheduler Lambda function

The deploy.sh script is used to deploy the CloudFormation stack for the scheduler Lambda function. It supports the following options.

Option	Description	Example
-n stack-name	The name to use for the CloudFormation stack. Defaults to nr-reports-scheduler.	-n nr-reports-scheduler-stack
-t image-name	The image name used to tag the image in your local Docker registry. Defaults to nr-reports-scheduler when the deploy.sh is run directly or nr-reports-scheduler:$package_json_version when run using npm run deploy.	-t nr-reports-scheduler-test:3.2.0
-p prefix	The prefix to use when looking up environment variables. This option should always be specified with the value SCHEDULER when using the deploy.sh script directly.	-p SCHEDULER
--no-build	Do not invoke the build.sh script script before deploying. Defaults to false.	--no-build

The deploy.sh script will first invoke the build.sh script to build and push the scheduler Lambda container image as outlined in the section Building and pushing the scheduler Lambda container image. The build.sh script will be invoked with the --full and --push flags. The -t option specified when the deploy.sh script was invoked will be passed through as well as the -p option.

The deploy.sh script will then use the aws cloudformation deploy command to create the stack using the CloudFormation template and the CloudFormation parameters file ./nr-reports-scheduler/deploy/cf-params.deploy.yml.

You can either run the script directly or use the npm run deploy command while in the ./nr-reports-scheduler directory. For example, to build, push, and deploy the scheduler Lambda function using the NPM script using the default image name and version number and the default stack name, run the following command.

npm run deploy

Upon successful completion, output similar to the following will be displayed.

Deploying stack nr-reports-scheduler...

Waiting for changeset to be created..
Waiting for stack create/update to complete
Successfully created/updated stack - nr-reports-scheduler
Done.

Update the Lambda function

The update.sh script is used to update the scheduler Lambda function to a new version of the container image. It supports the following options.

Option	Description	Example
-n function-name	The name of the function that was specified for the SchedulerFunctionName parameter in the CloudFormation parameters file when the scheduler Lambda function was deployed.	-n NewRelicReportScheduler
-t image-name	The image name used to tag the image in your local Docker registry. Defaults to nr-reports-scheduler when the update.sh is run directly or nr-reports-scheduler:$package_json_version when run using npm run update. Only used when the --build option is specified.	-t nr-reports-scheduler-test:3.2.0
-p prefix	The prefix to use when looking up environment variables. This option should always be specified with the value SCHEDULER when using the update.sh directly.	-p SCHEDULER
--build	Invoke the build.sh script script before updating. Defaults to false.	--build

The update.sh script uses the aws lambda update-function-code command to notify the scheduler Lambda function with the name specified using the -n option that its image has been updated and that it needs to be refreshed. The function-name specified using the -n option should be the name of the function that was specified for the SchedulerFunctionName parameter in the CloudFormation parameters file when the scheduler Lambda function was deployed.

Additionally if the --build option is specified, the update.sh script will first invoke the build.sh script to build and push the scheduler Lambda container image as outlined in the section Building and pushing the scheduler Lambda container image. The build.sh script will be invoked with the --full and --push flags. The -t option specified when the update.sh script was invoked will be passed through as well as the -p option.

You can either run the script directly or use the npm run update command while in the ./nr-reports-scheduler directory. For example, if the name of the function specified in the SchedulerFunctionName parameter when the scheduler Lambda function was deployed was NewRelicReportScheduler, to build, push, and update the scheduler Lambda function using the NPM script using the default image name and version number, run the following command.

npm run update -- -n NewRelicReportScheduler --build

NOTE: As some of the environment variables used by the scheduler Lambda function may contain secrets (license key, user API key, and/or other sensitive data) and since the output from the aws lambda update-function-code command displays the value of all environment variables, no output will be displayed after the UPDATE command header.

NOTE: When updating the scheduler Lambda function, you may need to increment the value of the SchedulerECRImageTag parameter specified in the ./nr-reports-scheduler/deploy/cf-params.deploy.json file.

Delete the Lambda function

The delete.sh script is used to delete the CloudFormation stack for the scheduler Lambda function. It supports the following options.

Option	Description	Example
-n stack-name	The name of the CloudFormation stack. Defaults to nr-reports-scheduler.	-n nr-reports-scheduler-stack

You can either run the script directly or use the npm run delete command while in the ./nr-reports-scheduler directory. For example, to delete the scheduler Lambda function using the NPM script using the default stack name, run the following command.

npm run delete

Upon successful completion, output similar to the following will be displayed.

Deleting stack nr-reports-scheduler...
Waiting for stack delete to complete...
Done.

Using the Java Scheduler

The Java Scheduler is a Java application that monitors reports defined in the New Relic Reports Builder and automatically creates, updates, and deletes Quartz jobs to trigger the CLI to run each report based on the schedule defined for it in the New Relic Reports Builder.

The Java Scheduler can be run as a Docker image built using the provided Dockerfile or as a standalone Java application anywhere that Java and Node.js (required to run the CLI) can be run.

Working with the Java Scheduler

The Java Scheduler is configured via environment variables. The following environment variables must be set regardless of whether the application is run using the Java Scheduler image or as a standalone Java application.

Key	Value
NEW_RELIC_API_KEY	Your New Relic User API key
NEW_RELIC_LICENSE_KEY	Your New Relic License Key
REPORT_ACCOUNT_IDS	The New Relic account ID(s) where schedules will be stored. Multiple accounts IDs may be specified separated by commas.

NOTE: As of v4.1.0, customers should use the New Relic Reports Builder app available in the New Relic catalog. Customers preferring to manually deploy their own instance of the app will need to include an environment variable with the key SOURCE_NERDLET_ID and a value that specifies the Nerdpack ID of their instance.

Optionally, the following environment variables may be set.

Key	Value	Default
LOG_LEVEL	A supported java.util.logging.Level	INFO
SYNC_SCHEDULE	A Quartz CRON expression for the interval at which to synchronize report definitions with the associated instance of the New Relic Reports Builder	0 */5 * * * ?
NEW_RELIC_REGION	A New Relic datacenter region identifier. One of US or EU.	US
NODE_CMD	The command used to run Node	node
REPORTS_HOME	The path to the nr-reports root directory (typically the directory where you cloned the nr-reports repository)	..
MANIFEST_DIR	The path to the directory where downloaded manifest stub files should be stored	conf

In addition, any environment variables that can be used by the reporting engine (such as engine options or channel parameters) should be specified in the environment where the scheduler is launched.

NOTE: If the LOG_LEVEL environment variable is set to FINE, the CLI will be run in verbose mode when it is run by the Java Scheduler application. If it is set to FINEST, the CLI will be run in debug mode when it is run by the Java Scheduler application.

NOTE: The NODE_CMD, REPORTS_HOME, and MANIFEST_DIR environment variables should not be set when running the Java Scheduler image.

Using the Java Scheduler image

A Dockerfile is provided to build a Docker image that can run the Java Scheduler application. The image is built from the Amazon Corretto 21 headless base image for Amazon Linux 2023 The image includes the Node 20 binary distribution for Amazon Linux 2023 from Nodesource.

Building the Java Scheduler image

To build the Java Scheduler image using the provided Dockerfile, open a terminal that supports Bash scripts and execute the following commands, making sure to replace path/to/nr-reports with the path to the directory where you cloned the nr-reports repository.

cd path/to/nr-reports/nr-reports-quartz-scheduler
./gradlew buildDocker

The result is an image that will be tagged with the following tags.

• nr-reports-quartz-scheduler:latest
• nr-reports-quartz-scheduler:$VERSION

$VERSION will be the latest git tag minus the leading v. For example, if the latest git tag is v3.5.0, the image tag will be nr-reports-quartz-scheduler:3.5.0.

Running the Java Scheduler image via the Docker CLI

The following example shows how you can run the Java Scheduler image using the docker CLI. To run the Java Scheduler image using a container service such as ECS, see the appropriate documentation for the container service.

docker run --rm \
    --name nr-reports-quartz-scheduler \
    -e NEW_RELIC_API_KEY='[YOUR_USER_API_KEY]' \
    -e NEW_RELIC_LICENSE_KEY='[YOUR_LICENSE_KEY]' \
    -e REPORT_ACCOUNT_IDS='12345,56789' \
    -e EMAIL_SMTP_SERVER='[YOUR_SMTP_SERVER]' \
    -e EMAIL_SMTP_PORT=[YOUR_SMTP_SERVER_PORT] \
    -e EMAIL_SMTP_SECURE='true' \
    nr-reports-quartz-scheduler

In this example, note how the environment variables meant to be used by the Java Scheduler application and the environment variables meant to be used by the reporting engine are all specified in the same way. This is because the environment used by the Java Scheduler is inherited by the CLI process created to run reports. This is the case whether the Java Scheduler is run using the Java Scheduler image or using the standalone Java Scheduler application.

NOTE: Customers preferring to manually deploy their own instance of the Reports Builder app will need to to specify an additional environment variable with the key SOURCE_NERDLET_ID and a value that specifies the Nerdpack ID of their instance on the docker run command. For example, -e SOURCE_NERDLET_ID=[YOUR_REPORTS_BUILDER_INSTANCE_ID].

Using the standalone Java Scheduler application

The Java Scheduler can also be run as a standalone Java application anywhere that Java and Node.js can be run.

Preparing to run the standalone Java Scheduler application

Prior to running the standalone Java Scheduler application, perform the following steps.

1. Ensure that you have met all prerequisites listed to run the Java Scheduler.
2. Ensure that the java command can be resolved from the command line or that the JAVA_HOME environment variable is set in the environment where the application will be run.
3. Ensure that the required environment variables listed in the section "Working with the Java Scheduler" are set in the environment where the application will be run.
4. Set any optional environment variables listed in the section "Working with the Java Scheduler" as necessary in the environment where the application will be run, or ensure that the defaults are appropriate. In particular, ensure that the command identified by the NODE_CMD environment variable can be resolved from the command line, and that the path identified by the REPORTS_HOME environment variable points to the nr-reports root directory (typically the directory where you cloned the nr-reports repository).
5. Set any environment variables meant to be used by the reporting engine as necessary in the environment where the application will be run.

Running the standalone Java Scheduler application

To run the standalone Java Scheduler application, open a terminal that supports Bash scripts and execute the following commands, making sure to replace path/to/nr-reports with the path to the directory where you cloned the nr-reports repository.

cd path/to/nr-reports/nr-reports-quartz-scheduler
./gradlew run

Using the Reports Builder

The New Relic Reports Builder is a New Relic application available via the New Relic catalog that works with the Java Scheduler or the New Relic Reports Scheduler Lambda to provide a full end-to-end scheduled reporting solution.

The Reports Builder is used to build dashboard and query reports and configure schedules and channels all without the need to manually build and maintain manifest files. The Java Scheduler and the New Relic Reports Scheduler Lambda communicate with the Reports Builder on a scheduled interval to obtain the list of reports and schedules. They use the information obtained to manage the actual scheduling and running of the reports.

More information on the Reports Builder such as how to enable the application and how to use the interface can be found in the New Relic Reports Builder documentation.

Using the New Relic Reports AWS Stack

The New Relic Reports AWS Stack is a CloudFormation Stack that creates all the resources necessary to schedule and run reports defined in an instance of the New Relic Reports Builder. Used together, these two components implement a full end-to-end scheduled reporting solution built on AWS and New Relic.

The stack creates the following resources.

• The scheduler lambda secret
• An AWS EventBridge Schedule Group to contain the schedule to run the AWS Scheduler Lambda function and the schedules created to run reports
• The AWS Lambda function
• A Lamba execution role that allows the AWS Lambda function to read the scheduler Lambda secret and optionally a secret containing your New Relic license key
• The AWS Scheduler Lambda function
• A Lamba execution role that allows the AWS Scheduler Lambda function to read the scheduler Lambda secret and optionally a secret containing your New Relic license key, to list all EventBridge schedules, and to create, read, update, delete EventBridge schedules in the created schedule group.
• An IAM role that allows the AWS EventBridge Scheduler to invoke both the AWS Lambda function and the AWS Scheduler Lambda function.
• An AWS EventBridge Schedule that will invoke the AWS Scheduler Lambda function every 5 minutes to poll the associated instance of the New Relic Reports Builder.

Preparing to work with the New Relic Reports AWS Stack

Prior to working with the New Relic Reports AWS Stack you will need to perform the following steps.

1. Perform all steps required to work with the AWS Lambda function.

2. Build and push the AWS Lambda container image.

3. Perform all steps required to work with the AWS Scheduler Lambda function.

4. Build and push the AWS Scheduler Lambda container image.

5. Enable the New Relic Reports Builder.

6. Copy the file cf-params-sample.json to a file named cf-params.deploy.json in the ./nr-reports-stack/deploy directory and update all values using the documentation provided inline in the the CloudFormation template.

   Make sure to provide values for the UserApiKey and NerdletAccountId parameters. Replace the ParameterValues for these keys with the values described in the section "The scheduler Lambda secret".

   NOTE: As of v4.1.0, customers should use the New Relic Reports Builder app available in the New Relic catalog and should not specify a value for the NerdletPackageId parameter or should leave it empty (the default). Customers preferring to manually deploy their own instance of the app will need to provide a value for the NerdletPackageId parameter that specifies the Nerdpack ID of their instance.

7. Optionally create an AWS Secrets Manager secret in which to store the New Relic license key used by the New Relic AWS Lambda Extension.

   This secret should have a single key-value pair with the key LicenseKey and your New Relic License Key as the value. Replace the ParameterValue for the NRLicenseKeySecret parameter in the cf-params.deploy.json file with the ARN of the secret.

   NOTE: You may also choose to specify the New Relic License Key as a Lambda environment variable by replacing the ParameterValue for the NRLicenseKey parameter in the cf-params.deploy.json file with your New Relic License Key. However, this is not recommended for security purposes.

Deploy the AWS Stack

The deploy.sh script is used to deploy the New Relic Reports AWS CloudFormation stack It supports the following options.

Option	Description	Example
-n stack-name	The name to use for the CloudFormation stack. Defaults to nr-reports-stack.	-n nr-reports-stack-test
-t image-name	Unused when deploying the AWS Stack	N/a
-p prefix	Unused when deploying the AWS Stack	N/a
--no-build	Must always be specified when deploying the AWS Stack.	--no-build

The deploy.sh script will use the aws cloudformation deploy command to create the stack using the CloudFormation template and the CloudFormation parameters file ./nr-reports-stack/deploy/cf-params.deploy.yml. Because the the CloudFormation template creates IAM resources, the special capability CAPABILITY_NAMED_IAM needs to be passed to the aws cloudformation deploy command. This can be done using the AWS_CF_DEPLOY_OPTS environment variable and can be set directly on the command line.

For example, to deploy the stack using the default stack name while in the ./nr-reports-stack directory, run the following command.

AWS_CF_DEPLOY_OPTS="--capabilities CAPABILITY_NAMED_IAM" ../deploy/lambda/deploy.sh --no-build

Upon successful completion, output similar to the following will be displayed.

Deploying stack nr-reports-stack...

Waiting for changeset to be created..
Waiting for stack create/update to complete
Successfully created/updated stack - nr-reports-stack
Done.

NOTE: As mentioned above, the --no-build option must be specified. This tells the the deploy.sh script that there is no code to build and to skip the build and push steps.

Delete the AWS Stack

The delete.sh script is used to delete the CloudFormation stack. It supports the following options.

Option	Description	Example
-n stack-name	The name of the CloudFormation stack. Defaults to nr-reports-stack.	-n nr-reports-stack-test

For example, to delete the CloudFormation stack using the default stack name while in the ./nr-reports-stack directory, run the following command.

../deploy/lambda/delete.sh

Upon successful completion, output similar to the following will be displayed.

Deleting stack nr-reports-stack...
Waiting for stack delete to complete...
Done.

Troubleshooting

Monitoring

The reporting engine can be monitored using the New Relic APM agent for Node.js if you are using the CLI, the CLI image, or the CRON image; or using Serverless monitoring for AWS Lambda if you are using the AWS Lambda function

To enable the APM agent for the CLI, the CLI image, or the CRON image, simply update the Node.js agent configuration just as you would for any other application. If you plan to use the agent configuration file method, you can find the agent configuration file at nr-reports-cli/newrelic.js. If you plan to use environment variables (recommended), refer to the appropriate documentation for setting environment variables for your runtime environment (shell versus local Docker image versus container service versus other environments).

To enable serverless monitoring for AWS Lambda, refer to our official documentation and the Deploying the AWS Lambda function section of this document.

Once enabled, an APM or Lambda function entity will be created with the name specified in the agent configuration and the reporting engine performance metrics, logs, and traces will be collected and associated with the entity.

Support

New Relic has open-sourced this project. This project is provided AS-IS WITHOUT WARRANTY OR DEDICATED SUPPORT. Issues and contributions should be reported to the project here on GitHub.

We encourage you to bring your experiences and questions to the Explorers Hub where our community members collaborate on solutions and new ideas.

Contributing

We encourage your contributions to improve Reports! Keep in mind when you submit your pull request, you'll need to sign the CLA via the click-through using CLA-Assistant. You only have to sign the CLA one time per project. If you have any questions, or to execute our corporate CLA, required if your contribution is on behalf of a company, please drop us an email at [email protected].

A note about vulnerabilities:

As noted in our security policy, New Relic is committed to the privacy and security of our customers and their data. We believe that providing coordinated disclosure by security researchers and engaging with the security community are important means to achieve our security goals.

If you believe you have found a security vulnerability in this project or any of New Relic's products or websites, we welcome and greatly appreciate you reporting it to New Relic through HackerOne.

License

New Relic Reports is licensed under the Apache 2.0 License.

New Relic Reports also uses source code from third-party libraries. You can find full details on which libraries are used and the terms under which they are licensed in the third-party notices document.