New Relic : Monitoring your Go database with New Relic

Object-relational mapping (ORM) tools provide a convenient abstraction layer between your database and your programming language. You use them to represent database transactions as code, rather than as explicit queries, significantly improving the readability and footprint of your code. In the Go community, one of the most popular ORM tools is GORM, due to its developer-friendly design and broad set of features.

Using GORM may help you develop your application's database layer more easily, but how will you know if it's meeting your performance expectations? This is where the New Relic Go agent comes into the picture. The Go agent is an SDK that enables you to instrument and observe a Go application in New Relic.

This post will show you how to use the SDK to manually instrument a Go application with a quick start example from GORM Guides and how to use New Relic One to monitor the cost of your database transactions and the runtime of your code. If you aren't a New Relic user yet, you can sign up for free. A free account includes 100 GB/month of free data ingest, one free full-access user, and unlimited free basic users. After you create your account, you can follow along with this example.

Here is a fully-instrumented version of the GORM quick start. This example uses SQLite, but this post will also discuss using PostgreSQL and MySQL.

Configuring New Relic's Go agent

Let's walk through an opinionated code example, and explain the decisions you make at each step. Note that the Go agent has several configuration options that you can set when it's initialized, which we don't cover here. For details, see the Go agent documentation.

The agent is initialized by calling the function. Within that function, you can call a number of config functions to set various options for how the Go agent will run and operate.

1. First, give the application a name. This name will be used to identify your application's data and dashboards in New Relic, so it is important that you make it something distinct and clear.

2. The Go agent requires a license key to operate. Without one, it will return an error. To keep things simple, this example fetches the key from an environment variable named .

3. The Go agent produces its own logs to keep track of events like data harvest cycles and communication with the New Relic backend. For your convenience, it also tells you the exact web address where your application data is being hosted in New Relic. Since this is a simple one-shot application intended as a code example, the example just logs to stdout and doesn't do anything further to handle logs.

Other configuration options are detailed in the Go agent logging documentation.

4. Finally, turn on the distributed tracing feature. Distributed tracing is a feature that allows agents to trace service requests across distributed systems, then build out a visualization of those requests in your New Relic dashboard. Enabling distributed tracing is not required to trace an application like this, but it is especially helpful for getting more granular information about the execution flow of a process across a distributed system. Look for more on this later in this blog post.

Because this one-shot application executes so rapidly, there is a chance that the app will finish running before the agent can send the data it collected to New Relic. The following lines of code ensure that all data collected gets sent to New Relic before the application exits. This code would not be necessary for monitoring a production application with a lot of data.

Using New Relic instrumented database drivers with GORM

The New Relic Go agent includes several instrumented database drivers for some of the most popular database providers. In a few easy steps, you can import these drivers from the Go agent's integrations library and use them in place of GORM's default database drivers.

1. First, import a New Relic database driver from the Go agent's integrations library. By importing it with a preceding underscore, the driver is implicitly registered as the database driver that will be used to communicate with SQLite.

2. For most databases, you need to create a custom GORM dialector object to configure GORM to use the New Relic database driver. The dialector object is passed to the function to configure how GORM opens a database connection. In the dialector object, the must be the name of the New Relic database driver you are using, and the must be the name or address of the database you are connecting to.

If you are using GORM with MySQL or PostgreSQL, you do not have to create a dialector object. Instead, the function will implicitly use the database driver you import, and you specify how to use it in a object that is specific to that database. The next example is for MySQL, but the code would be exactly the same for PostgreSQL.

Instrumenting the Go application

After your application is configured to use the Go agent, the last step is to start instrumenting your code.

1. Because the Go agent is an SDK, you need to explicitly start and stop tracing in your application. The simplest way to do this is to start a transaction trace.

When a transaction starts, the collection of trace data immediately begins. A transaction's name identifies it in New Relic.

2. After a transaction starts, you need to explicitly create a context from that transaction. In Go, contexts convey deadlines, cancelation signals, and other request-scoped values across APIs and processes. A background context is an empty but non-nil context that communicates no timeouts or cancellation policies, and for this reason, it's often used by default. A context object is generated in the line. This is a background context with additional metadata associating it with the transaction trace. This context is used to keep track of which service calls belong to this transaction trace, which is why a new GORM database object is generated from the context in the line . This is what that looks like with an example of some hard-coded database operations.

You need to repeat these steps for each distinct transaction that you want to trace in your code. More nuanced and advanced ways to trace and instrument your code are covered in the Instrument Go transactions documentation. If you are looking for a more in-depth walkthrough of some of those concepts, we'll be releasing a Go Agent Masterclass soon.

Collecting and analyzing the data

To run the application, you need to ensure you have the necessary go modules installed and then run as shown in the next snippet.

The New Relic Go agent automatically connects to New Relic, captures your transaction data, then sends it to New Relic. The application will output a URL in the application logs on your command line that takes you directly to your application data in New Relic. You can see the data from your application in this web UI.

The summary page is fairly sparse for this example application because it only recorded one very short transaction, but if you scroll down you can see some summary statistics about the trace. Statistics like response time and throughput are helpful for getting a high level view of the average performance of your application's transactions, and statistics like CPU utilization and memory usage are helpful for monitoring hardware performance. In the previous example, you can see that the average response time was 7.84 milliseconds. In order to figure out what factors led to that average response time, select Databases from the left hand pane.

The Databases view visualizes your database transactions. In this example, the majority of database transactions were SQLite update operations, and on average, updates took 0.341 milliseconds. By comparison, selects took 0.108 milliseconds and inserts took 0.155 milliseconds. Based on this data, it's clear that update queries are close to three times more expensive than insert and select operations. This is valuable information, but it does not show us what these average runtime costs mean within the flow of our application. This is where distributed tracing comes into the picture.

Distributed tracing helps put your data in context by breaking down the runtime cost of each subsection of a transaction, known as a span, and displaying it in the order it was executed. This builds a holistic view of a transaction that shows when a segment occurred in execution order, what occurred in that span, and the cost of the events that occurred during that segment. The transaction trace from our application contains spans for one insert, two selects, and four updates that are executed sequentially. The runtime cost of these spans adds up to 7.84 milliseconds, which was the total runtime of our transaction. Spans tracing update operations consumed an outsized portion of that runtime in comparison to the other spans captured both because they took longer to execute and because there are more of them.

Next steps

Interested in monitoring your Go application? Install the New Relic Go agent. And if you haven't already, sign up for New Relic. Your free account includes 100 GB/month of free data ingest, one full-platform user, and unlimited free basic users.