Go
Pyroscope uses the standard runtime/pprof
package to collect profiling data. Refer to
the official documentation for details.
#
Supported platformsSpy Name | Type | Linux | macOS | Windows | Docker |
---|---|---|---|---|---|
n/a | standalone | ✅ | ✅ | ✅ | ✅ |
#
Profiling Go applicationTo start profiling a Go application, you need to include our go module in your app:
# make sure you also upgrade pyroscope server to version 0.3.1 or highergo get github.com/pyroscope-io/client/pyroscope
Then add the following code to your application:
package main
import "github.com/pyroscope-io/client/pyroscope"
func main() { // These 2 lines are only required if you're using mutex or block profiling // Read the explanation below for how to set these rates: runtime.SetMutexProfileFraction(5) runtime.SetBlockProfileRate(5)
pyroscope.Start(pyroscope.Config{ ApplicationName: "simple.golang.app",
// replace this with the address of pyroscope server ServerAddress: "http://pyroscope-server:4040",
// you can disable logging by setting this to nil Logger: pyroscope.StandardLogger,
// optionally, if authentication is enabled, specify the API key: // AuthToken: os.Getenv("PYROSCOPE_AUTH_TOKEN"), // you can provide static tags via a map: Tags: map[string]string{"hostname": os.Getenv("HOSTNAME")},
ProfileTypes: []pyroscope.ProfileType{ // these profile types are enabled by default: pyroscope.ProfileCPU, pyroscope.ProfileAllocObjects, pyroscope.ProfileAllocSpace, pyroscope.ProfileInuseObjects, pyroscope.ProfileInuseSpace,
// these profile types are optional: pyroscope.ProfileGoroutines, pyroscope.ProfileMutexCount, pyroscope.ProfileMutexDuration, pyroscope.ProfileBlockCount, pyroscope.ProfileBlockDuration, }, })
// your code goes here}
important
If you want to use mutex or block profiling you have to explicitly enable it in your application. See Mutex Profiling section for more information.
important
Before golang 1.19 version, goroutine profile will lead to a long time STW proportional to the number of goroutines, see here for more details.
#
TagsIt is possible to add tags (labels) to the profiling data. These tags can be used to filter the data in the UI. We have a custom API that's in line with our other integrations (e.g Python or Ruby) as well as go-native pprof api:
// these two ways of adding tags are equivalent:pyroscope.TagWrapper(context.Background(), pyroscope.Labels("controller", "slow_controller"), func(c context.Context) { slowCode()})
pprof.Do(context.Background(), pprof.Labels("controller", "slow_controller"), func(c context.Context) { slowCode()})
#
Pull ModeGo integration supports pull mode, which means that you can profile applications without adding any extra code. For that to work you will need to make sure you have profiling routes (/debug/pprof
) enabled in your http server. Generally, that means that you need to add net/http/pprof
package:
import _ "net/http/pprof"
- For information on how to use
net/http/pprof
refer to official net/http/pprof documentation. - For more information on how to set up pyroscope server to pull data from your applications, refer to pull mode documentation.
#
godeltaprofgodeltaprof is a memory profiler for cumulative profiles(heap, block, mutex). It is more efficient because it does the delta/merging before producing pprof data, avoiding extra decompression/parsing/allocations/compression.
To start using godeltaprof in pull mode in a Go application, you need to include godeltaprof module in your app:
go get github.com/pyroscope-io/godeltaprof@latest
Integration is very simillar to net/http/pprof
, you need to import a new package and it will expose new endpoints /debug/pprof/delta_heap
, /debug/pprof/delta_block
, /debug/pprof/delta_mutex
_ "github.com/pyroscope-io/godeltaprof/http/pprof"
In the scrape config you need to enable new delta endpoints with use-delta-endpoints: true
, for example:
scrape-configs: - job-name: pyroscope1 enabled-profiles: [cpu, mem, block, mutex] use-delta-profiles: true
For push mode godeltaprof is used automatically since v0.6.0
#
Tracing IntegrationPyroscope can integrate with distributed tracing systems supporting OpenTelemetry standard which allows you to link traces with the profiling data, and find specific lines of code related to a performance issue. For more information, refer to tracing integration documentation.
#
Mutex Profilingnote
Mutex profiling requires pyroscope v0.20.0 or higher and client integration v.0.3.0 or higher.
Mutex profiling is useful for finding sources of contention within your application. It helps you to find out which mutexes are being held by which goroutines.
To enable mutex profiling, you need to add the following code to your application:
runtime.SetMutexProfileFraction(rate)
rate
parameter controls the fraction of mutex contention events that are reported in the mutex profile. On average 1/rate events are reported.
#
Block Profilingnote
Block profiling requires pyroscope v0.20.0 or higher and client integration v.0.3.0 or higher.
Block profiling lets you analyze how much time your program spends waiting on the blocking operations such as:
- select
- channel send/receive
- semacquire
- notifyListWait
To enable block profiling, you need to add the following code to your application:
runtime.SetBlockProfileRate(rate)
rate
parameter controls the fraction of goroutine blocking events that are reported in the blocking profile. The profiler aims to sample an average of one blocking event per rate nanoseconds spent blocked.
#
Sending data to Phlare with Pyroscope Golang integrationStarting with weekly-f8 you can ingest pyroscope profiles directly to phlare.
pyroscope.Start(pyroscope.Config{ ApplicationName: "phlare.golang.app", ServerAddress: "<URL>", // Optional HTTP Basic authentication BasicAuthUser: "<User>", BasicAuthPassword: "<Password>", // Optional Phlare tenant ID TenantID: "<TenantID>", ProfileTypes: []pyroscope.ProfileType{ pyroscope.ProfileCPU, pyroscope.ProfileInuseObjects, pyroscope.ProfileAllocObjects, pyroscope.ProfileInuseSpace, pyroscope.ProfileAllocSpace, },})
To configure the Golang integration to send data to Phlare, replace the <URL>
placeholder with the appropriate server URL. This could be the grafana.com Phlare URL or your own custom Phlare server URL.
If you need to send data to grafana.com, you'll have to configure HTTP Basic authentication. Replace <User>
with your grafana.com stack user and <Password>
with your grafana.com API key.
If your Phlare server has multi-tenancy enabled, you'll need to configure a tenant ID. Replace <TenantID>
with your Phlare tenant ID.
#
Golang profiling examplesCheck out the following resources to learn more about Golang profiling:
- Golang examples
- Golang Demo showing golang example with tags
- Golang blog post