Troubleshooting/FAQ (Frequently Asked Questions)
#
What are the minimum system requirements for running pyroscope? (RAM, Disk)- RAM (2GB ~ 4GB). See My pyroscope server runs out of memory for more details.
- Disk 10GB
#
My pyroscope server runs out of memoryIt's difficult to accurately estimate memory consumption in advance, as it is highly dependent on the workload - mainly the size of profiles and the ingestion rate.
It's fair to estimate resource consumption per agent/target based on the actual memory consumption under the actual load. As a pessimistic estimation we suggest 32 MB per agent/target.
Also keep in mind pyroscope reserves some memory at startup (~400 MB), and also makes heavy use of various internal caches that grow over time and affect overall memory consumption. There are 2 parameters to manage this:
cache-evict-threshold
: percentage of consumed memory at which cache evictions start. The parameter relies on the container limits.cache-evict-volume
: percentage of cache that is evicted per eviction run.
#
My pyroscope server is running out of spaceIn these cases, you can set up a more aggressive retention policy. See the Data retention page for more instructions
#
My pyroscope server is crashing due to out of spaceAlthough pyroscope tries to leave enough space, it may be possible that it gets stuck in a crashing state, where it can't even start due to lack of space.
There are 2 options: increase the disk and free some data.
#
Free some datawarning
We don't guarantee pyroscope will be able to deal with this correctly. Only use this as a last resort. Preferrably do a backup first.
It depends on where you are running, for kubernetes you can overwrite the Deployment's args
to run a dummy command (like sleep infinity
).
Then, you can the oldest .vlog
files in the trees
directory.