📅 September 16, 2024

⏱️ 2 min read (329 words)

So… we had a memory leak.

We are running a NextJS dockerised container in App Service. This is a new application I have been working on, and it was getting ready to be deployed into production. During our load testing of the application we noticed a strange pattern in the metrics. RAM was climbing up and up until eventually the container would die and restart.

NodeJS v22.8.0 Next.js 14.2.9

We are using the below package in our code to facilitate App Insights telemetry.

https://github.com/microsoft/ApplicationInsights-node.js

We were seeing this appearing in our logs everytime the graph dropped off.

After a lot of investigation trying combinations of upgrading NextJS, NodeJS, stripping literally everything out of our application and even deploying the sample app using create-next-app to see if we could highlight where the issue was.

Lots of troubleshooting until we finally decided to turn off Application Insights. The problem with this was during our troubleshooting were using the Live Metrics in Application Insights to show us the RAM usage. We began using the App Service Plan metrics built into the portal and removed Application Insights packages from the code.

It worked… We saw stable memory usage even under load in the App Service Plan metrics. We left the application over night to ensure it continued for a prolonged amount of time.

The next morning we confirmed all was still well with the application. We suspected the Live Metrics was an issue so we decided to try adding Application Insights back in but to disable the Live Metrics functionality. We released another version and monitored for a few hours again. We saw a stable memory usage. This confirmed it was the Live Metrics causing some kind of memory leak, or high memory usage over time.

We are yet to figure out why this is happening, but for now we have a stable application.

This is a summarised version of events, but maybe enough detail to help someone out investigating a similar issue!