In certain circumstances, it may be noticed that the memory usage of the "java" process exceeds the defined Neo4j resources.
The total memory used by the Neo4j process can be found in detail by collecting an NMT file + summing that to the page-cache allocation.

For example, let's say on a Linux server, the "java" process is using 100Gb of memory from an OS perspective (from the `top` command), while the Neo4j configuration is as follows:
- Heap: 31 Gb
- Page Cache: 50 Gb
- Direct memory: 2 Gb

Collecting an NMT, we see that the actual allocation (excluding page cache but including direct memory + JVM internals) is at 37 Gb.
This means that, with page-cache, we are actually using 87 Gb, not 100 Gb, there's a 13 Gb discrepancy that can cause unexpected OS oom-killer events.

This discrepancy it's usually caused by a JVM phenomenon called heap fragmentation, which can cause the JVM to not de-allocate memory properly, which means the OS believes the process is using more memory than expected.

In order to test whether this is actually happening, we can execute a JVM terminal command to defragment the memory (requires JDK 11.0.18+ or 17.0.2+):

jcmd <PID> System.trim_native_heap

The output of the command will be as follows:

~/ > jcmd 2132879 System.trim_native_heap 
Attempting trim... Done. 
Virtual size before: 25034596k, after: 25034472k, (-124k) 
RSS before: 394468k, after: 338444k, (-56024k) 
Swap before: 0k, after: 0k, (0k)

If we observe a significant change in the RSS value (higher than 1/2Gb), it means the process was suffering from heap fragmentation.

A pragmatic approach to mitigate this would be to create a cron job to automatically execute this command every 24 hours (as an example) to ensure the process always runs in a healthy state.

If this is not possible or we want a definitive fix, we can execute the Neo4j process with a different malloc allocator, for example, JEmalloc (http://jemalloc.net/).
This allocator is generally more efficient than the one we get on the OS out of the box, and it also enables some advanced telemetry stats that can be used to track memory leaks and allocation details.
Lastly, this allocator is actively in use on the Aura platform.

Reference:

https://bugs.openjdk.org/browse/JDK-8268893