THREAD DUMP ANALYSIS PATTERN – LEPRECHAUN TRAP

Description

Unlike other objects, GC process will not immediately garbage collect the objects which has finalize method implemented on them, even though there aren’t any active reference to it. Instead as first step, those objects are added to an internal queue of java.lang.ref.Finalizer object. There is a low priority Finalizer thread which executes finalize() method of each object in the queue. Only after the execution of finalize() method, object becomes eligible for Garbage Collection. In case if Finalizer thread ever gets blocked when executing finalize() method, it will have a severe cascading effect on the JVM.

Once Finalizer thread gets blocked internal queue of java.lang.ref.Finalize object will start to grow. This in turn would cause JVM’s memory consumption to go high. This in turn would result in OutOfMemoryError, jeopardizing JVM’s availability.

Thus when analyzing thread dumps it’s highly recommended to study the stack trace of Finalizer thread. In case if they are blocked, then fixes has to be instrumented to clear the block right away.

Example

Here is a sample stack trace of a Finalizer thread which got blocked

"Finalizer" daemon prio=10 tid=0x00007fb2dc32b000 nid=0x7a21 waiting for monitor entry [0x00007fb2cdcb6000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at net.sourceforge.jtds.jdbc.JtdsConnection.releaseTds(JtdsConnection.java:2024)
	- waiting to lock 0x00000007d50d98f0 (a net.sourceforge.jtds.jdbc.JtdsConnection)
	at net.sourceforge.jtds.jdbc.JtdsStatement.close(JtdsStatement.java:972)
	at net.sourceforge.jtds.jdbc.JtdsStatement.finalize(JtdsStatement.java:219)
	at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
	at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:101)
	at java.lang.ref.Finalizer.access$100(Finalizer.java:32)
	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:178)

Above stack trace was captures from a JVM which was using one of the older versions of JTDS JDBC Driver. In this version of driver, you can see JtdsStatement#finalize() method, which eventually calls JtdsConnection#releaseTds() method. Apparently this method got blocked and never returned back. Thus Finalizer thread got stuck indefinitely, causing OutOfMemoryError in the application. Upgrading the JTDS JDBC Driver version to the latest version, fixed the problem.

Why named as Leprechaun Trap?

Kids in western countries build Leprechaun Trap as part of St. Patrick ’s Day celebration. Leprechaun is a fairy character, basically a very tiny old man, wearing a green coat & hat as shown in the above picture. Kids lure him with gold coins to get him trapped. You can learn more about Leprechaun traps here. If you have Memory Leaks in your application, *you may* visit this Leprechaun traps link  mentioned in the previous sentence. On the other hand, if you have kinder garden going kid(s), *you should* visit Leprechaun traps page, as you will end-up building this trap as part of their homework 🙂