Thread Dump Analysis Pattern – THREAD MILL

Description

You might have experienced the application’s CPU to spike up suddenly & spike wouldn’t go down until JVM is recycled. You restart the JVM, after certain time period CPU consumption would once again start to spike up. Then you will have to recycle the JVM once again. Have you experienced it? If you have smile on your face now, then it’s certain you would have experienced this problem.

This type of problem typically happens when thread spins on an infinite loop. A thread would be spinning infinitely when one of the issue described in this article happens.

To diagnose these sort of problems, you would have to capture 3 thread dumps in an interval of 10 seconds. In between those thread dumps, if there are threads

a. on the same method (or one the same line of code)

b. they are in ‘RUNNABLE’ state,

then those are the threads which is causing CPU to spike up. Investigating the stack trace of those threads, will tell the exact method (or line of code), where threads are spinning. Fixing that particular method (or line of code) would resolve the problem.

Example

HashMap isn’t a thread safe implementation. When multiple threads tries to access HashMap’s get() and put() APIs concurrently it would cause threads go in to infinite looping. This problem doesn’t happen frequently, but it does happens.

Below is an excerpt from a thread dump which indicates the infinite looping that is happening in HashMap:

"Thread-0";quot; prio=6 tid=0x000000000b583000 nid=0x10adc runnable [0x000000000cb6f000]
   java.lang.Thread.State: RUNNABLE
    at java.util.HashMap.put(HashMap.java:374)
    at com.tier1app.HashMapLooper$AddForeverThread.AddForever(NonTerminatingLooper.java:32)
    at com.tier1app.HashMapLooper$AddForeverThread.method2(NonTerminatingLooper.java:27)
    at com.tier1app.HashMapLooper$AddForeverThread.method1(NonTerminatingLooper.java:22)
    at com.tier1app.NonTerminatingLooper$LoopForeverThread.run(NonTerminatingLooper.java:16)

Across all the 3 thread dumps “Thread-0” was always exhibiting same stack trace. i.e. it was always in the java.util.HashMap.put(HashMap.java:374) method. This problem was addressed by replace the HashMap with ConcurrentHashMap.

Why named as Thread mill?

In Thread Mill, one would keep running without moving forward. Similarly when there is infinite looping, CPU consumption goes high with out progress in the code execution path. Thus it’s named as ‘Thread mill’ pattern.

Advertisements

One thought on “Thread Dump Analysis Pattern – THREAD MILL

Add yours

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

Up ↑

%d bloggers like this: