Garbage Collection Log Analysis API

Garbage collection log is a vital artifact to study the performance characteristics of an application. It plays a key role in diagnosing memory/CPU related problems. However today’s world Garbage collection logs are analyzed manually. Most often it’s analyzed only when there is a performance problem. There is no programmatic way to analyze Garbage Collection logs in a proactive manner. Thus gceasy.io is introducing a RESTful API to analyze garbage collection logs. Here are couple of use cases where this API can be used.

Use case 1: Production Proactive GC Log analysis

In production environment, where there are several JVMs running, only selective few JVM’s Garbage Collection logs are analyzed. Because of the tediousness involved in analyzing all the Garbage Collection logs not every single JVMs Garbage Collections are analyzed. Most of times, only when there is a performance problems, Garbage Collection logs are analyzed. With Garbage Collection analysis API, you can analyze GC logs in a proactive manner. On a periodic basis you can invoke Garbage Collection analysis API to check whether any problem is brewing in the application. Garbage Collection log analysis API reports any vulnerabilities that is currently happening in the application, besides that its ML algorithms can even forecast certain types of memory problems that’s going to happen in near future.

Use case 2: Continuous Integration

As part of continuous integration after all tests are executed against the source code, Garbage Collection log analysis API can be invoked to see whether application is suffering from any memory problems. If API reports any memory problems, then build can be failed. In this way you can catch the memory problem right during code commit time instead of catching it in performance tests or production.

How to invoke Garbage Collection log analysis API?

Invoking Garbage Collection log analysis is an extremely simple:

  1. Register with us. We will email you the API key. This is a one-time setup process.
  2. Post HTTP request to http://api.gceasy.io/analyzeGC?apiKey={API_KEY_SENT_IN_EMAIL}
  3. Body of the HTTP request should contain the Garbage collection log that needs to be analyzed.
  4. HTTP Response will be sent back in JSON format. JSON has several important stats about the GC log. Primary element to look in the JSON response is: “isProblem“. This element will have value to be “true”, if any memory problems has been discovered. “problem” element will contain detailed description about the memory problem. Sample response is given below.

CURL command

Assuming your GC log file is located in “./my-app-gc.log”, then CURL command to invoke the API is:

curl -X POST --data-binary @./my-app-gc.log http://api.gceasy.io/analyzeGC?apiKey= --header "Content-Type:text"

It can’t get any more simpler than that? Isn’t it? 🙂

Sample Response

{
  "responseId": "03d4eab6-2070-4081-a1dc-273e213c5175",
  "graphURL": "http://api.gceasy.io/my-gc-report.jsp?p=L2hvbWUvcmFtL3VwbG9hZC9HQ0VBU1kxL3NoYXJlZC8yMDE2LTYtMTgvYXBpLWQwOTRhMzRlLWJkMjItNGMyOC05Njk4LTgxOWUzZDk1MWVhN2JmMjI5NmJhLTk0MDAtNDk1ZC04ZWI1LWM4ZWRkMTU3M2Q3Mi50eHQ=",
  "isProblem": true,
  "problem": [
    "Our analysis tells that your application is suffering from memory leak. It can cause OutOfMemoryError, JVM to freeze, poor response time and high CPU consumption."
  ],
  "jvmHeapSize": {
    "youngGen": {
      "allocatedSize": "1.3 gb",
      "peakSize": "1.3 gb"
    },
    "oldGen": {
      "allocatedSize": "2.67 gb",
      "peakSize": "2.67 gb"
    },
    "permGen": {
      "allocatedSize": "285 mb",
      "peakSize": "214.54 mb"
    },
    "total": {
      "allocatedSize": "3.87 gb",
      "peakSize": "3.64 gb"
    }
  },
  "gcStatistics": {
    "totalCreatedBytes": "6.07 tb",
    "measurementDuration": "29 hrs 15 min 28 sec",
    "avgAllocationRate": "60.38 mb/sec",
    "avgPromotionRate": "284 kb/sec",
    "minorGCCount": "10566",
    "minorGCTotalTime": "3 min 40 sec 168 ms",
    "minorGCAvgTime": "21 ms",
    "minorGCAvgTimeStdDeviation": "6 ms",
    "minorGCMinTIme": "10 ms",
    "minorGCMaxTime": "90 ms",
    "minorGCIntervalAvgTime": "9 sec 755 ms",
    "fullGCCount": "1071",
    "fullGCTotalTime": "37 min 15 sec 829 ms",
    "fullGCAvgTime": "2 sec 88 ms",
    "fullGCAvgTimeStdDeviation": "793 ms",
    "fullGCMinTIme": "100 ms",
    "fullGCMaxTime": "4 sec 920 ms",
    "fullGCIntervalAvgTime": "1 min 38 sec 431 ms"
  },
  "gcDurationSummary": {
    "groups": [
      {
        "start": "0",
        "end": "1",
        "numberOfGCs": 10622
      },
      {
        "start": "1",
        "end": "2",
        "numberOfGCs": 316
      },
      {
        "start": "2",
        "end": "3",
        "numberOfGCs": 565
      },
      {
        "start": "3",
        "end": "4",
        "numberOfGCs": 128
      },
      {
        "start": "4",
        "end": "5",
        "numberOfGCs": 6
      }
    ]
  }
}

JSON Response Elements

Element Description
responseId Unique transaction Id that is generated for every response. This is used for any debugging or diagnosis purposes
graphURL Graphical visualization of the GC log can be found in this location.
isProblem true’ is returned if any memory problems are found. ‘false’ is returned if no memory problems are found. This element can be used to build any alerts for proactive GC monitoring
problem Description of the memory problem is reported in this element. Like what type of memory problem it is. What are the side-effects and symptoms it might cause
jvmHeapSize The data points in this section is gathered from the GC log, thus It may or may not match with the size that is specified by the JVM system properties (i.e. –Xmx, -Xms,…). Say you have configured total heap size (i.e. –Xmx) as 2gb, whereas at runtime if JVM has allocated only 1gb, then in this report you are going to see the allocated size as 1gb only.
youngGen
allocatedSize Young Generation’s allocated size (i.e. specified at the JVM level)
peakSize Young Generation’s peak utilization size at runtime
oldGen
allocatedSize Old Generation’s allocated size (i.e. specified at the JVM level)
peakSize Old Generation’s peak utilization size at runtime
metaSpace
allocatedSize Metaspace’s allocated size (i.e. specified at the JVM level)
peakSize MetaSpace’s peak utilization size at runtime
permGen
allocatedSize Perm Generation’s allocated size (i.e. specified at the JVM level)
peakSize Perm Generation’s peak utilization size at runtime
total
allocatedSize Total allocated heap size (i.e. specified at the JVM level) includes Young + Old + Perm (or Metaspace)
peakSize Peak utilizationof the heap size at runtime
gcStatistics GC statistics summary is based on ‘real’ time reported in the GC Logs. For more details on differen between user, sys, real time refer to https://blog.tier1app.com/2016/04/06/gc-logging-user-sys-real-which-time-to-use/
totalCreatedBytes Total amount of objects created by the application
measurementDuration Total time duration application has been running
avgAllocationRate Objects creation rate by the application. When more objects are allocated, young generation gets filled up quickly and Minor GC runs more frequently.
avgPromotionRate Objects promoted rate from Young Generation to Old Generation. When more objects are promoted to Old generation, Full GC will be run more frequently.
minorGCCount Number of Minor GCs occurred in the application
minorGCTotalTime Total time spent in Minor GCs
minorGCAvgTime Average time taken by the Minor GCs (i.e. total time taken by all minor GCs / number of minor GCs)
minorGCAvgTimeStdDeviation Standard Deviation of the Minor GC average times
minorGCMinTIme Minimum Time of all the minor GCs
minorGCMaxTime Maximum time of all the minor GCs
minorGCIntervalAvgTime Average time interval between minor GCs
fullGCCount Number of Full GCs occurred in the application
fullGCTotalTime Total time spent in Full GCs
fullGCAvgTime Average time taken by the Full GCs (i.e. total time taken by all full GCs / number of full GCs)
fullGCAvgTimeStdDeviation Standard Deviation of the Full GC average times
fullGCMinTime Minimum Time of all the full GCs
fullGCMaxTime Maximum time of all the full GCs
fullGCIntervalAvgTime Average time interval between Full GCs
gcDurationSummary
groups Array of GC Duration summary group. Example of a group:
{
“start”: “0.2”,
“end”: “0.3”,
“numberOfGCs”: 5
},Indicates that 5 GCs completed between 0.2 second and 0.3 seconds
start Group’s start time reported in seconds
end Group’s end time reported in seconds
numberOfGCs Number of GCs that completed between ‘start’ time and ‘end’ time
gcCauses array  reporting the GC Causes
cause The reason which triggerred the GC. Values could be: Concurrent Mode Failure, Allocation Failure, System.gc() calls
count Number of time GC occurred because of this reason
commandLineFlags JVM arguments that were passed to the application
Advertisements

2 thoughts on “Garbage Collection Log Analysis API

Add yours

  1. Periactin drug no prescription usa
    buying Periactin without a script usa
    buy Periactin no prescription cod
    Periactin usa discount fedex no prescription

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

Up ↑

%d bloggers like this: