Javacodegeeks Netdna Cdn Content Uploads 2013 Jvm Troubleshooting Guide

This post includes tips and recommendations for tuning and troubleshooting performance on WebSphere Application Server based products. IBM WebSphere Application Server (WAS) is a market-leading Application Server for running Java applications. Information technology is the flagship product within IBM'due south WebSphere production line and it is the base of operations runtime product for several IBM products. These products range from Mobile servers (IBM Worklight Server), Integration servers (WebSphere Enterprise Service Bus), Operational Conclusion Direction servers (WebSphere ILOG JRules), to Business organization Process Management servers (IBM BPM Advanced), among many others. Considering these products share the same runtime, the tuning tips and recommendations in this post apply to all of them.

When to Tune for Performance?

Performance tuning needs to occur early in your projection. There should be time allocated for load and stress testing. Load testing involves testing the normal application load in terms of expected concurrent request and type of requests. Stress testing involves testing beyond the expected load level (e.g. 125% to 150% of expected load) and as well includes testing over extended periods of fourth dimension (e.g. 4 to eight hours). During the duration of these tests the goal is to monitor and measure out how the application performance and server vitals behave. On a normal load test at that place should non exist long sustained periods of loftier CPU usage. What constitutes an adequate high usage is relative. A conservative CPU usage is typically under l%, where as an aggressive CPU usage may exist equally high as 80%, but it boils down to the criticality of the application performance and the risk that can be incurred.

Java Virtual Motorcar (JVM) Tuning

During load tests and specifically for Coffee applications, it is extremely important to monitor for Java heap size usage. Every WebSphere Application Server instance runs in its own JVM. Default JVM settings are normally good plenty for small volume applications. JVM settings will likely need to exist tuned to support a combination of the following: large number of applications deployed, high book of transactions to exist handled meantime and/or large size requests. In that location are two main areas to watch for when it comes to JVM heap size: how quickly the heap size grows and how long it takes to perform a garbage collection.

Tune JVM Minimum and Maximum heap size

As the number of deployed applications grows, the utilize of heap size volition increase and may exceed the maximum heap size. This can lead to potential issues: garbage collections will occur more than frequently and will take longer time; our out of retentiveness errors can occur if at that place is not enough retentiveness to classify in the heap. The heap size growth is besides impacted by the expected number of concurrent requests and past the number or size of objects allocated throughout processing. Before increasing maximum heap size, information technology is important to measure whether the increase is needed due to legitimate application growth, or acquired by potential memory leaks. If it is due to legitimate growth, the maximum heap size should exist incremented non to exceed 50% of overall physical memory on the server. This measure may vary depending on what other processes are running on the server (e.k. other JVMs) and how much retention the Os allocates for system processes. They main goal is to avoid paging to disk as much as possible. Paging memory to disk translates into larger garbage collection times and consequently slower application response times.

The default initial heap size for WAS is 50MB and default maximum size is 256MB. In about cases the initial heap size should be set up lower than the maximum heap size, however in cases where optimal performance is a priority specifying the same value for the initial and maximum heap size is recommended. The JVM heap size settings can be inverse from the administrative console: Servers > Server Types > WebSphere awarding servers > server_name > Coffee and process direction > Process definition > Java Virtual machine.

Review Garbage drove policy

The IBM JVM in WAS supports four garbage drove policies. Starting with version 8.0, gencon is the default policy. From personal experience, gencon is the policy that yields the all-time throughput and overall smaller collection pause times. Of grade, this may vary depending on the specifics needs of your application, just I normally recommend using gencon as a starting point.

Beyond JVM Tuning

JVM tuning is but one surface area of the tuning that needs to be done in WAS. Depending on the nature of the application, hither are other settings that may need tuning.

Monitor and tune Thread puddle sizes

Thread pool settings can be inverse from the assistants console at: Servers > Server Types > WebSphere awarding servers > server_name > Thread Pools. A thread pool maximum size can be increased to ameliorate concurrent processing. Depending on the nature of the application, different thread pools are more relevant than others. For instance, Web Container thread pools are more relevant for web applications. The Default thread pool is another relevant thread pool that is used past virtually applications. The actual number of threads allocated to each thread pool should be monitored, to ostend that in that location is a legitimate need to increase its size.

Monitor and tune JDBC Connection pool sizes

If your awarding connects to a JDBC data source, connection pool sizes come into play. These can exist changed from the administration panel: Resources > JDBC > Data sources > data_source_name > Connection pool properties.

Monitoring Tools

Tivoli Performance Viewer

Ideally your organization should use a robust monitoring solution to monitor for server and JVM wellness indicators and proactively alert when certain thresholds are reached. If your arrangement does not provide such tools, developers can use the Tivoli Performance Viewer included in WAS. The Functioning Viewer allows monitoring for CPU usage, Coffee heap size usage, thread pool sizes, JDBC connexion puddle sizes, amongst many other indicators. The Performance Viewer is accessible from the assistants console at: Monitoring and tuning > Performance Viewer > Electric current activity > server_name. You can and then expand the different sections of interest and check on the indicators to be monitored. In the screenshot below we are monitoring for Heap Size, Procedure CPU Usage, Web Container's thread pool size and WPSDB JDBC data source's pool size.

WAS Performance Management Tuning Toolkit

The Performance Viewer can be helpful to monitor for a small set of indicators within a single server (e.1000. during development). Nonetheless when you lot need to monitor several indicators across multiple servers on a cluster the Performance Viewer is hard to navigate. A better tool for monitoring multiple servers at the same time is the "WAS Functioning Management Tuning Toolkit".

This is a very useful tool that connects to the deployment manager of a cluster. Once connected you accept access to all of the performance indicators available on the Functioning Viewer. It is much easier to navigate and switch back and forth different servers and indicators.

Troubleshooting Application Performance Problems

Hither are a few tips and artifacts that tin can be used for troubleshooting application performance problems.

Enable verbose GC to identify frequency and time spent during garbage collection

The verbose output of the Garbage Collector can be used to analyze problems. To enable verbose GC output login to the Administrative console and navigate to: Servers > Server Types > WebSphere application servers > server_name > Coffee and process management and cheque "Verbose garbage collection". Verbose GC output will then be captured on the native_stderr.log file on the server logs. Verbose GC output can be analyzed with the "IBM Pattern Modeling and Analysis Tool for Java Garbage Collector".

This tool can provide useful information such as whether the Java Heap size was exhausted, number of garbage collections, break fourth dimension used for garbage collections. The tool also recommends configuration changes. The key items to expect for are: garbage collections that are taking too long to run and whether they are happening too oft. This analysis can also help measure the effect of unlike heap size configurations during load testing.

Capture Javacore files to expect for unexpected blocked threads

Javacore files are analysis files that are generated either captured manually or automatically when organization bug (due east.thousand. out of retentivity, deadlocks, etc.) occur. These javacore files include surround information, loaded libraries, snapshot information most all running threads, garbage collection history and deadlocks detected.

Javacore files are created by default on the <WAS_install-root>/profiles/<contour> directory and they are named as follows: javacore.YYYYMMDD.HHMMSS.PID.txt

A javacore file can exist captured in Linux using this command: kill -3 <pid of java process>

A useful tool to analyze these files is the "IBM Thread and Monitor Dump Analyzer for Java".

Typically yous should capture multiple javacore files when symptoms of a problem are happening or nearly to happen (east.chiliad. during a load test). This tool allows yous to compare thread usage amongst the javacore files. This allows identifying blocked threads and identifying what was blocking these threads. In some cases, the blocked threads can be expected (eastward.g. waiting for a HTTP response), but in other cases the stack trace may reveal unexpected blocked threads.

Analyze heap dumps to wait for potential memory growth

Heap dumps are snapshots of the retentivity of a Coffee process. Heap dumps are generated by default in WAS afterward an OutOfMemoryError occurs. In WAS they are saved as .phd (Portable Heap Dump) files. A heap dump can be useful to place potential memory leaks.

The IBM HeapAnalyzer can be used to analyze .phd files.

Go along in listen that phd files can be very big files depending on the heap size. You will likely need to increase the maximum heap size parameter when running the HeapAnalyzer. The heap size needs to be at least the size of the .phd file.

The HeapAnalyzer will prove the allocation of heap for each object form and it will place potential memory leaks. This tool does not show variable values, which makes information technology hard to isolate culprits when you take multiple applications deployed that use similar object types.

Ask IBM Support for help

Equally an IBM customer you have access to IBM support and the ability to create back up tickets (PMRs), so leverage that do good! IBM back up can oft aid analyze these files and help rule out if there are potential retentiveness leaks acquired by the product. They will not assistance yous diagnose problems where the application lawmaking is the culprit, but they will at least rule out the leak is caused by a known or new APAR (Authorized Plan Analysis Report).

We can help!

Troubleshooting performance bug can be wearisome and very time consuming. This post highlights just a few tips and tools to use, but there are many other relevant tests and diagnostics that may be needed depending on your specific state of affairs. Summa has all-encompassing tuning experience. Delight feel free to contact the states to talk over you specific problem.

causeytworiblest.blogspot.com

Source: https://www.us.cgi.com/blog/2013/12/05/websphere-application-server-performance-tuning-and-troubleshooting