Agile Development Practices have widely been adopted in R&D Organizations. A core component is Continuous Integration where code changes are continuously integrated and tested to achieve the goal of having "potential shippable code" at the end of every Sprint/Iteration.

In order to verify code changes agile team members write Unit or Functional Tests that get executed against every build and every milestone. The results of these tests tell the team whether the functionality of all features is still given and that the recent code changes have not introduced a regression.

Verify for Performance, Scalability and Architecture
Now we got all these JUnit, NUnit, TestNG, Selenium, WebDriver, Silk or QTP tests that verify the functionality of the code. By adding dynaTrace to the Continuous Integration Process these existing tests automatically verify performance, scalability and architectural rules. Besides knowing that the Product Search feature returns the correct result we want to know:

• How much CPU and Network bandwidth it takes to execute the search query?
• How many database statements are executed to retrieve the search result?
• Whether product images will be cached on the browser?
• How JavaScript impacts the Page Load Time in the browser?
• Whether the last code change affected any of these Performance, Scalability or Architectural Rules?

dynaTrace analyzes all Unit and Browser Tests and validates execution characteristics such as number of database statements, transferred bytes, cache settings ... against previous builds and test runs. In case there is a change (regression) the developers are notified about what has actually changed.

dynaTrace automatically detects abnormal behavior on all subscribed measures, e.g.: execution time, number of database statements, number of JavaScript files, ...

Automatically validate rules such as number of remoting calls or number of bytes transferred

Compare the difference between the problematic and the Last Known Good run

Besides being confident about functionality these additional checks ensure that the current code base performs, scales and adheres to architectural rules.

Step-by-Step Guide to enable dynaTrace in CI
In order to integrate dynaTrace we need to modify the CI Process. The following is a High-Level Step-By-Step Guide that explains all steps in a typical CI environment. When a new build is triggered the Build Server executes an Ant, NAnt, Maven (or any other type of automation script) script that will execute the following tasks:

1. Check-out current Code Base
2. Generate new Buildnumber and then Compile Code
3. (dynaTrace) Start Session Recording (through REST)
4. (dynaTrace) Set Test Meta Data Information (through REST)
5. Execute JUnit, Selenium, ... Tests - (dynaTrace Agent gets loaded into Test Execution Process)
6. (dynaTrace) Stop Session Recording (through REST)
7. Generate Test Reports including dynaTrace Results (through REST)

dynaTrace provides an Automation Library that provides both a Java and .NET Implementation to call the dynaTrace Server REST Services. It also includes Ant, NAnt and Maven Tasks that make it easy to add the necessary calls to dynaTrace. The Demo Application includes a fully configured sample including Ant, JUnit and Selenium.

Once these steps are done dynaTrace will automatically

• identify Unit and Browser Tests
• learn the expected behavior of each individual test
• raise an incident when tests start behaving unexpected

Captured results of tests are stored in individual dynaTrace Sessions which makes it easy to compare and share.
More details on Test Automation in the Online Documentation.

Conclusion
Take your Continuous Integration Process to the next level by adding Performance, Scalability and Architectural Rule Validations without needing to write any additional tests. This allows you to find more problems earlier in the development lifecycle which will reduce the time spent in load testing and minimizes the risk of production problems.

Related reading:

1. Week 6 – How to Make Developers Write Performance Tests I had an interesting conversation with our Test Automation team...
2. How to use your Selenium Tests for automated JavaScript/AJAX Performance Analysis With more than 3 million downloads, Selenium is the world’s...
3. Web Performance Optimization Use Cases – Part 3 Automation In the first and second post of this series I...
4. Hands On Webinar with dynaTrace Dev-Edition on March 30th My colleague Ardeshir Arfaian (aka Ardy) is hosting a Webinar...