From the Blogosphere
How We Use XL TestView By @PaulvanderEnde | @DevOpsSummit #APM #DevOps
Let me first explain how we manage the configuration of our Continuous Integration systems
By: XebiaLabs Blog
Jul. 31, 2015 04:45 PM
How We Use XL TestView to Improve the Test Quality of XL Deploy
Here at the XebiaLabs we care a lot about quality. That means we have to do a lot of testing for our products. Since we have a relatively small development team, and a code base that grows rapidly, that means we have to automate our tests. To get an impression, for XL Deploy, we have automated tests for components like the user interface and the server. But we also have automated tests for all our plugins, in combination with different types of middleware, operating system, etc. This all adds up to an enormous test landscape with an Continuous Integration system containing a couple of hundreds of Jenkins jobs, and thousands of unit and integration tests.
But even if you have all of your tests automated, you still have another challenge. How are you going to monitor all those thousands of tests and track all the test results? We do a lot of integration testing with virtual servers and real middleware. Since there are so many moving parts, it’s a matter of time before you run into stability problems. With just a Continuous Integration system, it’s hard to analyse which builds/tests are flaky, which tests fail most often, where the biggest problems are so where to start improving. Continuous Integration systems generally lack the possibility to analyse your test results.
Well, it seems we have a product at XebiaLabs to solve this problem: XL TestView. At XebiaLabs we regularly have Innovation Days where developers try to tackle problems we are facing. This time we took the opportunity to integrate the Continuous Integration system of XL Deploy with XL TestView, to get more insight into our test results.
Before I explain how we did that, let me first explain how we manage the configuration of our Continuous Integration systems. Like I already mentioned, we have a lot of Jenkins jobs. How do we organise and administrate them? Some time ago we have scripted our entire Jenkins infrastructure configuration with Jenkins job dsl. This allows us to automatically generate all job configuration with the push of a button.
Since all the information about our test infrastructure is also in this system, we decided to extended the Jenkins job DSL to make it possible to create test specifications in XL TestView using the REST api. When we create a Jenkins job, we also create a new test specification in XL TestView. So now when we automatically configure Jenkins, we also automatically configure XL TestView for XL Deploy, and directly configure all Jenkins jobs to upload the test results to XL TestView using the Jenkins plugin for XL TestView. This plugin collects all the test results from the Jenkins workspace and uploads them. All this made it possible to track all our thousands of tests in XL TestView, in just 1 day.
And this is what that looks like:
Jenkins job configuration to upload the test results to XL TestView:
A big list of test specifications in XL TestView
XL TestView dashboard showing the first results:
Learn more and try it for yourself! Visit https://xebialabs.com/products/xl-testview/ and get started!
The post How we use XL TestView to improve the Test Quality of XL Deploy appeared first on XebiaLabs.
Latest Cloud Developer Stories
Subscribe to the World's Most Powerful Newsletters
Subscribe to Our Rss Feeds & Get Your SYS-CON News Live!
SYS-CON Featured Whitepapers
Most Read This Week