Jenkins - My favourite developer tool in 2011
Ranked #1,739 in Internet, #101,196 overall
Testing, automation, monitoring and building with Jenkins (previously Hudson)
In short, Jenkins improves the quality of software. It provides a simple method to run regular tests on your code, directly out of a source control system, with great reporting and overview tools.
Any task you have which starts to seem long, boring or error prone is an ideal candidate for Jenkins, and with some ingenuity, there are very few problems that can not be automated with it.
Jenkins can be used with a combination of other tools for sophisticated automation, with the aim of using the simplest means possible.
Jenkins can be used with Java, C, C plus plus, Python, Perl, Ruby, Bash Scripts, C# and to test or build code written in most other computer languages.
There are plugins available to automate it with most scripting languages, as well as build tools like Gnu Make, Ant, Maven, Raven and Gradle - to name a few.
Jenkins has plugins to integrate with source control systems, including the popular Subversion, CVS and GIT as well as less popular offerings like Clearcase, Accurev and other solutions.
Jenkins is also FREE. It runs on Windows, Linux and Mac, and can scale to run on simple desktops or largish server farms.
Any task you have which starts to seem long, boring or error prone is an ideal candidate for Jenkins, and with some ingenuity, there are very few problems that can not be automated with it.
Jenkins can be used with a combination of other tools for sophisticated automation, with the aim of using the simplest means possible.
Jenkins can be used with Java, C, C plus plus, Python, Perl, Ruby, Bash Scripts, C# and to test or build code written in most other computer languages.
There are plugins available to automate it with most scripting languages, as well as build tools like Gnu Make, Ant, Maven, Raven and Gradle - to name a few.
Jenkins has plugins to integrate with source control systems, including the popular Subversion, CVS and GIT as well as less popular offerings like Clearcase, Accurev and other solutions.
Jenkins is also FREE. It runs on Windows, Linux and Mac, and can scale to run on simple desktops or largish server farms.
The perfect place for repetition
Of course the worst thing is that they will distract you or your developers from the other development you are doing. Anything that causes a person to loose their context and chain of thought will slow you down a lot, as they then have to get their thoughts back to where they were.
Jenkins has a system that allows tasks to be scheduled to run regularly. Regularly can be every 10 minutes, once a day, one a week or many other timings. It will then run and repeat the task exactly as you specified it every time.
This, done properly, gives you and your team the confidence and time to focus on other tasks.
Making short work of complicated tasks
Getting these jobs under control and into Jenkins delivers real value to a team. First - identify all the steps. Then identify any variable parts - parameters. Write scripts in automation tools to apply these steps, and then have Jenkins orchestrate the whole procedure. It is now tamed down to one button in Jenkins and a couple of parameters, making it much less painful to run.
You now get the next benefit...
Continuous Integration
Jenkins is a continuous integration system. Continuous integration is about running your jobs and tests with repeatability - that is, knowing that they will give a trusted result, and running them often, so that any problems are caught early. It means that you have the outputs from previous builds available, or reproducible to compare versions and determine the point at which problems or regressions occur.
Jenkins started life as a system known as Hudson. The name Hudson is now used for a commercial fork of the project, while the majority of the Jenkins development and support community have continued with Jenkins alongside the project founder Kohsuke Kawaguchi.
Jenkins started life as a system known as Hudson. The name Hudson is now used for a commercial fork of the project, while the majority of the Jenkins development and support community have continued with Jenkins alongside the project founder Kohsuke Kawaguchi.
Repeatability gives you wings!
If something goes wrong - you can either take a look at the problem logs, or you can run it again and watch the process as it occurs - perhaps attaching additional reporting to aid debugging. Debugging with a perfectly reproducible bug is something any coder wishes for.
Of course - this means that you need to ensure consistency...
Unit Testing
One method of testing code is to use unit testing. This is a method of dividing code along module boundaries, and testing the interfaces.
Each module (or unit), has its interface methods tested with a range of inputs and the result is checked against expected outcomes. The inputs should often include bad inputs to ensure code behaves in a well defined way when things go wrong.
The goals of good testing:
Nearly every major programming language has a Unit testing framework, and many of those can be run in a command line setting and producing error reports. Jenkins is able to run these, and if their output is in the right formats, it can produce graphs and browser based interfaces to check test failures or note regressions.
Each module (or unit), has its interface methods tested with a range of inputs and the result is checked against expected outcomes. The inputs should often include bad inputs to ensure code behaves in a well defined way when things go wrong.
The goals of good testing:
- Eliminate undefined behaviour
- Ensure required behaviour is acheived
Nearly every major programming language has a Unit testing framework, and many of those can be run in a command line setting and producing error reports. Jenkins is able to run these, and if their output is in the right formats, it can produce graphs and browser based interfaces to check test failures or note regressions.
Ensuring build consistency - eliminate random elements or variation
This should be a combination of the following (in order of preference):
* Reset all state that you can to known settings. Preferably having tests for different states - don't rely on only one ideal state.
* Query the state before tests start - and chose tests based on this. You need to ensure coverage if you take this strategy.
Beware of things like timeouts and network instability. A great tip - avoid all proxies return trips between local servers.
It is also worth having jobs that run tests of target systems to ensure they are in good condition - problems on those should show up so you don't end up with those compromising the reliability of other tasks.
Eliminating Human Error
Any task that involves a few steps, an environmental setup and a bunch of other things is one that a human running regularly will introduce errors. The more automation that is used on these, the fewer errors, and therefore the loss of time or worse still reputation is caused.Jenkins is not a magic bullet - it will not eliminate errors by itself, but by having an automation, seeing where the automation fails and then correcting it, it will then be able to continue running without problems without much intervention. As builds and requirements change, then builds may need some rework, but well thought out builds that are kept simple should make this a fairly easy task.
Cutting out the drudge - long tasks
Jenkins can also take care of longish or even very long jobs (like overnight soaks) without needing a human babysitter, and can be setup to email the developers concerned with the results of task at the end. This means that you no longer need to check every five minutes on some log scrolling away for 5 hours, you just wait for the email, and if something goes wrong - you have a log to look through, and a workspace in Jenkins to determine what happened.
Jenkins Tip - Slowdown after configuring a job
Sometimes, when you reconfigure a job with a big workspace, Jenkins may become slower, and this seems only to get worse each time you hit the configure button on these jobs. This tip also works for Hudson.
This can be due to the Archive Artifact box causing an AJAX request to verify the file search terms typed into this box. It uses a fairly slow method (in comparison with Unix find) to perform this request - and these can really slow things down.
How can you tell it is this?
You need to look on the Jenkins threadDump page - this can be accessed by adding threadDump to your base Jenkins URL:
http:///jenkins/threadDump.
Look on this page for thread stack traces that have the term 'ArtifactArchiver/checkArtifacts'.
What to do now
if your box is slowing down (ie the Java process is pulling a lot of cpu) and you see this, then you need do two things:
The greasemonkey script prevents the configuration page from starting the ajax requests. It means that the validation is disabled, but the runaway threads will not occur.
This can be due to the Archive Artifact box causing an AJAX request to verify the file search terms typed into this box. It uses a fairly slow method (in comparison with Unix find) to perform this request - and these can really slow things down.
How can you tell it is this?
You need to look on the Jenkins threadDump page - this can be accessed by adding threadDump to your base Jenkins URL:
http:///jenkins/threadDump.
Look on this page for thread stack traces that have the term 'ArtifactArchiver/checkArtifacts'.
What to do now
if your box is slowing down (ie the Java process is pulling a lot of cpu) and you see this, then you need do two things:
- First you will need to restart your Jenkins box - sorry, it may clear itself up in a few hours, but you'll probably want to restart
- Then you will need to use Jenkins with a little greasemonkey script I have - it is now at greasemonkey_checkurl on GitHub.
The greasemonkey script prevents the configuration page from starting the ajax requests. It means that the validation is disabled, but the runaway threads will not occur.
More programming Lenses
Love coding? Why not have a read of my other programming related lenses.
Do you use Jenkins in your work area?
Do you use Jenkins (or Hudson - deployments are still around)?
How do you use it, and do you have any good tips?
Or are you a fan of other CI systems (for example Cruise Control)?
Links and simple HTML are permitted. Comments are moderated - no profanity or spam please.
-
-
robbbin
Nov 13, 2011 @ 2:17 pm | delete
- I use Jenkins. Every time I push something on my GIT repository automatically a POST via wget/curl is executed using the hook "post-receive". If the building is successful the application is deployed on Tomcat. BTW all the life cycle is managed via Maven/ANT and Jenkins run specific Maven commands based on profile.
-
Show me some love!
This module only appears with actual data when viewed on a live lens. The favorite and lensroll options will appear on a live lens if the viewer is a member of Squidoo and logged in.
About Danny Staple
Image sources
Most images in the Lens have come from morguefile.com.
by dannystaple
I have two children - a girl and a baby boy, and we live in Richmond, Uk. I program computers for my day job. In my spare time I build stuff, grow stuff,... more »
- 78 featured lenses
- Winner of 18 trophies!
- Top lens » The Lego RCX, Inside And Out
Feeling creative?
Create a Lens!
Explore related pages
- Noble Collection Harry Potter Wands Noble Collection Harry Potter Wands
- Tron Legacy Outfit Accessories Tron Legacy Outfit Accessories
- PayPal Buy Now Button Security PayPal Buy Now Button Security
- X10 Home Automation X10 Home Automation
- Which Programming Language Is Best? Which Programming Language Is Best?
- C plus plus Behaviour Driven Development tools C plus plus Behaviour Driven Development tools
