Breaking the Build at the New Job

I broke the build

My first break of the CI build on a new team came about two weeks into starting the new job. We had made a small change to a dynamically created URL on a single page of the application. Normally this would fall into the category of “too simple to test” for me. My pairing partner pointed out when we started the story that we should put a test in for the change. I argued that changing a URL was too simple a change to bother writing a test for. He disagreed, but let me go ahead after some further discussion. After checking in the feature, about an hour later a developer on the other side of the room piped up with “Someone broke the build.” My pairing partner smiled and said, “I guess we didn’t need to test that.”

It turned out there was another test I didn’t realize had a dependency on that same link and by changing it we had broken an assertion on the specific URL. A bit embarrassed I learned an important lesson. My assumptions about “too simple to break” needed some adjusting.

Over the course of several weeks I came to appreciate that many things were worth testing. A particularly revealing example was testing a new javascript dialog. Javascript is always a pain point when it comes to unit testing, and I’ve often let that coverage of javascript functionality be covered by functional tests usually with Selenium. My pairing partner walked me through a complete TDD approach asserting every part of the function we created and the various DOM elements we used to create it. I did ask a few questions about whether we really needed to test that we had created a DOM element in some exact order at points, but he was firm about testing in every piece. By the end of the feature I had a new appreciation for doing very low level unit testing.

I love that I’m being challenged on not writing enough tests. For so long in past shops I was the hardcore TDD advocate constantly questioned about the real need for testing some component. Now I’m learning to test in even very small changes.

Developer Expectations

I came across a note of mine from last year on my baseline expectations for developers:

  • All code is checked into source control on an hourly basis or at most daily.
  • Every project has an automated build. (Maven, Ant)
  • All projects are setup in continuous integration (Hudson)
  • All code follows the current Java/Groovy coding standards.
  • Unit test coverage of new code must meet a 70% target. TDD is preferred.
  • Code reviews or regular pair programming are required.
  • Code should meet a standard of low cyclomatic complexity through refactoring and design.
  • Some level of functional, integration, and acceptance tests should be performed.
  • High value documentation is maintained.

Multiple IDEs

According to a Forester report the multiple IDEs for Java developers is standard practice:

While Eclipse is common, it won’t become the only Java IDE in enterprise IT shops any time soon. Less than one out of five Java developers have a single primary Java IDE and no other.

— Jeffery S. Hammond (Forrester Research)
IDE Usage Trends

As a victim of attempts to standardize on a single IDE, this is a good counter-weight argument. My personal experience is I primarily work on Java code in IntelliJ IDEA, but every once in a while I want to use some specific Eclipse plug-in. More regularly I’ll drop back to TextMate for some Groovy or Ruby work as well.

Despite the efforts of commercial vendors to sell IT management on the concept of higher productivity with their brand of IDE, the idea hasn’t worked. At the end of the day as a developer in any shop I want to know I can pull down a codebase and run an automated build with no IDE at all. I don’t care if the original developer did it in Eclipse or Netbeans. The key to being a successful coding shop is to remember that IDEs are just tools and should never be required to build, test, or deploy an application.

Continuous Integration Game One Year Later


Hudson has a Continuous Integration Plugin game that gives you points for successful builds, adding tests, removing TODOs, etc. As a plugin it started out simple, but now allows you to get points based on:

  • Doing a successful build.
  • Adding new successful tests.
  • Fixing PMD, FindBugs, or Checkstyle warnings.
  • Fixing a compiler warning.
  • Fixing a TODO.

Around a year ago I tried it out with my development team. It looked like a fun way to reinforce good practices, but I wasn’t sure how it would play out. As a dedicated gamer I loved the idea, but development teams contain all sorts of other personalities.

At the end of the year it was a mixed success. One developer really took to the plugin and even reported bugs and modified the rules to make it reflect reality a bit better. He brought up the scoreboard in a few team meetings and gently tried to foster some competition. At the end of the year he easily led the leader board. I had one of the higher scores myself as a caretaker of Hudson since I often ran builds to clarify something that had been configured wrong or to setup a new project. For the most part the rest of the team knew it was there, but never mentioned their score or ribbed anyone about how low or high they were on the scoreboard.

The key behavior you want on any development team is that the developers care about the CI build and fix it quickly when it breaks. As the year went along this behavior began to be integrated within the team. It took a bit of prodding in some cases, but eventually all the developers felt bad if the build was broken for long and stopped to fix the build before moving on again. The CI game may have played a small role in this, but I was happy with the results. I’d recommend trying it with any team and see how your results go. It didn’t encourage any negative behavior and the results could be impressive with more gaming centric developers.

Staying A Specializing Generalist

Long ago Scott Ambler discussed the idea of a specializing generalist, an element critical for Agile teams. Stumbling across the definition of a generalist who has developed some deep skills in certain areas, but broad skills in a variety of functions felt natural. As a very old web developer I long ago spread out from my development roots. Passion and a love of learning led to a much wider spread of abilities. Just a few were:

  • Project management which I happened into building web sites, intranets, and e-commerce projects. Eventually this would lead to picking up a PMP and moving into a Scrum Master role.
  • Java and J2EE after starting my career with Perl and CGIs. I started with raw servlets and JSPs. From there I moved into Struts, DAOs, JSTL. Then to help with consulting gigs and get a broader view of the large J2EE stack I picked up a bunch of Sun certifications ending with the Sun Certified Enterprise Architect. I consider the path of preparing for all the certification tests and the project worth far more than the cert itself.
  • Build tools were always a passion as I wanted to automate as much as I could. This led to a long time setting up the ant builds for every project and eventually looking into maven and the wonderful rake of the ruby world. Not too long after I discovered cruisecontrol and started setting up CI for every project. The last few years I’ve focused primarily on Hudson which is a fine build server, but I’m always looking for new ideas in CI servers.

As I stepped into management roles staying a generalist was critical. I can’t speak for other professions, but if you don’t stay well steeped in the technologies and practices that your team do much of your feedback will be dismissed as the ‘Pointy Haired Boss’. You need to be able to checkout the current version of a project, do a build, and talk to your tech lead about why they chose to use iBATIS over Hibernate for the ORM. You need to be able to walk through the code in some nasty legacy project and really feel the pain as a developer explains how the one JSP is 4300 lines long.

It’s also insurance against downturns. If you have skills in a variety of areas and can come up to speed quickly on new technologies many options are open. If you’re a manager who’s let the technology side slip away you have to fight for a very limited number of spots out there. Plus being a specializing generalist means you’re always having new learning experiences which is a great payoff all on its own.