RSpec Stubs with no_args

We’ve been getting pretty particular about our stub/mock expectations at work. A few months ago I would been perfectly happy with:

1
TwitterGateway.stub(:new).and_return(double)

I didn’t worry about specifying that I didn’t pass any arguments to the constructor. After it was pointed out that the stub didn’t really fully specify its’ expectations I changed to this style:

1
TwitterGateway.stub(:new).with().and_return(double)

Then a colleague pointed out a nice bit of syntactic sugar. You can simply specify a no_args matcher if no arguments are passed in:

1
TwitterGateway.stub(:new).with(no_args).and_return(double)

A more complete stub with better readability. Reminds me of how I fell in love with RSpec the first time I saw it back in 2006.

10,000 Tests and Counting

I played a “Yeah” sound effect in campfire a few weeks ago in celebration of checking in our 10,000th test. It was a milestone worth celebrating with both Creme Brûlée Bread Pudding and a chocolate chip cookie. Stepping back a few years I had to fight policy battles just to allot any development time to testing or even check it into CVS with the production code.

Some good things about 10,000 tests and counting:

  • We have pretty good confidence that we can catch breaking changes throughout the app. CI and a suite of much slower QA Acceptance tests add to that confidence.
  • We can run the entire suite of 10,000 RSpec examples in about 8 minutes on the newest Macbook Pros with 16GB of RAM and 4 cores plus hyper threading.
  • Finding old crufty areas of the codebase that aren’t’ tested is a rare surprise rather than a common experience.
  • Even our large “god” classes are generally well tested.
  • We’re constantly thinking about ways to increase the speed of the overall run to at least keep it under the 10 minute threshold rule of thumb. This tends to lead to good refactoring efforts to decouple slow tests from their slow dependencies.

Some not so good things:

  • Many of the ‘unit’ tests are really light integration tests since they depend on database objects, Rails ActiveRecord objects in our case.
  • Some of our ‘god’ classes have 3000+ lines of tests and take 2-3 minutes to run on their own.
  • We have to rely on methods like parallel tests to distribute our unit test running.
  • If it doesn’t look like a change will impact anything outside the new code we sometimes skip running a full spec and let the CI server catch issues.
  • Running individual specs that use ActiveRecord often take 5-8 seconds to spin up, which is painfully long for a fast TDD cycle.
  • Our full acceptance test suite still isn’t consistent enough and running on CI so we have even more of a dependency of trusting the indirect integration testing in our unit test suite.
  • We’d like to use things like guard or auto test, but we haven’t been able to make them work with such a large number of tests.

Even with all the cons of a really large test suite, I love that we have it and run it all day long.

One Year with Jasmine

It’s been about one year since we introduced Jasmine as our default for Javascript testing. Looking back it’s easy to declare it a success:

  • We test all our new javascript instead of just deciding it isn’t worth the effort.
  • Javascript is broken out into files instead of having the temptation of just leaving it inline in a view template.
  • Functions are broken down and refactored to be small and testable.
  • We’ve even been able to test some complex Closure javascript using Jasmine instead of JSUnit.
  • Running the full Jasmine suite is a part of every CI build.

If we’d only achieved a few of these things I’d consider it a big success. For a long time my default approach was to handle Javascript testing through functional Selenium based tests. While these are valuable tests, they certainly don’t help doing TDD with Javascript. Jasmine has finally allowed me to stay in a BDD/TDD workflow when switching between Ruby and Javascript.

Jenkins: My Personal Bodyguard

I’ve been running Jenkins as my CI solution for years now, but for the first 6 months at my new job I used it in an entirely new way. The typical CI pattern is you setup to run against checkins to your master branch run all the unit tests, and depending on how sophisticated you are integration tests, static analysis, or even performance tests.

In my current shop we had a standard CI build using CC.rb against checkins to our master branch. It ran about 5000+ specs sequentially and if everything’s green master is available for production pushes. That fulfills the standard case for CI just fine, but I quickly found that it a little light for my needs. The issue is we relied on feature branches for rolling out all of our stories and while those stories are being worked there’s no CI job at all. Another pain point is even with parallel specs and a nice 4-8 core processor the full test run takes 5-10 minutes so it doesn’t get run nearly as often as one might like.

I’d let Jenkins be my default choice for CI in Javaland, but I was really open to switching it out for a more Ruby style CI server. I figured the first place to look was our already running CC.rb implementation. I tried out CC.rb long ago and remembered liking it a lot better than the original cruise control, but with years of development on it I figured it had evolved. Turns out CC.rb is a pretty basic solution for CI and it hasn’t had much love over the years. It runs a build showing the console output and a very basic web console. Coming from Jenkins the lack of features and plugins was pretty shocking.

Given the Ruby community’s love for XP practices there must be a better option available. I looked at numerous CI servers for Ruby including CI Joe, Integrity, and Travis. All of them followed the lines of providing a very simple CI server much like CC.rb. Travis CI appeared to be a bit more with its use of a hosted open source solution, but it is completely tied to hosting your project on Github. None of these projects showed the depth of a solution like Jenkins.

So the solution was to run Jenkins as my own personal bodyguard. We were looking at swapping out the CC.rb implementation, but it wasn’t a priority right away. So I dived in and setup Jenkins to run a build on my laptop. Instead of just building master I added whatever git feature branch I was working on and hooked up an audible trigger to play James Brown “I feel good!” for successes and “Houston I think we have a problem” for failures. I expected to use the rake plugin to launch the build, but it turned out it was much easier with using things like rvm and bundler to simply execute a script. My final scripts for the build looked something like the following:

1
2
3
4
5
bash -l -c "rvm use 1.8.7-p330@acme"
bash -l -c "bundle install"
bash -l -c "rake db:migrate"
bash -l -c "rake parallel: prepare"
bash -l -c "rake parallel: spec"

So finishing up a bit of functionality, I’d commit and about 10 minutes later my laptop would exclaim “I feel good”, or occasionally “Houston I think we have a problem” to let me know I missed something. So I had my own personal bodyguard build and I could happily let it run in the background.

This year we replaced CC.rb with Jenkins and dropped feature branches so I’ve retired my laptop Jenkins, but I recommend the practice if you find yourself in a similar spot.