One one of my projects where we’ve now got a reasonable amount of unit test coverage I did notice a peculiar trend. I’ve been emphasizing with the development team that we want to move to TDD from running a day long class with all of them to posting daily charts of the number of unit tests and the percentage of unit test coverage everyday outside my cubicle wall. I think I’ve been pretty clear on my expectations.

So on this project the developers wrote basically almost zero tests for the first 1.5 Sprints. Then after the felt they’d nailed most of the functionality they started to get around to refactoring, fixing Checkstyle warnings, and writing tests. As it turned out there were initially a lot of Checkstyle warnings about methods that were too large, or lines that were over 120 characters, or imports that weren’t needed, at least several hundred. There were also virtually zero tests.

They did start writing a few unit tests, but they quickly got more interested in fixing Checkstyle warnings. Given an option between fixing a Checkstyle problem and writing a unit test they would choose doing a Checkstyle fix. My intuition tells me that this stems from:

• A lot of the Checkstyle warnings were low hanging fruit and just faster and easier to fix like lines that were over 120 characters or longer methods that needed simple extract method type refactorings.
• They could see faster progress fixing the Checkstyle warnings since it went from say 800 warnings down to less than 100 in the space of a few days.
• Not being that familiar with unit testing and realizing they had to write a large number of tests for all of their code the task seemed simply more daunting. The benefits also seemed less obvious since they weren’t using TDD to drive their design.
• They still aren’t completely sold on the benefits of writing unit tests, though they did realize they were catching some things they missed initially like a lot of situations that could result in null pointer exceptions.

According to Ken Schwaber his recommendation is to wait about 6 months after introducing Scrum before you start to introduce many of the XP practices, because the team is going to have a hard enough time getting adjusted to Scrum. I remember nodding along in agreement when Ken mentioned this.

I’ve watched my company begin to understand that they can cancel software projects. From what I can tell this is relatively new to the culture. Though no one truly likes to be on a cancelled project, it’s a whole lot better than delivering a project 3 years later that doesn’t even have a single customer.

So in the last month I’ve seen at least two projects cancelled:

• One because a third party software system coming online duplicated the custom application functionality. This one got canceled after the first Sprint had kicked off, but better than nothing.
• Another because the business drivers behind the project have evaporated. This was a successful project with multiple phases delivered, but they’ve now decided that the lowest priority phase is not worth doing and that the product is complete.

Both of these are signs that we’re starting to get Agile a bit and that we’re understanding the power of failing fast instead of marching along to an inevitable disaster.

Attended my first Java Users Group (JUG) meeting in Sacramento tonight. My Dad’s attended a computer club meeting in Las Vegas for the last 20 years or so, but I’d never actually gotten around to going to one of these. But given that I feel the need to meet other geeks I decided to check it out plus there was a bit of free food.

The meeting didn’t have an agenda beyond open discussion, so that was a bit worrisome. It appears they’ve had a lot of presentations in the past, but they’ve had a recent drought the last two months. I got a good idea of the cross section of developers and shared a few insights. The majority of the 20 or so people in attendance varied from people new to java to fairly experienced J2EE web developers from what I could tell.

The meeting was actually run by a consultant who spoke off the cuff about a lot of topics and included interestingly enough a java job market update from a local recruiter. Obviously there’s some self interest there, but his basic message of the night was that he had a lot of call recently for EJB experience. His thought was that no one wants to do EJB anymore so the demand for places that still use it is going up. He also explained the realities of experience and resumes for people new to Java. Basically ever since the dotcom days it’s hard to break in until after you have at least a few years of experience.

I brought my 12″ iBook with me and at one point explained to the group that I hadn’t had any issues running Java, Eclipse, IntelliJ IDEA, or JBoss on my Mac running 1.5. Many of them hadn’t seen a lot of Macs even though when I go to conferences these days all the leading edge developers have Powerbooks now. (I actually expect to see a lot of the new Powerbooks announced today with the duo Intel chips in the hands of developers soon.)

Anyway I might just volunteer to run a TDD presentation or a Fitnesse or Cruisecontrol talk since I feel like I should give back a bit to the community. And there’s the concept that eventually my company will have an open position or two and I’d like to have an easier time finding a good candidate. Plus, if I can test infect a few developers so much the better.

One idea I relearned at my recent ScrumMaster training course in Seattle was the idea that Scrum itself is a large catalyst for organizational change. Scrum is a simple set of practices that focus around process improvement using rapid cycles of inspection and adaptation. Ken’s non-scientific estimate was that only 25% of organizations would be able successfully to implement Scrum.

The tough part is Scrum brings a ton of dysfunctional corporate behavior to the surface. When you have 30 days to deliver working software you can’t let impediments like a 30+ day purchase order cycle for new servers get in your way. A lot of impediments around in corporate settings revolve around turf battles. Say the facilities folks won’t allow a team to take over a collocated space like a conference room. At some point as ScrumMaster you have to just move yourself in without permission or raise the issue up the chain until the CEO weighs in on how important it is to maintain a rigid cubicle culture.

Another example is that corporate IT won’t allow any installs of any software by developers so even using a tool like Eclipse or Tomcat requires weeks of lead time to be installed. Someone has to explain to the CEO that the developers are among the highest paid line employees at the company because of their development skills and letting the helpdesk hamstring them because of licensing or security concerns is a tremendous waste of money.

So this week I’ll start challenging the conventional wisdom again with some battles that I’ve conceded in the past.

I came across a post proposing that Washington State agencies adopt agile ideas to track projects:

Our message: stop trying to track project progress by analyzing task level detail (estimated vs. actuals) and instead adopt agile, track velocity, and use empirical measure to predict completion dates.

This sounds like a great idea, and my past history doing a lot of professional service contracts for California state government agencies would suggest that California has the same very broken RFP process.

Essentially the process goes something like this:

• Agency sends out an RFP proposal for some IT system.
• As a consulting shop you either prepare to respond if you knew something about it beforehand. Otherwise as a blind RFP you assume your chances of landing the work are minimal. If your pipeline is pretty dry and you have some people on the bench you’re more likely to respond despite the low odds.

• The RFP is generally pretty sketchy on the requirements, but it does make clear that you must include a detailed Gantt chart with lots of tasks, deliverables, and dates.

  Sometimes as an added bonus the RFP explains that you must use RUP or the IEEE Software Development process or at least follow all aspects of the PMI’s PMBOK.

• And of course it’s always a fixed price bid, so you do the following:

• Throw together an initial plan and cost it out. This often comes to more than a million dollars since there are plenty of unknowns in the RFP.

• Then decide what the agency actually plans on spending which is usually say $200,000.

  Somehow work down to just under that number.

• Then hope that you’ll be able to negotiate the scope of the requirements once you land the job.

• Assuming you actually land the job, at some point you end up working long unbillable hours for free to deliver something that meets their baseline requirements.

• They’re very impressed you actually delivered something, but they don’t have any budget authority to hire outside consultants again for another two years. It’s a horribly broken system which allows for a lot of expensive, time consuming failures that are bad for the state agency and bad for the vendor. Unfortunately given the pace of change and the unfortunate push for the lowest bidder in government IT projects I don’t see it changing soon.

The perfect world would allow a vendor to get hired in based on experience and quality of staff. Then try out the vendor for a Sprint or two at low risk and low cost. If it doesn’t work out fine, let the vendor go and find a new one. If it does then sign up for a longer contract or even just continue to evaluate every Sprint.

Victor Szalvay has pretty much the same suggestion:

We were asked how this would fit into today’s Washington state RFP mechanism which requires detailed up-front task plans. I asked the audience at large if this task-based approach currently yields successful results. Judging my the silence and the shaking of heads, I’d say the Washington state government is ready for real change. I suggested they hire vendors that make short-term iterative commitments, pay those vendors based on delivery of the short-term commitments and fire those that do not.

If I ever return back to the the state or local government sector I’d want to work for a CIO who was willing to try this. Given that the Agile movement seems to be catching on overall maybe we’ll see Scrum in government before too long, but there are a lot of institutional issues to overcome.