As a developer, especially a developer who is comfortable building servers and slinging code in vi, you don’t want to hear the no word from the sysadmin. You want to be agile and install your own app. You want to configure the extra JNDI settings yourself. And you’re wrong.
I’ve worked in environments where sysadmins ruled with an iron fist on their environments and never let developers on production even to grab log files. Other places developers had wide open access. Wide open access is a bad, bad thing unless you’re still a small startup. Here’s the best model for a corporate IT shop.
- Developers own development for the most part. Sysadmins may backup the server on some filesystem, but that’s about the extent of it. Source control, wikis, continuous integration all run here. Developers are the primary users and everyone else is just visiting.
- Test is all about the testers. Testers should have access to tools like bug trackers and automated testing tools or servers. All the test servers should be locked down with developers and testers given read access to things like log files. Here the sysadmins begin to exert more control because test is the first step formal step in moving an app to production.
- Staging is a pure environment and developers are locked out here.
- Production is a no touch environment. Sysadmins have full control and troubleshooting issues means the developer sits with the sysadmin and doesn’t do the typing.
Developers need to stay focused on quality and not jump around troubleshooting environments. Good sysdamins will say no to developers touching production, but they’ll work closely with them to automate deployments and configuration. Remember they’re saying ‘no’ for your own good.
Hi Ed,
Good post and you propose a good model for enterprise IT shops. One point to note though – if you do fully silo the dev. and test, it is likely that teams will face machine sprawl. Especially virtual machine sprawl if they are using virtualization. Virtual lab automation products help address this (in the model you propose) by bringing a couple of key functionality to the table:
1. Centralized self-serve IT in the lab within bounded policies/quotas as determined by the IT admin. This means that the admin can allow dev/test to create their own multi-machine configs as long as they are within their access policies and quotas (like storage usage etc.)
2. Collaboration – the ability to share configurations between teams – so a tester can share a bug with a captured config. with a dev. easily.
There are other benefits as well (such as automation), and you/other readers could take a look at:
http://blog.vmlogix.com/
http://www.esj.com/news/article.aspx?EditorialsID=3127&pg=1
http://blog.vmlogix.com/2008/07/21/automation-in-virtual-labs-what-exactly-do-we-mean-by-it/
Would love to hear your comments and thoughts either here or on our blog!
Thanks for a great post once again,
- Srihari Palangala, VMLogix
Quite an empirical statement. I’ll bite.
As a developer, I find that unfettered access to production environments decreases the amount of impact an issue will have. When a SysAdmin says “no” to production access and a well seasoned developer is on the other side of that “no” you have the makings of antagonism.
So when you say “best” you mean “Greatest pain in the ass”. Having a sysadmin do “all the typing” leaves the well qualified and intelligent developer on the bad side of a parental power struggle.
I agree. Developers should never touch production.
I had written a long response to this, but when I hit submit, it didn’t take.
Good sysadmins are really easy to work with though. Generally they’ll say no, but could you come over to my cube and we’ll work through it. Then we’ll automate the solution and roll it out to other servers.
As a developer you should have the app pretty much sewn up and ready to be deployed to all the environments once you’re done with it. Deployment should be automatic.
If developers start having production access they become part of the support tree and start getting relied on and do less actual development.
I did ignore the case where your sysadmins are very junior versus a team of senior developers. That has the potential to be a rough relationship, but my approach would be to help train up the sysadmins.
Most of the time developers should be kept off the production. However I understand to some point Shaun’s concerns. When you start forbidding something to people and egos start playing role chances are good you’ll invite antagonisms to the team.
Anyway, you shouldn’t apply any rule thoughtlessly. Suppose you’re a vendor of a system which is a part of core network of mobile operator. Now, you have an emergency situation since one fourth of calls are dropped. You won’t block access to production to anyone who’s able to fix the issue and do it fast. Even if she’s a secretary. By the way the example isn’t made up. OK, issue was fixed by the secretary but by a developer.
We chose to go for a compromise. We limited access to production anvironments to most of developers. Only a lead developer of a project can enter the production. Since they’re experienced people it works well for now.