Operation Bootstrap

Web Operations, Culture, Security & Startups.

Continuous Integration & Deployment - We Aren't Installing Software Anymore

| Comments

If you pay any attention to some of the folks who are being very open about their successes and challenges scaling a startup then you have surely heard of Etsy. On the surface, most folks response when I first mention Etsy is “Yeah, my wife loves their site!”. What Etsy does for revenue isn’t nearly as interesting to most of us as what they do to make sure that revenue isn’t impacted by the software they build.

If you haven’t heard of any of this before – get a cup of your favorite beverage, find some time and watch this video presented by Etsy originally at SXSW and again in Brooklyn to share with others.

Etsy is just one example of a company who has built a culture, tools and infrastructure around continuous deployment. The very idea of continuous deployment is not well understood by most of the people I talk to. “So, you push code every day? When do you test it?”. Folks can come up with a million reasons why it isn’t right for their organization. Even in the face of painful, multi-hour, often broken from the start deployments, Continuous Integration still gets a “I’m not sure that would work for us” type of response. Weeks of QA effort, tens of hours spent deploying code, and tens of hours more spent debugging the result are apparently what work for most folks.

I am trying to move my organization now in this direction but it’s a difficult journey. As I’m sure is common for others in my position, I work with many talented individuals with a lot of experience. Many of those individuals have seen certain ideas “work” in the past. “Work” is in quotes because typically it was one aspect of a larger system that worked such as “Configuration management worked great for us doing xyz”, or “we just packaged it all up in RPM’s and it was fine”. That’s one piece, and in reality the idea of continuous deployment doesn’t prevent doing those same things – as long as you can do them with speed, agility, and make the overall process easy.

I would love to see some talks about the process folks went through to move in this direction at a company. Etsy talks a little about this – and it comes down to just doing some things. The idea of a “one button deploy” was the starting point and allows abstraction of all the backend complexity away from the deployer making it possible to extend the ability to deploy to more and more people. You make continuos small changes to the deployment system to make the deploys faster, safer, more communicative.

My view of continuous deployment is that it’s a challenge to the organization to be more agile. How can we get engineers as close as possible to being able to make changes to software in production without actually having them do that. If you aren’t up for the challenge then don’t do it – let some smaller, leaner startup who’s more willing to take risks pass you by in the night while your QA team in China is cranking away. Wouldn’t it be great to be able to test small incremental changes safely using production systems, production traffic, and production datasets? Wouldn’t it be great to know that once you, the guy who built the code, has tested it that it’s pretty much ready to go and that any real problems should show up in all the automated testing & monitoring that the company has invested in? Wouldn’t it be great to know that if a problem cropped up you could fix it and get that fix into production the same day?

It sound pie in the sky to some folks, but companies are making this work. It starts with a committment to continuous integration and that becomes a forcing function to better build & release process, better automated testing, better deployment tools, better communication of change, and ultimately to better availability.

This leads to the one thing that is elusive to so many companies: an environment that great engineering minds want to come work at because they can get their ideas to market faster. They can focus on building and they can build better because they can get real world feedback.

So if these are all new ideas to you and you think your deployment process sucks today – start paying attention to this stuff.

Comments