How Seek's dev team hit 400 new deployments a month

A four-year, cloud-first turnaround has seen jobs site Seek turn its deployment rate around from two late-night, junk food-fuelled production deploys every month to roughly 400.

“[In 2012] we bought a bunch of pizzas and a bunch of beer, then about 3am the code was released and the server went back online. That was how it was done,” Seek’s DevOps manager Andrew Hatch told the Future IT Infrastructure forum in Melbourne last week.

Glossing over “the pain and suffering I have been through in this past four years”, he said the online classifieds business was now averaging 20 deploys into its production environment a day.

“We used to have this combative [approval] process when you went to your masters and begged to release a change. We couldn't sustain that. Now it is an email or a message on Slack," he said.

“We can make a change and have it in front of the customer in 20 minutes."

Hatch and Seek’s security boss Andre Bertrand said the business - once “a bit fat and heavy” - found itself with new competitors snapping at its heels in 2012, and realised it had to learn how to move a lot faster.

The organisation restructured its IT operations the following year to bring website software development and its product business together, merge IT operations and infrastructure, and create a new line of reporting for the solutions and software architects that are working on improving and getting rid of legacy code.

Since then the organisation has adopted a cloud-first mantra which means all new code is deployed in the AWS public cloud by default, rather than its Adelaide data centre.

Old platforms have proved a bit more of a headache, however, and the cloud journey has by no means been an easy one.

“We did try to move our test environments as they were into the cloud. It didn't work at all. It needed a completely different way of architecting, and it was a big mistake for us," Bertrand said.

"Cloud is not cheap, unless you really know it in and out and you've gone to Spotify and poached all their best developers and architects. 

"If you're like Seek and you dove in head first and basically threw the floodgates open because you were originally so rigid, it can get very expensive - and it did get very expensive for us.”

The Seek infrastructure team learned the hard way to automatically shut down public cloud development environments at 6pm every day and on public holidays.

They are now working to embed DevOps processes into the development team to build a sense of ownership and pride around the products they are building for customers.

“We put developers on call. Our mantra is if you build it you support it," Hatch said.

“People are taking responsibility for what they're doing and there is pride associated with that."