Macquarie Bank looks to break free of IaaS

Macquarie Bank is hoping to move to a ‘NoOps’ model for managing the public cloud environment that will eventually be home to all of its systems.

Chief technology officer for Macquarie’s banking and financial services (BFS) group Jason O’Connell revealed the strategic shift at a Google cloud summit last week.

Macquarie said back in February 2020 that it wanted to have all its IT infrastructure in the cloud in FY22, and O’Connell suggested that could now be completed within the 2022 calendar year.

“We’re at the point now where next year hopefully we’ll be 100 percent on public cloud, and that’s an amazing achievement for an established bank,” he said.

The bank is driving its existing "systems" into the cloud via one of two paths: either containerising them, or performing a lift-and-shift from Macquarie’s data centre.

“We have been migrating a lot of systems into cloud that couldn’t go into containers just as infrastructure-as-a-service (IaaS), so a lot of those would be lift-and-shift from our data centres into cloud,” O’Connell said.

However, it appears the amount of lift-and-shift is problematic insofar as requiring an ongoing level of operational overhead, whereas Macquarie would rather put that resourcing towards more innovative work.

“Being on systems where we had to run them ourselves in IaaS ... wasn’t quite the end state for us,” O’Connell said.

“We did get a lot of agility and ability to effectively automate our data centres but it came with a lot of operational overheads too, and those operational overheads restrict innovation. 

“How can we just pick up a new technology and use it immediately when we’re so burdened by operations?”

O’Connell said that the bank’s preferred end state is more cloud-native solutions and NoOps, where the environment is so heavily automated that a team of people is no longer required to manage it.

“[We’re] not as agile as we could be compared with cloud-native solutions,” he said.

“Where we really want to go is SaaS [software-as-a-service] and cloud-native PaaS [platform-as-a-service] ... to really unlock agility and be ready for the unexpected.

“So this is where we were looking at cloud-native solutions where we effectively are NoOps, where those services are managed for us by either a vendor or a cloud provider.”

O’Connell said that BFS’ use of Google cloud is an example of how Macquarie hopes to elevate its 100 percent cloud approach from a management perspective.

“We want to move away from managing infrastructure-as-a-service, and in Google cloud we said we’re not going to manage infrastructure-as-a-service, we’re not going to do compute and so how we manage the cloud changed radically,” he said.

“Rather than having pipelines where you would spin up infrastructure, tear it down and spin it up again, instead we’re looking at how do we provision a service … that could last for 10 years, a lifetime, who knows, and we’re provisioning it once.”

O’Connell continued: “We manage the whole of Google cloud with Kubernetes so we have operators in Kubernetes that provision all of our resources, and then we build our own higher-order operators on top of that so that our automation is done via Kubernetes event-driven operators. 

“Then … we’ve got OPA [open policy agent] Gatekeeper, where we’re declaratively defining policy on everything that we’re managing.

“And then finally we added GitOps for all of these resources, so using Anthos Config Management, in order to have Git sync so that when you do a pull request to create something, as soon as that is merged it will then apply into the cloud. It’ll sync into the cluster and then the operator will pick it up and take effect. 

“So this is a very different way of managing the cloud, I think, than what people are used to, but also I think this gives us a lot more ability to compose things together and to be event-driven. 

“I kind of think this is the future of managing the cloud, actually.”

The gradual shift away from IaaS also impacts the skills required of Macquarie’s developers.

“When we ran servers on-premises, developers didn’t worry much about infrastructure,” O’Connell said.

“Then you start looking at cloud and we had two paths - one was the Kubernetes side in which our developers needed to learn Kubernetes, and the other was straight onto IaaS, and then the developers needed to learn how to automate their infrastructure. 

“This was a big change for them - a lot of them became certified and learnt how to manage infrastructure and automate it when they didnt have to do this before. 

“But now we’re starting to move towards those higher order cloud-native services, and this is where it’s changing again. 

“We don’t need developers to learn how to run things and build for stability on the cloud; instead we want to have them learn how to use these cloud-native services. What services are available? How do they piece them together? 

“We’re actually starting to get a lot of our engineers and developers certified at the moment so this is an amazing program where they get the opportunity to get certified, and then also we get the benefit of having more highly skilled engineers as well.”