Bendigo and Adelaide Bank meet open-banking mandates with AWS cloud migration

To balance performance with cost savings, Bendigo started running its non-production workloads on Amazon EC2 Spot instances, which let customers take advantage of unused Amazon EC2 capacity in the cloud.

Bendigo started its digital transformation journey on AWS so it could comply with Australia’s open-banking mandates. In March 2020, Bendigo quickly migrated its infrastructure to the cloud using Amazon Elastic Compute Cloud (Amazon EC2), which provided secure and resizable compute capacity for virtually any workload.

The bank also achieved greater resiliency and scalability by implementing other AWS features for fault-tolerant workloads.

With this new infrastructure, Bendigo has reduced its compute costs by approximately 60 per cent, increased the resiliency of its workloads by nearly 30 per cent, and improved the performance of its banking system by roughly 20 per cent.

Meeting open-banking regulations by migrating to the cloud on AWS

Bendigo is aiming to be Australia’s bank of choice by providing its 2.1 million customers with data transparency and simple banking solutions. Bendigo holds over $83.4 billion in deposits, and it serves over 110,000 shareholders while employing over 7,000 people across 317 branches.

In May 2018, the Australian government passed the Consumer Data Right legislation, which required all major financial institutions to comply with open-banking practices by July 2020. This legislation also meant that financial institutions must provide their customers with access to and control over their personal data. Facing these industry changes, Bendigo expected an increase in demand on its on-premises infrastructure. To prepare, the bank decided to undergo a digital transformation on AWS.

Bendigo chose AWS as its cloud service provider because of the maturity of its services, and in March 2020, it kicked off its cloud migration using Amazon EC2. The bank also began running its workloads using Amazon EMR, a cloud big data solution for running large-scale distributed data processing jobs, interactive SQL queries, and machine learning applications using open-source analytics frameworks.

VIEW ALL

To meet open-banking mandates, the company configured clusters that would help it maintain a high availability across its banking systems. After completing this project, Bendigo wanted to improve its cloud architecture and reduce compute costs by running its clusters using Amazon EC2 instance fleets.

According to Adam Hobbs, development infrastructure engineer for Bendigo, cost savings wasn’t on the radar initially.

“We realised that we wanted to improve the performance of our architecture,” Hobbs said.

In 2021, the Bendigo team entered the second phase of its digital transformation journey, and the team began experimenting with new solutions and features to meet the bank’s computing needs.

Using Amazon EC2 Spot instances to reduce compute costs

Bendigo engaged with the AWS team to test Amazon EC2 instances that would maintain a high availability while optimising compute costs. During this time, the Bendigo team trialled Amazon EC2 Spot instances, which deliver Amazon EC2 instances at a lower price compared to Amazon EC2 On-Demand Instances.

Bendigo’s developers use its non-production environment for iterating features for its digital offerings, the bank identified this as an area where it could reduce compute costs. Now, the bank runs its non-production workloads entirely on Spot Instances.

Leandro Silva, data lead for the open banking project at Bendigo, further explained that by making these changes, Bendigo has reduced its compute costs by approximately 60 per cent.

“Amazon EC2 Spot Instances have been an effective way for us to save on compute costs while maintaining an elastic workload,” Silva said.

The bank also tested different Amazon EC2 instances for running its Amazon EMR clusters, and it found the best performance using diversified Amazon EC2 instance types, which include varying combinations of CPU, memory, storage, and networking capacities.

By diversifying its instance types, the bank has reduced the likelihood of service interruptions and downtime and made its workloads more fault tolerant.

“We went through stages of configuring different instance types,” Hobbs added, noting that under the new model, the bank has improved the resiliency of its near-real-time workloads by roughly 30 per cent.

“We spent some time with the AWS team identifying which instance types have the highest availability, which has delivered a much better experience,” Hobbs said.

Bendigo’s Amazon EMR clusters consist of different node types, which use Amazon EC2 instances for cluster capacity planning.

Hobbs added that core nodes had also been set-up.

“We went from a five-node cluster to a 30-node cluster,” Hobbs said.

To facilitate effective cluster scaling, the bank implemented task nodes, which it can spin up or spin down to increase capacity as needed and save on compute costs. Since then, the company has grown to use 60 core nodes for its production environment.

With multiple task nodes, Bendigo’s Amazon EMR clusters can use other available task nodes if one fails, minimising service interruptions and downtime. By increasing the elasticity of its infrastructure, the bank has increased the performance of its system by approximately 20 per cent while driving down compute costs.

The bank also started using managed scaling for Amazon EMR, which automatically resizes its clusters for best performance at the lowest possible cost. Using managed scaling, the company can manage its compute costs more effectively by setting up minimum and maximum limits for its compute capacity.

According to Ash Austin, service owner of the cloud platforms team at Bendigo, working on the cloud provides deeper insights about workloads, which help to monitor usage and true costs.

“Working on the cloud helps us enhance our risk and vulnerability management in a highly secure, robust, and controlled environment,” Austin said.

Building a data lake to enhance machine learning capabilities

Bendigo is currently modernising its core banking system on AWS so that all of its branches can use a unified banking system that provides transactional updates in near real time.

The bank is also building a data lake using Amazon Simple Storage Service (Amazon S3), an object storage service offering industry-leading scalability, data availability, security, and performance. By securely storing its unstructured data in the cloud, the bank is laying the foundation for integrating machine learning and artificial intelligence alongside its digital offerings.

“AWS has been great at helping our organisation find the right tools for the job and navigate the regulatory landscape,” Austin added.

“It brings us the global expertise that we need to accelerate our digital transformation journey.”

[Related: APAC ranks the third-highest region globally targeted by ransomware]