HSBC: Driving breakthrough compute capacity and speed with Dataflow running on Google Cloud
About HSBC
HSBC is one of the world’s largest banking and financial services organizations, serving more than 40 million global customers from offices in 64 countries and territories.
Tell us your challenge. We're here to help.
Contact usHSBC delivers new counterparty credit risk (CCR) and derivative valuation adjustment (XVA) engine with in-house built analytics library NOLA 2.0, powered by Google Cloud. The cloud-native risk management solution boosts calculation speed 10x by combining Graph Theory from Mathematics and Dataflow Elastic compute from Google Cloud.
Results
- 10 months from prototype to production
- Calculation capacity and speed increased 10x while costs lowered
- Solution expanding from two to 40 sites
Calculation capacity increased 10x to three billion calculations per second
Future-proofing risk management on the cloud
Counterparty credit risk (CCR) and derivative valuation adjustment (XVA) calculations are among the most computationally intensive and complex calculations in any bank. They require extreme compute capacity and billions of daily calculations that are critical to understanding, measuring, and controlling a financial institution’s counterparty exposure. The CCR/XVA calculations are used to capture credit, funding, and capital costs for derivatives and form a core part of trading, risk management, accounting, and regulatory requirements.
Following the financial crisis, HSBC, one of the world’s largest financial and banking institutions, scaled up its CCR management capabilities using on-premises IT systems powered by vendor-based analytics solutions. While this setup served the bank well for over 10 years, it was not capable of meeting future regulatory and business demands.
In January 2021, HSBC embarked on an ambitious program of work to enhance its Business and CCR management capabilities to deliver a faster, more efficient, and more cost-effective platform while meeting the new regulatory requirements introduced by Basel III.
“We knew we needed to make changes to our internal processes, and we wanted to build something using cutting-edge, cloud-first technology,” said Faisal Yousaf, Global Head of Treasury Risk Management & Risk Analytics at HSBC. “We wanted to be ambitious, to build a solution in a cost-effective manner that could adapt very quickly and respond to multiple regulations in the various jurisdictions we operate in.”
“We wanted to be ambitious, to build a solution in a cost-effective manner that could adapt very quickly and respond to multiple regulations in the various jurisdictions we operate in.”
—Faisal Yousaf, Global Head of Treasury Risk Management & Risk Analytics, HSBCBoosting compute capability and calculation speed by 10x with Dataflow
In addition to the calculation of CCR and XVA, HSBC wanted to develop a solution able to accurately measure the impact of new transactions and exposures and/or market scenarios on the fly.
“We knew that a cloud-native solution gave us the ability to scale and run at a reduced cost. We did a proof of concept using Google Cloud, and we quickly realized that this could be very successful,” said Dominic Williams, Global Head of Traded Risk Technology at HSBC.
HSBC’s solution, powered by an internally developed analytics library referred to as NOLA 2.0, uses an Apache Beam open-source pipeline running at scale on Google Cloud’s Dataflow. Dataflow’s unified streaming and batch data platform gives organizations the flexibility to define either workload in the same programming model, run it on the same infrastructure, and manage it from a single tool that’s fully managed and highly automated.
NOLA 2.0 is structured using a Directed Acyclic Graph (DAG) embedded with HSBC’s own pricing libraries encapsulated within the pipeline. “Dataflow enables the DAG, which is the DNA of everything we do, to be very well executed,” said Jean Jacques Kamdem, Global Head of Traded Credit Analytics at HSBC. “Dataflow’s auto-scaling elastic compute capability is critical when it comes to the most computationally intensive part of our use case, the sensitivity analysis.”
Working together through daily meetings, the team developed a solution that quickly scaled from prototype to production. “It was a great collaboration, with quite a few development teams involved, and response times were fantastic,” said Williams. “Google Cloud made sure we got out of the development curve and into the production and operations curve as quickly as possible.”
From prototype to production, it took only 10 months to launch NOLA 2.0 for HSBC’s key trading hubs in Hong Kong and London. The solution multiplies the previous calculation capacity, enabling increased analytical capabilities and greater risk management insight and control at lower operating cost.
Using internally developed analytics tools—rather than relying on an external vendor—HSBC can now achieve quicker time to market in response to emerging regulatory changes and business demands.
Taking a cloud-first approach to meeting regulatory requirements with Google Cloud
A key component of NOLA 2.0 is its ability to meet a variety of external regulatory and internal governance requirements, which are only expected to increase in the future. As part of the development process, HSBC assessed and validated the solution’s ability to successfully and securely transition data and compute to the cloud.
“HSBC adopts very stringent cloud data standards, and where appropriate, we obtained the relevant external approvals,” said Michele Marzano, Global Head of Traded & Treasury Risk and Analytics Transformation at HSBC.
“We knew that a cloud-native solution gave us the ability to scale and run at a reduced cost. We did a proof of concept using Google Cloud, and we quickly realized that this could be very successful.”
—Dominic Williams, Global Head of Traded Risk Technology, HSBCTurning more active risk management into competitive advantage
NOLA 2.0 has resulted in a number of business and technical benefits for HSBC. “By boosting our compute capacity, we can run things faster and have greater risk visibility than ever before,” said Marzano. “Where required, traders and risk managers can have multiple rounds of accurate risk exposure numbers throughout the day, which takes us well ahead of our competition.” This framework supports full portfolio re-pricing intra-day and therefore better risk management and control for the bank.
As one of several projects HSBC has launched using Google Cloud, NOLA 2.0 has unlocked the ability to change at a velocity previously unavailable. “The fact that with a Continuous integration/continuous delivery (CI/CD) approach we can make changes to our analytics, build an image, deploy that image within minutes, and then run it at scale is just phenomenal,” Williams said. “When we were on-premises, the provisioning of infrastructure itself would take two weeks for virtual machines and months for physical machines. The ability to spin up new environments very quickly is a game changer.”
Increases in compute speed and launch velocity also provide long-term benefits for HSBC and its customers. “Now we will have the ability to look at different stress scenarios that could occur within the world, like the impact of climate change or of inflation interest rates,” said Yousaf. “We can run a whole host of scenarios and ask, how is our portfolio going to change? That allows us to partake in more active hedging, more active risk management, and to position the trading book to take advantage commercially.
Opening to the door to AI and opportunity
Going forward, as part of HSBC’s commitment to innovation, the team plans to leverage its engagement with Google Cloud to extend NOLA 2.0 beyond London and Hong Kong to 38 additional sites. HSBC is also planning to work with Google Cloud to develop and deploy additional technical solutions, including those using the power of artificial intelligence to support more complex calculations.
Beyond technical solutions, HSBC sees its collaboration with Google Cloud as opening a wealth of opportunities more quickly than anticipated. “The pace at which we've delivered this is particularly exciting. We've managed to make a very substantial change in what is a short period of time for any major organization,” concluded Yousaf. “We can really be ambitious now. We can be bold about what we want to do next. There are lots of ideas coming to the table, and we know we can deliver.”
“It was a great collaboration, with quite a few development teams involved, and response times were fantastic. Google Cloud made sure we got out of the development curve and into the production/operations curve as quickly as possible.”
—Dominic Williams, Global Head of Traded Risk Technology, HSBCTell us your challenge. We're here to help.
Contact us
About HSBC
HSBC is one of the world’s largest banking and financial services organizations, serving more than 40 million global customers from offices in 64 countries and territories.