Data science platform at a bank: telematics and MLOps
Building the platform data science runs on at a bank
Good models are only one part of a working data-science function. The data, engineering, governance, tooling and team around them decide whether anything survives in production.
- What it is
- Programme of work
- Role
- Lead Data Scientist, leading a data science capability and team
- Organisation
- Absa Group
- Period
- March 2024 to present
- Status
- Active
- Built with
- Databricks
- Apache Spark
- MLflow
- Python
- Topics
- MLOps
- Telematics
- Climate risk
- Geospatial machine learning
- Insurance
- Data engineering
- Technical leadership
The problem
When I joined, much of the analytics ran on application infrastructure that was not built for big data, with little documentation, no shared team strategy and little collaboration. Capable people were building good models, but the path from an idea to a governed production system depended on individual knowledge and one off effort.
Why it mattered
A data-science capability is more than a collection of models. Underwriting, retention, fraud, telematics and climate risk all need work that can be reproduced, operated and improved by someone other than its author. Without the surrounding engineering and governance, a good model is a one-off.
The context
A large, regulated bank, mid migration from on premises data and workflows to the cloud (AWS and Databricks). Confidentiality limits what can be said about specific systems, so what follows is scope and consequence, not internals.
What I did
I led the modernisation strategy for the data science products and architected, from scratch, the data engineering capability that the data science capability now sits on. Platform: Databricks brought into the bank, CI/CD pipelines, and integration with legacy systems. The pipelines run on Ubunye Engine, which processes at least 100 million events a day and is the engineering layer for 10+ products. People and ways of working: I lead a team of three data scientists, set the best practices, teach them, and document everything in Confluence. We introduced AI and model governance aligned to the bank’s standards. Applied systems: telematics for products such as Activate, about 7 to 10 million trip records a day; flood forecasting used by underwriting teams, now on the same platform; and the hyper personalisation strategy, building customer lifetime value and segmentation models.
What changed
Telematics processing, about 7 to 10 million trip records a day, was modernised from cycles of months to under a day. Databricks, CI/CD and Ubunye Engine gave a reproducible path from experiment to production, so several analytical products can run at once with less manual intervention and clearer ownership.
Who benefited
Underwriting and business teams, through earlier visibility of physical risk and better behavioural understanding; the data scientists on the team, who can ship more reliably and depend less on any one person; and ultimately customers, through more relevant decisions and interactions.
What remained
A data engineering capability on Databricks and Ubunye Engine that 10+ products sit on, modernised telematics and flood forecasting, AI and model governance, and a team that depends less on any one person.
Technical context
Databricks, Spark and PySpark, Ubunye Engine, Databricks Asset Bundles, MLflow, Unity Catalog, CI/CD, AWS, legacy system integration, geospatial modelling, MLOps and model lifecycle management, AI governance, cloud migration from on premises.
Related research
- Directly connectedProposed: self supervised flood mapping from radarProposed doctoral research, not yet registered: physics informed self supervised learning for flood extent mapping from synthetic aperture radar.
- Shares geospatial machine learning, climate riskMachine learning for seasonal climate forecastingIBM Research work on forecasting 2m temperature and precipitation weeks to months ahead with machine learning, published at NeurIPS 2020 CCAI and EGU 2022.
Publications
- Shares climate riskLong-Range Seasonal Forecasting of 2m-Temperature with Machine LearningTests whether two machine learning models, a convolutional and a recurrent neural network, can beat climatology when forecasting 2 m temperature up to 52 weeks ahead at six locations. The models improved on climatology up to 30 weeks lead time by correlation and up to 52 weeks by RMSE skill score, but only at some locations, and the authors note further work is needed for the models to add value in the tropics.
- Shares climate riskML-based Probabilistic Prediction of 2m Temperature and Total PrecipitationProposes a daily probabilistic model that forecasts 2 m temperature and total precipitation globally, aimed at the skill gap between weather and seasonal forecasting. Physics based ensembles, climate modes and recent climatology are combined as inputs to Extreme Gradient Boosting, U-Net and Natural Gradient Boosting models. The authors report that it consistently outperforms both ECMWF 46 day forecasts and climatology. Co-authored across IBM Research in South Africa, Kenya, Brazil and the US.
Related work
- Shares databricks, mlopsUbunye Engine: portable Spark pipelines for data and MLUbunye Engine is an open source Python framework for config driven Spark pipelines. The same task folder runs on a laptop, Docker, Kubernetes or Databricks.
- Shares mlops, technical leadershipGenerator optimisation and streaming at VodacomReal time analytics and optimisation for a national telecoms network: generator dispatch across 15,000+ sites and tens of millions of events a day.
- Shares telematics, pythonTFiltersPy: Kalman and particle filters for PythonTFiltersPy is an open source Python library of Bayesian filters: Kalman, extended, unscented, ensemble and particle, behind one scikit-learn style API.
- Shares geospatial machine learning, climate riskClimate forecasting and geospatial ML at IBM ResearchApplied ML research at IBM Research: climate forecasting models deployed into IBM PAIRS Geoscope, published work, and a provincial COVID-19 risk dashboard.
Related writing
- Shares technical leadershipBuilding a Data Science Capability From the Ground Up: A Real Life RoadmapTwo years ago I inherited a small data science team with no handover, no documentation, and production models nobody could explain. This is the conversation I wish someone had with me before I started: the challenges, the costs, the thinking behind every move, and the one belief that carried it all. Build systems that outlive the people who built them.
- Shares pythonHow I Learned to Build My Own Python Libraries (From Curiosity to Real Work)creating from 1st principles