Topic
MLOps and production machine learning
Getting a model into production and keeping it there is a different problem from building it. The work here covers that problem from three sides: modernising a data science function at a bank toward a governed path from experiment to production on Databricks, running optimisation and streaming systems for a national telecoms network, and Ubunye Engine, an open source framework whose model registry, lineage and run anywhere guarantee are tested rather than promised.
3 work · 1 writing · Wikidata Q60753505
Work
- Data science platform at a bank: telematics and MLOpsAt Absa Group: architected the data engineering capability data science runs on, with Ubunye Engine at 100M+ events a day for 10+ products.
- Generator optimisation and streaming at VodacomReal time analytics and optimisation for a national telecoms network: generator dispatch across 15,000+ sites and tens of millions of events a day.
- Ubunye Engine: portable Spark pipelines for data and MLUbunye Engine is an open source Python framework for config driven Spark pipelines. The same task folder runs on a laptop, Docker, Kubernetes or Databricks.
Writing
Related topics
Subjects that share work with this one. Those with their own page are linked.
- Machine learning
- Software engineering
- Apache Spark
- Data engineering
- Python
- Technical leadership
- Databricks
- Docker
- Kubernetes
- Reproducibility
- Apache Flink
- Apache Kafka
- Climate risk
- Data science