Topic
Data engineering
Most of the data science on this site only worked because of the engineering underneath it: streaming pipelines for tens of millions of network events a day, a telematics workload modernised on Databricks, and decision support systems built over municipal data. Ubunye Engine is the open source result of meeting the same pipeline plumbing problem in every one of those places, and the case studies say what the data layer had to do in each.
4 work · Wikidata Q104659521
Work
- Decision support systems for municipalities at the CSIRDjango based predictive analytics and decision support systems serving 17 municipalities, including the City of Cape Town, built at the CSIR.
- Generator optimisation and streaming at VodacomReal time analytics and optimisation for a national telecoms network: generator dispatch across 15,000+ sites and tens of millions of events a day.
- Insurance data science: telematics, flood risk and MLOpsLeading insurance data science at ABSA Insurance: telematics processing cut from months to under a day, flood risk across 230,000+ properties, MLOps.
- Ubunye Engine: portable Spark pipelines for data and MLUbunye Engine is an open source Python framework for config driven Spark pipelines. The same task folder runs on a laptop, Docker, Kubernetes or Databricks.
Related topics
Subjects that share work with this one. Those with their own page are linked.
- Python
- Apache Spark
- MLOps
- Databricks
- Decision support systems
- Docker
- Kubernetes
- Technical leadership
- Apache Flink
- Apache Kafka
- Climate risk
- Data science