DataOps Unleashed - The Evolution of a Data Platform

Presented by

James Felder, Senior Data Engineer, Cox Automotive

About this talk

DataOps Unleashed is the first-ever event for the global DataOps community. We came together on Wednesday, March 17th, 2021 for DataOps Unleashed – a gathering of DataOps, CloudOps, AIOps, MLOps, and data-oriented DevOps professionals, including all data team members and their management, up to the CDO level. We shared the latest trends and best practices for running, managing, and monitoring data pipelines and data-intensive analytics workloads. James Fielder, Senior Data Engineer at Cox Automotive, shows how a small data team manages DataOps for his organization’s global footprint, highlighting their use of Databricks on Microsoft Azure. Designing a data platform is no easy task, particularly when there are new technologies, techniques, and approaches appearing every week. At Cox Auto UK we have been on a journey from manually deployed Hadoop clusters to a full platform as a service setup using Azure Databricks. This journey hasn’t always been smooth however and we’ve learned some things along the way! In this talk, we examine how we have made design choices while evolving our platform, our decision to open source some of our work, and what our past, present, and future look like.

Related topics:

More from this channel

Upcoming talks (0)
On-demand talks (85)
Subscribers (5774)
At Unravel, we see an urgent need to help every business understand and optimize the performance of their applications, while managing data operations with greater insight, intelligence, and automation. For these businesses, Unravel is the AI-powered data operations company. We offer novel solutions that leverage AI, machine learning, and advanced analytics to help you fully operationalize the way you drive predictable performance in your modern data applications and pipelines.