Name: Databricks for Data Engineers
Start: 2016-10-26T17:00:00Z
End: 2016-10-26T17:48:47.000Z
Location: BrightTALK

No matter at what stage of your data journey you’re in, this channel will help you get a better understanding of the fundamental concepts of the Databricks Lakehouse platform and the problems we’re helping to solve for data teams.

Organisations are looking to streamline their data management by melding their data warehouse and data lake into a single data lakehouse. A lakehouse provides a highly performant and reliable single source of data at a lower cost.
Learn how digital native businesses, Cascade and Liven, have used the Databricks Lakehouse Platform to:
- Address multiple data science use cases — including recommendation models, logistics and demand forecasting — as well as fraud detection
- Enable their business analysts to access dashboards to view business performance and address issues immediately
- Have a single automated system to manage their data pipelines to support these use cases and more

Building a Data Lakehouse at Cascade and Liven (Feb 2022)

At Databricks, our mission is to make big data simple. We solve the conundrum of how to store, manage and get insight from the sheer variety of data types that exist today using a simple, open and collaborative cloud data platform that we refer to as the Databricks Lakehouse. 

No matter at what stage of your data journey you’re in, this webinar series will help you get a better understanding of the fundamental concepts of the Databricks Lakehouse platform and the problems we’re helping to solve for data teams in being able to work collaboratively and be more productive using one common platform for all data uses-cases.

Introduction to Databricks Lakehouse (Non-technical overview)

Lakehouse is the modern data management paradigm that enables companies to get the most out of their data by leveraging the best of data lakes and data warehouses. This session walks through the evolution of data management and how the lakehouse model enables teams to work in a unified system that provides performant queries, support for streaming, data science, machine learning and BI capabilities -- all built on a single source of truth.

We will also explore Delta Lake - an open source storage layer that brings reliability to data lakes - enabling ACID transactions, scalable metadata handling, and unifies streaming and batch data processing.

Introduction to Databricks Lakehouse (Technical Overview)

As organisations continue to become more data-driven, a collaborative environment for easier access and visibility into the data, models trained against the data, reproducibility, and insights uncovered within the data is critical.


Explore how Databricks lets data scientists and ML engineers quickly move from experimentation to production-scale machine learning model deployments — all on the same platform. In this workshop, we’ll work with a single data set throughout the lifecycle as well as scikit-learn, MLflow and Apache Spark™ on Databricks.

A Guide to Data Science using Databricks

The rise of the lakehouse architectural pattern is built upon tech innovations enabling the data lake to support ACID transactions and other features of traditional data warehouse workloads. Join this session for a walkthrough of the Databricks Lakehouse architecture for Data Engineering.

Learning outcomes:
- How Databricks Lakehouse combines the reliability, performance and governance of data warehouses with the openness and flexibility of data lakes 
- Technology foundations of Delta Lake (Apache Spark™) and Databricks SQL
- How to build highly scalable data pipelines and tackling merged streaming and batch workloads
- Powering data science with Delta Lake and MLflow

A Guide to Data Engineering using Databricks

On-Demand Webinar
Creating and testing machine learning models is time consuming, and delivering results in production at high volumes is prone to error. Automating the development of models is not only helping companies reduce errors but it’s also uncovering powerful business insights - faster.

Join our upcoming webinar, How to Automate Machine Learning and Scale Delivery, to learn how to:
Use automation to dynamically select optimal machine learning models for your use case
Automate the process of deploying models to production with high-volume data pipelines
Configure your automation to quickly scale up and down as data volumes change

How to Automate Machine Learning and Scale Delivery

To remain competitive in today’s real-time world, it’s no longer enough to know what happened yesterday or even in the last hour. Businesses need access to data now, as it’s being generated, to help them make faster and more precise decisions. With AWS, you can easily and cost-effectively ingest, process, and analyze real-time, streaming data, such as application logs, website click streams, video, and IoT telemetry, at any scale, to create actionable insights in seconds.

In this webinar, you’ll learn:
How your organization can leverage real-time, streaming analytics to capitalize on time sensitive opportunities, meet customer demands, and reduce operational risks
How to tackle the architectural and business challenges of transitioning from batch to real-time analytics
How these solutions can help your organization quickly and cost-effectively implement a real-time analytics strategy

Harness the Power of Real-Time Analytics for Rapid Decision-Making

Making quality data available in a reliable manner is a major determinant of success for data analytics initiatives be they regular dashboards or reports, or advanced analytics projects drawing on state of the art machine learning techniques. Data engineers tasked with this responsibility need to take account of a broad set of dependencies and requirements as they design and build their data pipelines.

Join Suraj Acharya, Director, Engineering at Databricks, and Singh Garewal, Director of Product Marketing, as they discuss the modern IT/ data architecture that a data engineer must operate within, data engineering best practices they can adopt and desirable characteristics of tools to deploy.

In this webinar you will learn:
- A framework for describing the modern data architecture
- Best practices for executing data engineering responsibilities
- Characteristics to look for when making technology choices

Data Engineering Best Practices

In this Deep Learning Fundamental Series Part 2, we will cover the principles for training your neural network including activation and loss functions, batch sizes, data normalization, and validation datasets.

All these concepts will be brought to life by demonstrating how Databricks simplifies deep learning - letting you quickly access ready-to-use ML environments, as well as prepare data, and train models faster. After this session, if requested, you will receive the presentation and associated notebooks so you can run the samples yourself.

What is Deep Learning? Deep Learning Fundamentals

Azure Databricks is a Unified Analytics Platform built with a security-first mindset that enables you to run analytics and Machine Learning workloads at scale without compromising on security.

Join Anna Shrestinian, Product Manager at Databricks, and Kevin Clugage, Sr Director of Partner Marketing, as they describes how a number of Azure-specific features fit into the Databricks model for data security, and how to utilize them with best practices designed to make it easier to manage and operate a secure environment. We will also feature a live demo of Azure Databricks to see a few concrete examples of these capabilities in action.

In this webinar, you will learn:
How to use Platform Security features for networking and storage with Azure Databricks such as VNET Injection, No Public IPs and Encryption.
Deploy, Operate and Govern at Scale for Authentication and Authorization with Azure Databricks using Azure Active Directory single sign-on, Azure Data Lake Storage Gen2 credential passthrough and integration with Azure Key Vault.

Security Best Practices 101

Data Security Best Practices 101

Deep Learning has shown tremendous success, but what makes it so special? What are neural networks, and how do they work? What are the differences between popular Deep Learning frameworks like Keras or TensorFlow, and where should you start?

In this webinar, we will show the relationship between neural networks and simpler ML models, and understand what gives neural networks their expressive power. 

We will bring these concepts to life and demonstrate how Databricks simplifies deep learning by letting you quickly access ready to use ML environments as well as prepare data, and train models faster.

About the presenter: 
Denny Lee is a Global Developer Advocate at Databricks. He is a hands-on distributed systems and data sciences engineer with extensive experience developing internet-scale infrastructure, data platforms, and predictive analytics systems for both on-premise and cloud environments. He also has a Masters of Biomedical Informatics from Oregon Health and Sciences University and has architected and implemented powerful data solutions for enterprise Healthcare customers.

Deep Learning Fundamentals: Introduction to Neural Networks

See how Apple, Finra, FIS, Overstock.com, Hewlett-Packard, Shell, Hotels.com and many others overcame these challenges to connect data science and data engineering using Databricks, the company founded by the original creators of Apache Spark™. The results?  Faster performance, scaled data processes, simplified infrastructure, streamlined workflows, and greater collaboration.

Accelerate Innovation by Unifying Data and AI

Netflix is bringing engaging, culturally diverse stories to people all across the globe. With each original movie or TV show, we learn more about what our members want – and rest assured, they want an increasingly broad, deep, global, dynamic library!

Whether it’s planning how to satisfy these global tastes with the right content portfolio or personalizing recommendations to each member, Netflix relies heavily on data science techniques. This talk will highlight some of our core data science strategies and applications involving predictive models & algorithms, experimentation, and analytics.

How Netflix Data Science Powers Global Entertainment

In the last year Hotels.com has begun it’s journey to becoming an algorithmic business. Matt will talk about their experiences of exponential growth in Data Science Algorithms whilst at the same time the team have migrated to using Spark as their core underlying architecture from SAS / SQL, migrated to the cloud from on-premise are transforming the capability of the data science function. He will also highlight the key enablers that have made this successful including CEO support, the internal concepts of organic intelligence and how Databricks has helped make this happen. He will also highlight the pitfalls on the journey.

Hotels.com’s Journey to Becoming an Algorithmic Business

Deep Learning Fundamentals - Training your Neural Network

Comcast is the largest cable and internet provider in the US, reaching more than 30 million customers. Over the last couple years, Comcast has transformed the customer experience using machine learning. For example, Comcast uses machine learning to power the X1 voice remote, which was used over 8B times in 2018 by our customers to find something they love to watch, get the latest sports statistics, control their home, or check their bill and troubleshoot their service using natural language.

What all these different applications have in common is that to create and operate the machine learning models powering these applications we need to ingest many TBs of data on daily basis in an efficient and resilient manner, and need a machine learning platform that allows for fast exploration of new ideas while at the same time automatic deployment of the resulting machine learning models into a production environment that can handle Comcast scale.

In this talk we describe our data and machine learning infrastructure built on Databricks Unified Analytics Platform including how Databricks Delta is used for the ingest and initial processing of the raw telemetry from our video and voice applications and devices. We then explain how this data can be used by both the product organizations to gain deeper insights into how our products are being used, as well as by our research and engineering teams to train and fuel the machine learning models at the heart of of these products. This keynote will also include an end-to-end demonstration of our machine learning platform that is centered around Databricks and MLFlow and how it integrates with other open source machine learning frameworks such as Tensorflow, PyTorch, Sklearn, H20 and Kubeflow to name a few.

Winning the Audience with AI: Comcast’s Journey to Building an Agile Data and AI

Apache Spark has become an indispensable tool for data engineering teams. Its performance and flexibility made ETL one of Spark’s most popular use cases. In this webinar, Prakash Chockalingam - seasoned data engineer and PM - will discuss how Databricks allows data engineering teams to overcome common obstacles while building production-quality data pipelines with Spark. Specifically, you will learn:

- Obstacles faced by data engineering teams while building ETL pipelines; 
- How Databricks simplifies Spark development;
- A demonstration of key Databricks functionalities geared towards making data engineers more productive.

Databricks for Data Engineers

Apache

Spark

Databricks

data

engineering

Cloud computing has exploded over the past few years, delivering a previously unimagined level of workplace mobility and flexibility. The cloud computing community on BrightTALK is made up of thousands of engaged professionals learning from the latest cloud computing research and resources. Join the community to expand your cloud computing knowledge and have your questions answered in live sessions with industry experts and vendor representatives.

Cloud Computing

Practicing business intelligence allows your company to transform raw data into sets of insights for targeted business growth. The business intelligence and analytics community on BrightTALK is made up of thousands of data scientists, database administrators, business analysts and other data professionals. Find relevant webinars and videos on business analytics, business intelligence, data analysis and more presented by recognized thought leaders. Join the conversation by participating in live webinars and round table discussions.

Business Intelligence and Analytics

Welcome to the big data and data management community on BrightTALK. Join thousands of data quality engineers, data scientists, database administrators and other professionals to find more information about the hottest topics affecting your data. Subscribe now to learn about efficiently storing, optimizing a complex infrastructure, developing governing policies, ensuring data quality and analyzing data to make better informed decisions. Join the conversation by watching live and on-demand webinars and take the opportunity to interact with top experts and thought leaders in the field.

Big Data and Data Management

As an IT professional, many of the problems you face are multifaceted, complex and don’t lend themselves to simple solutions. The information technology community features useful and free information technology resources. Join to browse thousands of videos and webinars on ITIL best practices, IT security strategy and more presented by leading CTOs, CIOs and other technology experts.

Databricks for Data Engineers

Presented by

About this talk

More from this channel