www.databricks.com Open in urlscan Pro
2606:4700::6812:3b3  Public Scan

Form analysis
0 forms found in the DOM

Text Content

/* SAVE ON TRAINING */ Register today and get 50% off training with code
TRAIN50FOTY

Register now

June 10–13, 2024
San Francisco
 * / Agenda
 * / WHY ATTEND
 * / Pricing
 * / Speakers
 * + More
   * / FAQ
   * / Sponsors
   * / Event Archive

Log InRegister Now

SESSION


STRANGER TRIUMPHS: AUTOMATING SPARK UPGRADES & MIGRATIONS AT NETFLIX


OVERVIEW

EXPERIENCEIn PersonTYPEBreakoutTRACKData Engineering and
StreamingINDUSTRYEnterprise Technology, Media and
EntertainmentTECHNOLOGIESApache Spark, ETL, OrchestrationSKILL
LEVELIntermediateDURATION40

With Apache Spark™ 4 in the pipeline for this year, many of us are looking at
what will be involved in upgrading to the latest and greatest Spark – not to
mention the ever-evolving world of AI libraries. This talk examines how Netflix
has automated large parts of our upgrade and how you can use these techniques
for your data platform. We will share:

 

 * Our cool open source tools that rewrite Spark code
 * Our tools for testing Spark jobs in production
 * How we track the state of jobs
 * Re-using those same tools to migrate to a containerized environment
 * User experiences

 

In this session, you will learn how to: upgrade your Spark pipelines without
crying and validate Spark pipelines even when you don't trust the tests (by
extending the write-audit-publish pattern). This talk is ideal for Data
scientists, ML engineers, and anyone who's inherited legacy data products
platform engineers managing Spark infrastructure.


SESSION SPEAKERS


HOLDEN KARAU

/Engineer
Netflix / Totally Legit Co


ROBERT MORCK

/Software engineer
Netflix

Organized by
*LIGHT / °DARK
Follow

Event Info
 * Agenda
 * WHY ATTEND
 * Pricing
 * Speakers

More
 * FAQ
 * Sponsors

Archive
 * Event Archive


 * Event Terms
 * Code of Conduct
 * Privacy Notice
 * Your Privacy Choices
 * Your California Privacy Rights
 * 

Apache, Apache Spark, Spark, and the Spark logo are trademarks of the Apache
Software Foundation. The Apache Software Foundation has no affiliation with and
does not endorse the materials provided at this event.






WE CARE ABOUT YOUR PRIVACY

By clicking “Accept All Cookies”, you agree to the storing of cookies on your
device to enhance site navigation, analyze site usage, and assist in our
marketing efforts.
Reject all cookies Accept all cookies
Manage Preferences



PRIVACY PREFERENCE CENTER




 * YOUR PRIVACY


 * STRICTLY NECESSARY COOKIES


 * PERFORMANCE COOKIES


 * FUNCTIONAL COOKIES


 * TARGETING COOKIES

YOUR PRIVACY

When you visit any website, it may store or retrieve information on your
browser, mostly in the form of cookies. This information might be about you,
your preferences or your device and is mostly used to make the site work as you
expect it to. The information does not usually directly identify you, but it can
give you a more personalized web experience. Because we respect your right to
privacy, you can choose not to allow some types of cookies. Click on the
different category headings to find out more and change our default settings.
However, blocking some types of cookies may impact your experience of the site
and the services we are able to offer.
More information

STRICTLY NECESSARY COOKIES

Always Active

These cookies are necessary for the website to function and cannot be switched
off in our systems. They are usually only set in response to actions made by you
which amount to a request for services, such as setting your privacy
preferences, logging in or filling in forms. You can set your browser to block
or alert you about these cookies, but some parts of the site will not then work.

PERFORMANCE COOKIES

Performance Cookies


These cookies allow us to count visits and traffic sources so we can measure and
improve the performance of our site. They help us to know which pages are the
most and least popular and see how visitors move around the site.

FUNCTIONAL COOKIES

Functional Cookies


These cookies enable the website to provide enhanced functionality and
personalization. They may be set by us or by third party providers whose
services we have added to our pages. If you do not allow these cookies then some
or all of these services may not function properly.

TARGETING COOKIES

Targeting Cookies


These cookies may be set through our site by our advertising partners. They may
be used by those companies to build a profile of your interests and show you
relevant advertisements on other sites. If you do not allow these cookies, you
will experience less targeted advertising.

Back Button


COOKIE LIST

Filter Button
Consent Leg.Interest
checkbox label label
checkbox label label
checkbox label label

Clear
checkbox label label
Apply Cancel
Confirm My Choices
Reject All Allow All