Ready to dive into the lake?
lakeFS is currently only
available on desktop.

For an optimal experience, provide your email below and one of our lifeguards will send you a link to start swimming in the lake!

lakeFS Community

CI/CD Safety Net for Databricks ETLs

Automated Testing in Isolated Environments with GitHub Actions and lakeFS

Back by popular demand, we’re hosting a live community training session where we’ll demonstrate how to build a bulletproof ETL development process.

 

Deploying new ETL code to production can be nerve-wracking

Will it handle real world data volumes and complexities?

In this live webinar, we will show you how to build a CI/CD safety net using open source lakeFS and GitHub Actions.

 

This hands on, step-by-step demo will teach you how to:

  • Leverage GitHub Actions to trigger a Databricks notebook on every Pull Request
  • Automate the creation of an isolated copy of your production data lake using a lakeFS branch (zero-copy clone)
  • Run your code changes against this isolated environment, ensuring they work flawlessly before hitting production
  • Integrate automated testing throughout the entire process, automatically identifying successful or failed tests
  • If the tests fail after running your code against the isolated environment, the Pull Request will be blocked, preventing potentially risky code from reaching production

What you’ll gain in this session:

  • Confidence in Production: Deploy ETL updates with peace of mind, knowing they’ve been thoroughly tested against realistic data
  • Reduced Risk: Eliminate the chance of unexpected errors or data corruption in production
  • Seamless CI/CD Integration: Effortlessly integrate automated data testing into your existing CI/CD pipeline
  • Automated Safety Net: Gain complete confidence in your ETL code with a fully automated testing process

Stop worrying about production surprises.

Join us on Monday, June 3 for the live training at 12:00 PM Eastern Time (find your time zone here).

Speakers

Iddo Avneri

VP Customer Success, lakeFS

Amit Kesarwani

Director, Solution Engineering, lakeFS

Register for the live training

Git for Data – lakeFS

  • Get Started
    Get Started
  • Who’s coming to Data+AI Summit? Meet the lakeFS team at Booth #69! Learn more about -

    lakeFS for Databricks
    +