Automated Testing in Isolated Environments with GitHub Actions and lakeFS
Back by popular demand, we’re hosting a live community training session where we’ll demonstrate how to build a bulletproof ETL development process.
Deploying new ETL code to production can be nerve-wracking
Will it handle real world data volumes and complexities?
In this live webinar, we will show you how to build a CI/CD safety net using open source lakeFS and GitHub Actions.
This hands on, step-by-step demo will teach you how to:
- Leverage GitHub Actions to trigger a Databricks notebook on every Pull Request
- Automate the creation of an isolated copy of your production data lake using a lakeFS branch (zero-copy clone)
- Run your code changes against this isolated environment, ensuring they work flawlessly before hitting production
- Integrate automated testing throughout the entire process, automatically identifying successful or failed tests
- If the tests fail after running your code against the isolated environment, the Pull Request will be blocked, preventing potentially risky code from reaching production
What you’ll gain in this session:
- Confidence in Production: Deploy ETL updates with peace of mind, knowing they’ve been thoroughly tested against realistic data
- Reduced Risk: Eliminate the chance of unexpected errors or data corruption in production
- Seamless CI/CD Integration: Effortlessly integrate automated data testing into your existing CI/CD pipeline
- Automated Safety Net: Gain complete confidence in your ETL code with a fully automated testing process
Stop worrying about production surprises.
Join us on Monday, June 3 for the live training at 12:00 PM Eastern Time (find your time zone here).
