Delivering high-quality data products requires strict testing of pipelines before deploying them into production.
Today, to test ETLs, you either need to use a subset of the data, or you are forced to create multiple copies of the entire data. Testing against sample data is not good enough. The alternative — testing against your entire data set — is costly and time consuming.
In this webinar, you will learn how to develop and test on the entire production data set with zero-copy, directly from a notebook, using the new and improved lakeFS Python client, enabling:
- Effortless credential management utilizing environment variables (opposed to hard coded secrets).
- Out of the box seamless integration with pre-signed URLs
- Efficient pagination handling
- Intuitive and consistent API structure
