For an optimal experience, provide your email below and one of our lifeguards will send you a link to start swimming in the lake!
TL;DR lakeFS Cloud offers Audit Logs for compliance, operational stability, monitoring access, activities and security analysis. In the latest version of lakeFS Cloud, we introduced our new Audit Logs feature, providing detailed information on all user actions across all regions. Audit Logs – what is it and why should you care? Using the lakeFS Audit …
Introduction: Dockerize Your Data Pipeline I can remember times when my company started using a new technology — be it Redis, Kafka, or Spark — and in order to try it out I found myself staring at a screen like this: At the time I thought nothing of doing this. And even wore it as a badge of pride …
The Everything Bagel II: Versioned Data Lake Tables with lakeFS and Trino Read More »
Recently, we’ve heard from several community members experimenting with new development workflows using lakeFS and dbt. The timing isn’t surprising given dbt’s more recent support of big data compute tools like Spark and Trino that are some of the most commonly-used technologies by lakeFS users managing a data lake over an object store. The combination …
dbt Tests – Create Staging Environments for Flawless Data CI/CD Read More »
“I can remember everything. That’s my curse, young man. It’s the greatest curse that’s ever been inflicted on the human race: memory.” — Jedediah Leland, Citizen Kane (1941) lakeFS makes data corruptions easy to avoid and fix by allowing you to travel back in time to any state of your data. This new capability has …
Update (May 26th, 2021): We officially released the lakeFS Airflow provider. Read all about it in the latest blog post. In this post, we’ll see how easy it is to use lakeFS with an existing Airflow DAG, to make every step in a pipeline completely reproducible in both code and data. This is done without …
Building Reproducible Data Pipelines with Airflow and lakeFS Read More »
If you’re interested in playing around and exploring lakeFS, you can now easily get started using the Katacoda demo which provides a personalized sandboxed environment – all from your browser, without installing anything. lakeFS is an open source platform that delivers resilience and manageability to object-storage based data lakes. With lakeFS you can build repeatable, …
The lakeFS Katacoda Sandbox Environment – Interactive Data Versioning Learning Read More »
This post aims to cover our experience running Presto in a local environment with the ability to query Amazon S3 and other S3 Compatible Systems. We will: Describe the components needed and how to configure them. Provide a dockerized environment you could run. Show an example of running the provided environment and querying a publicly …
The Quick Guide for Running Presto Locally on S3 Read More »
This tutorial aims to give you a fast start with lakeFS and use it’s git-like terminology in spark. It covers the following: Quick start to install lakeFS using Docker Compose. How to create a repository, add files to it, create a branch and make changes to the repository using spark jobs. How to review changes …
© Copyright 2022 | All Rights Reserved | Privacy Policy | Terms of Use
