Thought Leadership

lakeFS’ First Birthday – The Story

Oz Katz, Einat Orr, PhD
July 28, 2021

The idea About two years ago, we left SimilarWeb, where we had the privilege of managing some of the most interesting and complex data architecture projects in the world. The…

Product

New in lakeFS: Data Retention Policies

Yoni Augarten, Guy Hardonag
July 26, 2021

“I can remember everything. That’s my curse, young man. It’s the greatest curse that’s ever been inflicted on the human race: memory.” — Jedediah Leland, Citizen Kane (1941) lakeFS makes…

Data Engineering

What is Data Lifecycle Management (DLM)?

Paul Singman, Einat Orr, PhD
July 14, 2021

What is Data Lifecycle Management? Datasets are the foundational output of a data team. They do not appear out of thin air. No one has ever snapped their fingers and created…

Tutorials

Introducing the lakeFS Video Tutorial Series

Paul Singman
July 6, 2021

As a developer advocate, it is my responsibility to ensure that understanding how to use lakeFS is simple for users. To this end I’m excited to announce the initial release…

Build Reproducible Experiments with Kubeflow and lakeFS

Tal Sofer, Paul Singman
June 30, 2021

Introducing Kubeflow and lakeFS Kubeflow is a cloud-native ML platform that simplifies the training and deployment of machine learning pipelines on Kubernetes. An ML project using Kubeflow will consist of isolated…

Product

Advancing lakeFS: Version Data At Scale With Spark

Tal Sofer
June 23, 2021

Combining lakeFS and Spark provides a new standard for scale and elasticity to distributed data pipelines. When integrating two technologies, the aim should be to expose the strengths of each…

Data Engineering

How To Measure Data Engineering Teams

Einat Orr, PhD
June 14, 2021

Data teams love calculating and tracking everything with metrics. We already have the infrastructure in place to do so… yet often fail to apply the same strategy for our own…

Product

Guarantee Consistency in Your Delta Lake Tables With lakeFS

Oz Katz
June 2, 2021

How to use lakeFS to keep multiple Delta tables consistent and guarantee data quality….

Product

Air & Water: The Airflow and lakeFS Integration

Itai Admi
May 25, 2021

Today we are excited to announce the official release of the lakeFS Airflow provider! What this package does is allow you to easily integrate lakeFS functionality to your Airflow DAGs….

Data Engineering

Solving Data Reproducibility

Paul Singman
May 19, 2021

Debugging an issue is never fun, but why make it harder? In this post, we show how reproducing data is possible whether interacting with a single file, entire table, or data…

Go Versions: Manage Multiple Go Versions with Go

Barak Amar
May 12, 2021

Updated on April 5, 2022 As a user of the Go programming language, I’ve found it useful to enable the running multiple versions within a single project. If this is…

The lakeFS Blog