Data Engineering

Hive Metastore (HMS): What it is & What Can Replace it

Einat Orr, PhD
August 9, 2021

What is Hive Metastore? Hive Metastore (HMS) provides a single repository of metadata that you can quickly analyze to make educated, data-driven decisions. It’s an important component of many data…

Thought Leadership

lakeFS’ First Birthday – The Story

Oz Katz, Einat Orr, PhD
July 28, 2021

The idea About two years ago, we left SimilarWeb, where we had the privilege of managing some of the most interesting and complex data architecture projects in the world. The…

Data Engineering

What is Data Lifecycle Management (DLM)?

Paul Singman, Einat Orr, PhD
July 14, 2021

What is Data Lifecycle Management? Datasets are the foundational output of a data team. They do not appear out of thin air. No one has ever snapped their fingers and created…

Data Engineering

How To Measure Data Engineering Teams

Einat Orr, PhD
June 14, 2021

Data teams love calculating and tracking everything with metrics. We already have the infrastructure in place to do so… yet often fail to apply the same strategy for our own…

The State of Data Engineering in 2021

Einat Orr, PhD
May 5, 2021

Let’s start with the obvious: the lakeFS project doesn’t exist in isolation. It belongs to a larger ecosystem of data engineering tools and technologies adjacent and complementary to the problems…

Data Engineering

Ensuring Data Quality in a Data Lake Environment

Einat Orr, PhD
January 19, 2021

The quality of the data we introduce determines the overall reliability of our data lake. And the ingestion stage is a critical point for ensuring the soundnes of our service…

Why Data Versioning as an Infrastructure Matters

Einat Orr, PhD
December 15, 2020

The demand for infrastructure that contributes to the collection, storage, and analysis of data is growing with the increasing amounts of data managed by organizations. Every organization that manages data…

Data Mesh Applied: How to Move Beyond the Data Lake with lakeFS

Einat Orr, PhD
December 1, 2020

The data mesh paradigm The Data Mesh paradigm was first introduced by Zhamak Dehghani in her article How to Move Beyond a Monolithic Data Lake to a Distributed Data Mesh. …

How to Manage Your Data the Way You Manage Your Code

Einat Orr, PhD
October 5, 2020

50 years ago it was very hard to collaborate over code. When developing large scale software projects it was difficult to manage changes to source code over time, as revision…

Why We Built lakeFS: Atomic and Versioned Data Lake Operations

Einat Orr, PhD
August 3, 2020

lakeFS is an open source platform that delivers resilience and manageability to your existing object-storage based data lake. With lakeFS you can build repeatable, atomic and versioned data lake operations…

Pick up the Slack with lakeFS