Data Engineering

Introducing lakeview: A Visibility Tool for AWS S3 Based Data Lakes

Lakeview is a new open source visibility tool for AWS S3 based data lakes. Think of it as ncdu, but for Petabyte-scale data. It’s goal

Oz Katz
October 12, 2020

How to Manage Your Data the Way You Manage Your Code

50 years ago it was very hard to collaborate over code. When developing large scale software projects it was difficult to manage changes to source

Einat Orr, PhD
October 5, 2020

Data Engineering

Diary of a Data Engineer

A glimpse into the life of a data engineer. Day 1: Finally, an easy one Got a pretty simple task for a change – read

Oz Katz
September 14, 2020

Data Engineering

How to Pick the Right Postgres for your Application

Lots of applications require a Postgres database. Before you can install them, you will need a Postgres database. How do you pick the right Postgres

Ariel Shaqed (Scolnicov)
September 7, 2020

Data Engineering

Metadata Management: Hive Metastore vs AWS Glue

Introducing the concept of metadata catalogs and explaining the benefits and pains of using Hive Metastore and AWS Glue

Yoni Augarten
August 24, 2020

Why We Built lakeFS: Atomic and Versioned Data Lake Operations

lakeFS is an open source platform that delivers resilience and manageability to your existing object-storage based data lake. With lakeFS you can build repeatable, atomic

Einat Orr, PhD
August 3, 2020

Data Engineering

Manage your data lifecycle at scale with lakeFS Enterprise