Data Warehouse vs. Data Lake: Guide & Key Difference

Idan Novogroder
June 12, 2023

Guide to Enterprise Data Architecture Part 3 If you look at where companies keep their analytical data, you’ll quickly see that this space has split into two major architectural and…

Tutorials

Schema Validation with lakeFS: A Step-by-Step Guide

Iddo Avneri
June 8, 2023

Introduction Schema validation ensures that the data stored in the lake conforms to a predefined schema, which specifies the structure, format, and constraints of the data. It’s important for: Data…

How to Migrate or Clone a lakeFS Repository: Step-by-Step Tutorial

Amit Kesarwani
June 6, 2023

Introduction If you want to migrate or clone repositories from a source lakeFS environment to a target lakeFS environment then follow this tutorial. Your source and target lakeFS environments can…

Data Engineering

Analytical Data: Guide to Enterprise Data Architecture

Iddo Avneri
June 5, 2023

Organizations can accomplish more with their data than ever before thanks to advances in analytical data processing and data democratization initiatives led by the spread of visualization tools, low-code and…

Putting the Write-Audit-Publish Pattern into Practice with lakeFS

Robin Moffatt
June 1, 2023

Step-by-step tutorial on implementing Write-Audit-Publish with lakeFS, including support for multiple data formats….

Data Engineering

How to Implement Write-Audit-Publish (WAP)

Robin Moffatt
May 31, 2023

Comparison of tools and their suitability for Write-Audit-Publish including Apache Iceberg, Apache Hudi, Delta Lake, Project Nessie,and lakeFS…

Data Engineering Patterns: Write-Audit-Publish (WAP)

Robin Moffatt
May 30, 2023

This blog explains the concept of Write-Audit-Publish, which is a pattern in data engineering to enforce data quality in data pipelines….

What is OLTP? OLTP Database Examples, Key Features & Tools

Iddo Avneri
May 28, 2023

Data is a goldmine for every organization, no matter the industry. But to make the most of it, businesses need technology to maintain and manage transactional data like payments, inventory…

The State of Data Engineering 2023

Einat Orr, PhD
May 21, 2023

A lot has happened since 2022, from the rise of Generative AI to the economic slowdown and job losses impacting data practitioners in 2023. In fact, it’s safe to say…

15 Data Engineering Best Practices to Follow in 2026

Einat Orr, PhD
May 18, 2023

Key Takeaways The software engineering world has profoundly transformed in the past decades. This was possible thanks to the emergence of methodologies and tools that helped establish and apply new…

Backfilling Data: A Foolproof Guide to Managing Historical Data

Iddo Avneri
May 15, 2023

If you work with a smaller dataset or do one-off jobs, the way you manage backfills isn’t that crucial. But what if you face constantly growing datasets with billions to…

The lakeFS Blog