Webinar-Lottie.svg

lakeFS Acquires DVC, Uniting Data Version Control Pioneers to Accelerate AI-Ready Data

webcros.svg

Webinar October 22: Governing Autonomous AI Agents with lakeFS + MinIO

The lakeFS Blog

Airflow and lakeFS

Building Reproducible Data Pipelines with Airflow and lakeFS

Update (May 26th, 2021): We officially released the lakeFS Airflow provider. Read all about it in the latest blog post. In this post, we’ll see

Working with Embed in Go

Working with Embed in Go 1.16 Version

The new Golang v1.16 embed directive helps us keep a single binary and bundle out static content. This post will cover how to work with

Ensuring Data Quality in a Data Lake Environment

Ensuring Data Quality in a Data Lake Environment

The quality of the data we introduce determines the overall reliability of our data lake. And the ingestion stage is a critical point for ensuring

AWS S3 Inventory

Getting Insights from Amazon S3 Inventory

Lists are everywhere. And there’s a good reason for that: lists have an order to them. They reduce our stress by assuring us that everything

MinIO with lakeFS

Git-like Operations Over MinIO with lakeFS

lakeFS is an open source tool that delivers resilience and manageability to object-storage based data lakes. lakeFS provides Git-like operations over your MinIO storage environment

Data Versioning as an Infrastructure

Why Data Versioning as an Infrastructure Matters

The demand for infrastructure that contributes to the collection, storage, and analysis of data is growing with the increasing amounts of data managed by organizations.

Monolith

Loosely Coupled Monolith vs Tightly Coupled Microservices

TL;DR With some thoughtful engineering, we can achieve a lot of the benefits that come with a microservice oriented architecture, while retaining the simplicity and

Data mesh

Data Mesh Applied: How to Move Beyond the Data Lake with lakeFS

The data mesh paradigm The Data Mesh paradigm was first introduced by Zhamak Dehghani in her article How to Move Beyond a Monolithic Data Lake

Object Storage: Everything You Need to Know

While Object Storage is not novel technology, it can still be overwhelming when getting started. Here’s a definitive guide to object-based storage with everything you

chaos data engineering

Chaos Data Engineering

Modern Data Lakes are a complexity tar pit. They involve many moving parts: distributed computation engines, running on virtualized servers connected by a software defined

System tests

System Tests: Lessons Learned From Developing For OSS Project

Overview In this article, I will try to cover some do’s and don’ts for system testing from the perspective of an open-source project. To keep

[hubspot type=form portal=8040338 id=9f5646ec-3e20-4568-9d6e-b82fca022065]