Ready to dive into the lake?

lakeFS is currently only
available on desktop.

For an optimal experience, provide your email below and one of our lifeguards will send you a link to start swimming in the lake!

lakeFS Community

Go

Data Engineering Go

lakeFS with DynamoDB – How Key Value Store is Used by lakeFS

Itai David

This blog discusses advanced topics within lakeFS. If you are new to lakeFS, or would like to expand your knowledge of how lakeFS works, make sure to check out our documents section. In the Beginning There Was Postgres Up until recently, lakeFS was using a strongly consistent SQL DB, namely PostgreSQL, where all metadata was …

lakeFS with DynamoDB – How Key Value Store is Used by lakeFS Read More »

Go

Building Rich CLI Applications with Go’s Built-in Templating

Barak Amar

Overview The templating package text/template implements data-driven templates for generating textual output. Although we do not benefit from executing the template output more than once, we found it easy to use and helpful for outputting text with colors, marshaling data, and rendering tabular information. By mapping additional functions by name, it is possible to extend …

Building Rich CLI Applications with Go’s Built-in Templating Read More »

Go Project

Loosely Coupled Monolith vs Tightly Coupled Microservices

Barak Amar

TL;DR With some thoughtful engineering, we can achieve a lot of the benefits that come with a microservice oriented architecture, while retaining the simplicity and low operating cost of being a monolith. What is lakeFS? lakeFS is an open source tool that delivers resilience and manageability to object-storage based data lakes. lakeFS provides Git-like capabilities …

Loosely Coupled Monolith vs Tightly Coupled Microservices Read More »

Go Project

Improving Postgres Performance Tenfold Using Go Concurrency

Tzahi Yaacobovicz

In this article I will show how Go concurrency enabled us to cut through a daunting DB performance barrier. This blog post continues our journey to big data performance. The first post on this issue discussed in-process caching in Go.  The Pain lakeFS is a versioned directory over objects stores like AWS S3 and GCS …

Improving Postgres Performance Tenfold Using Go Concurrency Read More »

Go Project

In-process Caching In Go: Scaling lakeFS to 100k Requests/Second

Barak Amar

This is a first in a series of posts describing our journey of scaling lakeFS. In this post we describe how adding an in-process cache to our Go server speed up our authorization flow. Background lakeFS is an open-source layer that delivers resilience and manageability to object-storage based data lakes. With lakeFS you can build …

In-process Caching In Go: Scaling lakeFS to 100k Requests/Second Read More »

Git for Data – lakeFS

  • Get Started
    Get Started
  • The annual State of Data Engineering Report is now available. Find out what’s new in 2023 -

    +