Project

Go Project

Improving Postgres Performance Tenfold Using Go Concurrency

Tzahi Yaacobovicz
March 8, 2021

In this article I will show how Go concurrency enabled us to cut through a daunting DB performance barrier. This blog post continues our journey to big data performance. The first post on this issue discussed in-process caching in Go.  The Pain lakeFS is a versioned directory over objects stores like AWS S3 and GCS …

Improving Postgres Performance Tenfold Using Go Concurrency Read More »

Go Project

In-process Caching In Go: Scaling lakeFS to 100k Requests/Second

Barak Amar
March 8, 2021

This is a first in a series of posts describing our journey of scaling lakeFS. In this post we describe how adding an in-process cache to our Go server speed up our authorization flow. Background lakeFS is an open-source layer that delivers resilience and manageability to object-storage based data lakes. With lakeFS you can build …

In-process Caching In Go: Scaling lakeFS to 100k Requests/Second Read More »

Data Engineering Project

The Quick Guide for Running Presto Locally on S3

Guy Hardonag
May 19, 2021

This post aims to cover our experience running Presto in a local environment with the ability to query Amazon S3 and other S3 Compatible Systems. We will: Describe the components needed and how to configure them. Provide a dockerized environment you could run. Show an example of running the provided environment and querying a publicly …

The Quick Guide for Running Presto Locally on S3 Read More »

LakeFS

  • Get Started
    Get Started