Machine Learning

lakeFS Community: Leonard Aukea nominated for Machine Learning Professional of the year!

Adi Polak
November 16, 2022

Our community is full of people with incredible skills and know-how. And this nomination proves us right! Our community member @Leonard Aukea has been nominated for Machine Learning Professional of…

Thought Leadership

What a Year With lakeFS Taught Me

Michal Wosk
November 10, 2022

A year ago, in November 2021, I took a bold decision to leave my role at Microsoft and join team lakeFS as VP Marketing. Upon joining I wrote an article…

4 Ways to Reduce Cloud Data Storage Costs

Oz Katz
November 4, 2022

In the past year, words like recession, business slowdown and monetary cuttings are being heard more and more often. Not just in the economic press and in the media, these…

How to Build an Isolated Testing Environment for Data with lakeFS

Barak Amar
October 27, 2022

Overview Our routine work with data includes developing code, choosing and upgrading compute infrastructure, and testing new and changed data pipelines. Usually, this requires running our tested pipelines in parallel…

How to Develop Spark ETL Pipelines in Isolation

Amit Kesarwani, Vino SD, Iddo Avneri
October 26, 2022

You’re bound to ask yourself this question at some point: Do I need to test the Spark ETLs I’m developing? The answer is yes; you certainly should – and not…

lakeFS with DynamoDB – How Key Value Store is Used by lakeFS

Itai David
September 20, 2022

This blog discusses advanced topics within lakeFS. If you are new to lakeFS, or would like to expand your knowledge of how lakeFS works, make sure to check out our…

EPCOR Implements Write-Audit-Publish Framework for Data Pipelines with lakeFS

Stephen Seewald, Raghvendra Verma, Cory Matheson
September 14, 2022

It is no secret that modern businesses run on big data. If your business was a car, big data would be the engine that powers it. All businesses want to…

Top Data Engineering Conferences & Events You Don’t Want to Miss in the Fall of 2022!

The lakeFS Team
August 23, 2022

As the Covid-19 pandemic loosens its grip on the world, we’re all eager to start travelling and meeting in person again. The great news is that in persone conferences are…

Data Manageability: The revolution that is turning Data Trust into the New North Star

Einat Orr, PhD
August 17, 2022

A few weeks ago, I was looking at a dashboard in our internal BI system. It’s a simple system. Redash over PostgreSQL that has just a few hundreds of thousands…

One Spark job, Many Data Sources – How to Easily Use lakeFS with Spark

Jonathan Rosenberg, Tal Sofer
August 15, 2022

lakeFS is an interface to the data lake, or the parts of the data lake one chooses to version control. The lakeFS interface is S3 compatible, and hence easily used…

Data Mesh: What is it and What Does it Mean for Data Engineers?

The lakeFS Team
August 10, 2022

Organizations have practically always needed data analytics, and they jumped on the analytics bandwagon as soon as the first computers appeared on the scene. In the 80s, businesses built data…

The lakeFS Blog