Ready to dive into the lake?
lakeFS is currently only
available on desktop.

For an optimal experience, provide your email below and one of our lifeguards will send you a link to start swimming in the lake!

lakeFS Community

Tutorials

Machine Learning Tutorials

lakeFS-spec: An Easy Way To Work With lakeFS From Python

Jan Willem Kleinrouweler, appliedAI, Max Mynter, appliedAI

TL;DR In this blog post, we will explore how to add data versioning to an ML project; a simple end-to-end rain prediction project for the Munich area. The data assets will be stored in lakeFS and we will use the lakeFS-spec Python package for easy interaction with lakeFS. Following model training with initial data, we …

lakeFS-spec: An Easy Way To Work With lakeFS From Python Read More »

Data Engineering Machine Learning Product Tutorials

Introducing The New lakeFS Python Experience

Oz Katz, Nir Ozeri

Since its inception, lakeFS shipped with a full featured Python SDK. For each new version of lakeFS, this SDK is automatically generated, relying on the OpenAPI specification published by the given version. While this always ensured the Python SDK shipped with all possible features, the automatically generated code wasn’t always the nicest (or most Pythonic) …

Introducing The New lakeFS Python Experience Read More »

Data Engineering Machine Learning Tutorials

Unlocking Data Insights with Databricks Notebooks

Idan Novogroder

Databricks Notebooks are a popular tool for interacting with data using code and presenting findings across disciplines like data science, machine learning, and data engineering. Notebooks are, in fact, a key offering from Databricks for generating processes and collaborating with team members thanks to real-time multilingual coauthoring, automated versioning, and built-in data visualizations.  How exactly …

Unlocking Data Insights with Databricks Notebooks Read More »

Best Practices Machine Learning Tutorials

Import Data to lakeFS: Effortless, Fast, and Zero Copy

Idan Novogroder

When adopting a new technology in our organizational infrastructure, one of the foremost considerations is its initial cost. In other words: how many working hours will we have to invest to start using this technology in our system? Often, this question will tip the scales in favor of using a certain solution over another. It …

Import Data to lakeFS: Effortless, Fast, and Zero Copy Read More »

Data Engineering Machine Learning Tutorials

AWS Trino and lakeFS Integration

Amit Kesarwani

A Step-by-Step Configuration Tutorial Introduction In today’s data-driven world, organizations are grappling with an explosion in the volume of data, compelling them to shift away from traditional relational databases and embrace the flexibility of object storage. Storing data in object storage repositories offers scalability, cost-effectiveness, and accessibility. However, efficiently analyzing or querying structured data in …

AWS Trino and lakeFS Integration Read More »

Best Practices Tutorials

The Power of Databricks SQL: A Practical Guide to Unified Data Analytics

The lakeFS Team

In the universe of Databricks Lakehouse, Databricks SQL serves as a handy tool for querying and analyzing data. It lets SQL-savvy data analysts, data engineers, and other data practitioners extract insights without forcing them to write code. This improves access to data analytics, simplifying and speeding up the data analysis process.  But that’s not everything …

The Power of Databricks SQL: A Practical Guide to Unified Data Analytics Read More »

Best Practices Machine Learning Tutorials

ML Data Version Control and Reproducibility at Scale

Amit Kesarwani

Introduction In the ever-evolving landscape of machine learning (ML), data stands as the cornerstone upon which triumphant models are built. However, as ML projects expand and encompass larger and more complex datasets, the challenge of efficiently managing and controlling data at scale becomes more pronounced. These are the common conventional approaches used by the data …

ML Data Version Control and Reproducibility at Scale Read More »

Best Practices Data Engineering Tutorials

Databricks Unity Catalog: A Comprehensive Guide to Streamlining Your Data Assets

The lakeFS Team

As data quantities increase and data sources diversify, teams are under pressure to implement comprehensive data catalog solutions. Databricks Unity Catalog is a uniform governance solution for all data and AI assets in your lakehouse on any cloud, including files, tables, machine learning models, and dashboards. The solution provides a consolidated solution for categorizing, organizing, …

Databricks Unity Catalog: A Comprehensive Guide to Streamlining Your Data Assets Read More »

Best Practices Data Engineering Tutorials

How Data Version Control Provides Data Lineage for Data Lakes

Iddo Avneri

One of the reasons behind the rise in data lakes’ adoption is their ability to handle massive amounts of data coming from diverse data sources, transform it at scale, and provide valuable insights. However, this capability comes at the price of complexity.  This is where data lineage helps. In this article, we review some basic …

How Data Version Control Provides Data Lineage for Data Lakes Read More »

Data Engineering Tutorials

Prefect + lakeFS: How to Troubleshoot Data Pipelines and Reproduce Data

Amit Kesarwani

Prefect is a workflow orchestration tool empowering developers to build, observe, and react to data pipelines. It’s the easiest way to transform any Python function into a unit of work that can be observed and orchestrated. Prefect offers several key components to help users build and run their data pipelines, including Tasks and Flows. With …

Prefect + lakeFS: How to Troubleshoot Data Pipelines and Reproduce Data Read More »

Git for Data – lakeFS

  • Get Started
    Get Started