Shallow Copy For Data: What Are Your Options?

Idan Novogroder
January 15, 2024

In the past five years, we’ve seen many concepts and new tools in the data ecosystem contribute to implementing engineering best practices in data. This trend includes the data mesh,…

Machine Learning

Machine Learning Architecture: What it is, Key Components & Types

Guy Hardonag
January 9, 2024

Teams looking to build machine learning applications that are scalable, easy to maintain, and highly efficient can’t omit the step of building a machine learning architecture. Developing a solid ML…

lakeFS Transactions: Maintain Data Integrity Using ACID Principles

Nir Ozeri
January 7, 2024

We recently introduced the new High Level Python SDK, which provides a friendlier interface to interact with lakeFS, as part of our evergoing effort to make life simpler for data…

Machine Learning

Machine Learning Data: How To Prepare Data For Your Machine Learning Project

Guy Hardonag
January 2, 2024

Data is the core of any ML project. The vast volumes of machine learning data are the most important factor in training algorithms to deliver quality output. The problem is…

Product

Best of 2023: Advancing Data Version Control Best Practices with lakeFS

Keren Shohet
December 27, 2023

As 2023 nears its final stretch, let’s take a moment to reflect on how the year unfolded. One thing that especially stood out is the fact that data version control…

Databricks Autoloader: Ingesting Data with Ease and Efficiency

Idan Novogroder
December 18, 2023

You can ingest data files from external sources using a variety of technologies, from Oracle and SQL Server to PostgreSQL and systems like SAP or Salesforce. When putting this data…

lakeFS-spec: An Easy Way To Work With lakeFS From Python

Jan Willem Kleinrouweler, appliedAI, Max Mynter, appliedAI
December 14, 2023

TL;DR In this blog post, we will explore how to add data versioning to an ML project; a simple end-to-end rain prediction project for the Munich area. The data assets…

Introducing The New lakeFS Python Experience

Oz Katz, Nir Ozeri
December 11, 2023

Since its inception, lakeFS shipped with a full featured Python SDK. For each new version of lakeFS, this SDK is automatically generated, relying on the OpenAPI specification published by the…

What is Databricks and How Does It Unify the Power of Data Science and Engineering?

Oz Katz
November 27, 2023

Data-driven decision-making has become the foundation of business operations across every type of company, no matter the size or industry. Large volumes of data flow from many source systems to…

Unlocking Data Insights with Databricks Notebooks

Idan Novogroder
November 20, 2023

Databricks Notebooks are a popular tool for interacting with data using code and presenting findings across disciplines like data science, machine learning, and data engineering. Notebooks are, in fact, a…

Pre-Signed URLs: How lakeFS Manages Data It Cannot Access

Oz Katz
November 15, 2023

In the world of data management, security is a paramount concern. The more data we generate and store, the more critical it becomes to ensure that data is both accessible…

The lakeFS Blog