lakeFS Samples: The Quickest Way to Get Started

Iddo Avneri
March 7, 2024

lakeFS is a powerful solution for data version control that enables data practitioners to manage data as code using Git-like operations and achieve reproducible, high-quality data pipelines. While getting started…

Introducing lakeFS Transactional Mirroring (Cross-Region Mirroring)

Ariel Shaqed (Scolnicov), Idan Novogroder, Guy Hardonag
March 5, 2024

What is mirroring We are pleased to announce a preview of a long-awaited lakeFS feature: transactional mirroring across regions. Mirroring builds on top of S3 Replication to provide a consistent…

Machine Learning

Machine Learning Architecture Diagram: Key Elements

Idan Novogroder
March 4, 2024

Machine learning solutions come in handy for addressing various problems and achieving a wide range of goals. However, if we look at ML applications from a distance, we’ll instantly see…

How to Leverage Databricks Workflows to Implement dbt Cloud Jobs

Tal Sofer
February 28, 2024

In a previous article, we explored how dbt and Databricks work together. We will now review a specific feature of dbt Cloud – dbt Cloud Jobs and demonstrate how to…

Product

lakeFS: Where’s my data?

Ariel Shaqed (Scolnicov)
February 21, 2024

If you’ve come across our content, you may have noticed blogs diving into the technical details of lakeFS, and this is one of them. These are lakeFS internals and you…

What is LLMOps? Key Components & Differences to MLOPs

Idan Novogroder
February 15, 2024

Large Language Models (LLMs) are pretty straightforward to use when you’re prototyping. However, incorporating an LLM into a commercial product is an altogether different story. The LLM development lifecycle is…

Best Practices

The cost of poor data quality on business operations

Noy Davidson
February 12, 2024

As a business owner, data specialist, or business intelligence (BI) analyst, you’re likely aware of the critical importance of accurate data in making well-informed strategic decisions. There’s a reason why…

Best Practices

Databricks Architecture Overview: Components & Key Features

Idan Novogroder
February 8, 2024

Many organizations today use a complex mix of data lakes and data warehouses to build the foundation for their data-driven processes. They run parallel pipelines for handling data in planned…

Data Engineering in 2024: Predictions

Oz Katz
February 6, 2024

This article was originally published on Datanami and is republished here with permission. As we officially kick off 2024, I realized I have a few thoughts on the direction of the data…

dbt + Databricks: What are they and how do they work together best?

Tal Sofer
January 31, 2024

It’s clear that the adoption of dbt is picking up, as it now supports major big data compute tools like Spark and Trino, as well as platforms like Databricks. Incidentally,…

Hive Metastore – Did We Replace It With A Vendor Lock?

Oz Katz, Einat Orr, PhD
January 18, 2024

A data lake is often implemented using object storage, and our data resides in objects (files) that we can access through the storage API. Data professionals in analytics and ML…

The lakeFS Blog