lakeFS Acquires DVC, Uniting Data Version Control Pioneers to Accelerate AI-Ready Data
Open source software has fundamentally reshaped technology—delivering unmatched flexibility, low friction, and rapid innovation. For some teams, it’s a philosophical commitment. For others, it’s the fastest path to building. lakeFS supports both models. For most data teams, the journey starts with open source and evolves over time. lakeFS open source offers a robust foundation for […]
It usually starts the same way. A late night message pops up in the channel: “Which v5_final dataset did we use to train the model that just failed in production?” You sigh, open three S3 buckets, and begin the forensic treasure hunt.If that sounds familiar, you’ve already built a portion of an AI Factory. And
Modern machine learning pipelines involve a mix of tools for experiment tracking, data preparation, model registry, and more. MLflow, DataChain, Neptune, and Quilt are some MLOps tools serving these needs. However, one critical piece underpins them all: data version control. This is where lakeFS comes in. lakeFS is not an experiment tracker or ML platform;
We’re excited to introduce a powerful new capability in lakeFS Enterprise: the lakeFS Iceberg REST Catalog – a fully standards-compliant implementation of the Apache Iceberg REST Catalog specification. With this release, lakeFS now enables seamless version control for both structured and unstructured data at any scale. Think Git-style workflows, now for your Iceberg tables. No
We’re excited to unveil lakeFS 1.59.0, a release that brings a completely revamped user interface designed to make your data versioning experience more intuitive, elegant, and productive. Whether you’re a seasoned lakeFS user or just getting started, the new UI makes navigating your data repositories smoother, more informative, and, dare we say – delightful! A
Today’s organizations don’t just use a single data storage solution – they operate across on-prem servers, multiple cloud providers, and hybrid environments. This distributed approach has become necessary, but it comes with significant costs: teams struggle with siloed tools, duplicated processes, and an endless cycle of environment management that diverts focus from delivering actual value.
Supercharging Machine Learning Machine learning (ML) is essential in driving critical business decisions and innovation across various industries. To maintain competitive advantages, organizations continually refine and enhance their ML models through iterative fine-tuning and parallel experimentation. While these strategies are powerful, they come with substantial challenges related to data management, reproducibility, and resource optimization. lakeFS
One of the capabilities of lakeFS is that you can use it to create isolated environments for experimentation or development. Let’s say we want to build a machine learning model and need to prepare or clean some data. With lakeFS, we can do this in isolation without creating an entire copy of the dataset. Let’s
Reproducibility is a fundamental challenge in building reliable machine learning (ML) models and AI applications. It’s not just about debugging a model when it fails in production; it’s also about ensuring that experiments are consistent, avoiding unintended variance, and making incremental progress with confidence. Without reproducibility, ML teams risk wasting time on unreliable results and
Introduction Role-Based Access Control (RBAC) is an effective way to minimize the risk of data breaches by ensuring users only have access to the data and systems necessary for their job roles. Here’s how you can use RBAC to avoid data breaches: 1. Principle of Least Privilege (PoLP) 2. Define Clear Roles and Responsibilities 3.
The General Data Protection Regulation (GDPR) imposes strict requirements on how organizations collect, store, and manage personal data. Businesses must ensure data security, auditability, and access control while minimizing unnecessary data duplication. However, traditional data management practices often make compliance challenging—especially when handling large-scale datasets used in AI and analytics. lakeFS, an enterprise-grade data versioning
As a user of lakeFS, you probably started by using its web interface. But as an object store, lakeFS is also a server that uses storage and is used by client programs. Some of these programs you develop yourself. Others are developed for you – for instance the Garbage Collection clients. You can also use
Join our community of experts:
introduce yourself, share your knowledge and discover best practices from fellow peers
