lakeFS Acquires DVC, Uniting Data Version Control Pioneers to Accelerate AI-Ready Data
As AI moves into production, organizations are under growing pressure to govern the data behind it. Models are trained on petabytes of multimodal data. AI agents are modifying enterprise systems on their own. And new regulations (like the EU AI Act) are raising expectations around accountability, reproducibility, and access control. Yet for many teams, proving […]
Running an ML workflow means you need a way to quickly access massive datasets. Downloading data from cloud storage before each experiment comes with clear limitations: it’s sluggish, expensive, and difficult to scale. Is there a way out? One approach ML teams are testing is mounting cloud storage as a local drive. Doing that reduces
Databricks built its reputation on openness. Spark. Delta Lake. MLflow. A company that rose by betting on open ecosystems over proprietary silos. Which is why Unity Catalog feels like such a sharp turn. And just now, the Pattern Got Harder to Ignore. This Week at Databricks’ Data + AI Summit, the Pattern Got Harder to
Mounting object storage as a filesystem is the fastest way to get a notebook or Spark job reading S3, Azure Data Lake Storage, or GCS without rewriting it around an SDK. It is also the fastest way to discover what object storage does not give you: no atomicity across files, thin audit trails, and configuration
For the past few years, the hard work in AI has gone into models. Organizations spent that time learning, experimenting, and building the best models they could. That work paid off, and it cleared the way for what’s happening now, everywhere, at breakneck speed: agents. Companies have found real uses for agents across the organization,
Free Virtual Event for Enterprise AI Leaders Building AI that works in production is hard. For most enterprise teams, the biggest obstacle isn’t the model, it’s the data behind it. Study after study shows that organizations abandon AI projects due to poor data quality and inadequate data infrastructure. The gap between AI ambition and AI
As data quantities increase and pipelines become more complex, understanding where data originated, how it changed, and how it is used becomes increasingly important. This is why modern data systems generate more than just data – they also produce data denoting context: metadata. If teams don’t track metadata, they risk losing context that makes their
TL;DR CytoReason is a technology company transforming biopharma’s decision-making—from trial and error to data-driven—through its AI platform of computational disease models. Leveraging an extensive database of public and proprietary data, the company maps human diseases tissue by tissue and cell by cell. Researchers at leading pharma companies, including Pfizer and Sanofi, rely on CytoReason’s technology
Data practitioners rarely need convincing to prioritize data quality – it’s already a top-of-mind concern for most. If data is messy, incomplete, or outdated, teams are basically flying blind. Gartner estimates that poor data quality costs businesses around $12.9 million a year. But ask anyone about metadata quality and you might get a blank stare.
This post recaps a comprehensive tutorial published by Alex Merced from Dremio and Tal Sofer from lakeFS, highlighting how version control transforms multimodal data management for AI teams. The Challenge: Keeping Diverse Data Types in Sync and Queriable Modern AI pipelines consume more than just structured data. Training sets include images, model artifacts, logs, and
Multimodal data is nothing else than data gathered from several sources or formats, such as text, photos, audio, video, and sensor readings. What value does it bring to teams? It collectively provides a more complete, holistic view of the environment. This is especially relevant to AI systems learning to understand and interact with humans more
Making Sense of Large-Scale Data Through Metadata Picture this: Your ML team needs to find all images labeled “defective” from Q3 production runs tagged by a specific annotation workflow to retrain a quality control model. In a data lake with 10 billion objects, how do you find them? For most teams, the answer is: you
Join our community of experts:
introduce yourself, share your knowledge and discover best practices from fellow peers
