How lakeFS Works: Isolation and Governance for AI Data

As AI workloads grow, so does the challenge of managing the data behind them. According to Dun & Bradstreet’s AI Momentum Survey, 97% of organizations report active AI initiatives, but only 5% say their data is adequately ready to support them. That means more teams, pipelines, and agents need access to large, constantly changing datasets, making it increasingly important to control who can access and change data and validate changes before they reach production. When an auditor asks which data was used for model training or how an agent interacted with data, organizations need a reliable answer.

lakeFS gives teams a way to manage this complexity, providing data access controls, change isolation, validation, and history tracking in a layer between object storage and the tools, users, and agents that consume data. Our latest product demo shows how these capabilities work in practice and the decisions they support for people responsible for AI infrastructure.

Key takeaways

• lakeFS sits between object storage and the tools, users, and agents that consume data. No migration, and data stays in place.
• Every user or agent gets an isolated, zero-copy environment of the actual production source data.
• Automated quality gates block bad data before production, and any past version can be restored in one step.
• Every training run and agent action is tied to the exact data it used, with a tamper-proof history.
• Governance is built in: audit trail, role-based and geo-based access control, and shareable Datasets.

Lockheed Martin uses lakeFS as a cornerstone of its AI factory, Volvo experiments on large datasets without duplicating data, and Arm gives teams around the world controlled access to shared data. At Ellips, lakeFS has helped cut model development time significantly. As software engineer David van Son explains: “It used to take our entire ML engineering team two weeks to launch 2-3 new models. After implementing lakeFS, we now launch six new models in the same time with half the team.”

How lakeFS Works in Your AI Data Architecture

lakeFS sits between your object storage and the agents, tools, and users that need the data. It does not replace your storage, compute, or ML tooling, and it requires no migration: your data stays where it is. Every consumer reaches that data through one layer that handles isolation, quality, reproducibility, and governance.

Access rules, quality checks, and audit records are often scattered across bucket policies, pipeline code, and tickets, with every new team or agent adding more permissions and potentially more data copies. lakeFS centralizes these controls in one layer that applies to every consumer, including new agents. Our blog on agentic data access explores this in more depth.

The trade-off is another system for platform teams to operate and a change in how teams work: instead of writing directly to shared paths, they make changes in isolated environments and promote them when ready. Much of the value comes from adopting that workflow.

Isolation Without Duplication: Reducing Data Access Friction

lakeFS gives each user, team, or AI agent an isolated working environment over production-scale data in seconds. The environment is zero-copy: lakeFS creates it through metadata, so nothing is duplicated and no one else is affected by the changes made inside it. Access to real data becomes a policy decision instead of a copy job.

A lakeFS repository can hold large volumes of structured and unstructured data together: images, video, logs, Parquet files, and tables. From any version of that repository, a user or agent gets their own isolated environment (a branch, in lakeFS terms) without duplicating a byte of data. For example, a data scientist can test a new labeling approach on the full dataset. An agent can clean records against production data without write access to production itself.

Because most tools and agents expect files on a local disk, lakeFS can also mount an isolated environment as a local folder. lakeFS fetches data as tools read it and caches it locally, so GPUs aren’t left waiting on object storage, and teams don’t have to rewrite tools to work with remote data.

As a result, experiments that used to need a ticket, a copy pipeline, and a security review start immediately, and storage spend stops scaling with the number of experiments. The trade-off: isolated work has to come back into production, and someone has to own the rules for that.

Quality Gates: Keeping Bad Data Out of Production

lakeFS tracks every change to data, and your team reviews each one and accepts or rejects it before it reaches production. Automated checks run before anyone promotes a change, and a failed check blocks it. If something still slips through, lakeFS preserves every past version, so recovering to a known good state takes one click.

In the demo, deleting a few images from an isolated environment shows them as pending changes, not yet saved. Saving them creates a new version that can’t be altered. Promoting that work into production opens a review where reviewers see exactly what changed before anyone accepts it, the same way engineering teams review code.

lakeFS also runs hooks: automated checks such as schema validation or row-count thresholds that run before every promotion to production. If the data fails, lakeFS blocks the change, and bad data never reaches downstream models and agents (or the dashboard nobody has trusted since the last migration).

For an AI leader, data quality becomes a preventive control: the team decides what it will let into production, writes that down as policy, and can show an auditor the policy at work. Recovery changes too: rolling back is a single revert to the last good version, not a restore job and a week of re-running pipelines. Someone has to write and maintain those checks, though, and a gate nobody maintains turns into a gate everyone works around. Our guide to data compliance covers which controls auditors tend to ask for first.

Reproducibility: Tying Every Training Run and Agent Action to Its Data

lakeFS makes training and agent runs reproducible by tying every experiment, training run, and agent action to the exact data it used. The history records who changed what and when, no one can rewrite or delete it, and any past version stays recoverable. Teams can bookmark the versions that matter and return to them exactly as they were.

You can tag any version. Think of a tag as a named bookmark: “the data that trained LLM version two,” or “the dataset we submitted for regulatory review.” Anyone with access can return to that exact state later without reconstructing it from job logs or a notebook named final_v3_actually_final.

When a regulator, a customer, or your own validation team challenges a model’s behavior, the question “what data did it learn from?” becomes a lookup instead of an investigation. When a retrain regresses, the team compares two data versions rather than guessing what changed. Our practical guide to reproducible AI goes further into the workflow.

Retaining past versions means retaining old objects, so you need a retention policy that matches your regulatory obligations and a cleanup process for everything else.

Governance by Design: Audit Trails, Access Control, and Datasets

lakeFS builds governance into the workflow from the start. Every change, isolated environment, and promotion leaves a record of who did what, when, and why. Role-based and attribute-based access controls make sure the right people, tools, and agents reach the right data, and Datasets let teams share curated data without exposing what sits underneath.

Access control works on two dimensions. Role-based controls govern who, or which agent, can read, change, or promote data. IP-based controls can restrict access based on the source IP address, allowing organizations to enforce network-level access policies. For geographic restrictions, IP ranges can be mapped to regions or filtering can be enforced at the network layer.

Datasets turn the data you already have into curated collections, each with its own identity, permissions, and version history. You can share a dataset with another team, a partner, or an agent without copying data or exposing the repositories underneath. The result is governed data that’s easy to trust and reuse across the organization. Our article on data agents covers what changes once agents start consuming shared data across teams.

The work here is up front. A permission model that spans roles, regions, and datasets needs design time, and it works best when platform, security, and data owners agree on it before the first team onboards.

What Platform Architects Evaluate: Scale, Security, and Deployment

lakeFS can do everything shown in the demo with sub-second operations, even on billions of objects. It plugs into your existing security stack and deploys anywhere, including air-gapped environments. Your data stays exactly where it is throughout. The table below maps common evaluation questions to answers and to what each asks of your team.

In practice, adoption depends most on the last row. The technology installs without moving data; the workflow change is what takes deliberate rollout, usually one team and one pipeline at a time.

Conclusion

You don’t need to restructure your stack to start. Just pick one workflow that touches production data, a training pipeline or an agent that writes back to storage, and run it through lakeFS: give it an isolated environment, put one quality gate in front of production, and tag the dataset each run uses.

What good looks like after that first workflow: experiments start without a copy request, a bad change is blocked or reverted instead of investigated, and when someone asks which data trained a model, the answer is a tag. Then extend the pattern to the next team, and to your agents.

That’s how lakeFS works: the control plane for AI-ready data. Arm, Bosch, Lockheed Martin, NASA, Volvo, and the U.S. Department of Energy trust it with their most critical AI work. Book a demo to see what it can do in your environment.