Branching your database is the start. Branch your whole data ecosystem.
Databricks published a good post recently about Lakebase, walking through how Glaspoort, a fiber operator in the Netherlands, ships database changes with the same discipline they ship application code. Every environment branches from production. Every pull request gets its own fresh, disposable database. The migrations are the source of truth, not the running databases.
If you have ever shared one database across a team, you already know why they did this. Someone runs a migration that passed on their machine, it lands in the shared environment, and half the team’s work breaks for the afternoon. The fix is exactly what the post describes: stop sharing one fragile environment, and give everyone a cheap, throwaway copy that looks like production.
We have been making this argument for a while, so it is good to see it running in production and written up honestly, limitations and all. The Glaspoort team is candid about what they still do by hand. That is worth more than a clean success story.
Here is the part I want to add. The pattern is right. It just stops too early.
Why stop at the database?
Branch from production. Work in isolation. Gate the change. Throw the environment away when you are done. None of that is specific to a transactional database. It is how you should treat any data you cannot afford to corrupt.
And most of the data behind an AI system is not in a transactional database. It is sitting in object storage. Parquet files, sure, but also images, audio, video, training sets, model checkpoints, and feature tables in Iceberg, Delta, or Hudi. This is the data your models actually learn from, and it is the data that is hardest to version, because it is large, it is multimodal, and it is spread across systems that were never designed to branch.
So the question I would ask after reading the Glaspoort story is not “should we branch the database.” It is: why does branching stop at the database? If the pattern is good enough for the OLTP layer, it is good enough for the petabyte-scale data feeding your models.
That is the layer lakeFS works at.
Why does branching stop at the database? If the pattern is good enough for the OLTP layer, it is good enough for the petabyte-scale data feeding your models.
What branching the lake looks like
lakeFS is the control plane for AI-ready data, powered by a highly scalable data version control architecture. It brings Git-like operations to the data lake: branch, commit, merge, revert, tag.
The important part is what a branch actually is. When you branch in lakeFS, no data gets copied. A branch is a metadata operation, so you get an isolated, production-shaped copy of a petabyte-scale dataset in seconds, and it costs you nothing in storage. You run your experiment, or your pipeline change, or your agent, on that branch. When you are done, you throw it away. Production never notices.
If you liked the Glaspoort pattern, the mechanics will feel familiar:
- Branch from production, for free. One branch per experiment, per pipeline run, per agent. No cascade of child environments to tear down first.
- Gate the change before it lands. Open a pull request for data, and tie your data quality tests to the commit and merge with hooks. Bad data gets blocked before it reaches production, not discovered after.
- Roll back in seconds. When something does go wrong, revert to a known-good commit in one operation. Seconds, not a restore-from-backup afternoon.
- Reproduce anything. Every commit is an immutable snapshot, so any experiment or agent action ties back to the exact version of the data it used.
One honest difference, and the Glaspoort team is upfront about their side of it too. They deliberately never merge databases back into each other. They promote a change by replaying migrations against the target. For schema changes to a transactional database, that is the right call. lakeFS actually merges the data, atomically, the way Git merges code. These are different problems, and I do not want to pretend otherwise. Merging a schema migration and merging a few terabytes of Parquet are not the same job. For the file-based, multimodal data in a lake, atomic merge is what lets a team work on the same data without stepping on each other.
The part that matters most: your data does not live in one place
Here is where lakeFS does something different.
The Glaspoort pattern is clean, and it lives inside one platform. That is fine if your world is one platform. Most of the data ecosystems I see are not. The training data is in one place, the feature store in another, the raw unstructured data somewhere else, and often across more than one cloud.
lakeFS works across that mess:
- Your data stays where it is. lakeFS is a thin layer between your storage and the tools that read it. It does not copy your data and it does not replace your object store. Your bytes stay in your bucket, under your control.
- Any storage. S3, Azure Blob, GCS, any S3-compatible store, POSIX. On-prem, public cloud, private cloud, GovCloud, air-gapped.
- Any tool. Spark, Trino, Dremio, Airflow, Pandas, Polars, Hugging Face Datasets, MLflow, SageMaker, Vertex AI. lakeFS sits alongside what you already run. It does not ask you to move off it.
- Any format. Structured, semi-structured, and unstructured, together. Parquet, and the open table formats: Iceberg, Delta, Hudi.
I am not saying one approach beats the other. I am saying versioning should not stop at the edge of a single system, because the data that decides whether your AI works is scattered across your whole ecosystem. The control plane for it should reach that far too, including onto Databricks, where lakeFS already integrates.
Versioning should not stop at the edge of a single system, because the data that decides whether your AI works is scattered across your whole ecosystem.
Governance you do not have to apply by hand
The Glaspoort post is refreshingly honest about the manual parts. Object-level grants applied by hand. A single application user, with per-user authorization pushed up into the app layer instead of the database. They call the grants item the clearest thing on their list to improve, and I think that honesty is exactly right.
Doing governance by hand does not scale, which is why lakeFS builds it in: role-based access control, SSO, SCIM provisioning, branch protection rules, and an automated audit trail across every workload, so you know who changed what data, when, and why. If you operate under real compliance constraints, that audit trail is the difference between passing a review with the evidence already there and reconstructing six months of history from memory.
This gets more important with agents, not less. An agent touching your data needs the same isolation, the same gate, and the same audit trail a person does. In lakeFS, each agent gets its own zero-copy branch, its changes sit behind a pull request until a human approves them, and everything it did is recorded. Same control plane, whether the actor is a tool, a person, or an agent.
An agent touching your data needs the same isolation, the same gate, and the same audit trail a person does.
Take the pattern further
If the Glaspoort story landed for you, you already believe the core idea: data environments should be branched from production, isolated per unit of work, gated before they merge, and cheap enough to throw away that you actually keep them honest.
I just think that idea deserves to run across your whole data ecosystem, not only the transactional database. Every cloud, every tool, every format your models depend on, with governance and reproducibility built in instead of bolted on later. Without that, you are versioning one corner of your data and flying blind on the rest.
Try the quickstart, or take a look at lakeFS for Databricks, which turns your Unity Catalog into a versioned catalog so you can branch, commit, and roll back production data directly.
