Traditional RAG often feels like a black box. Vector search is powerful, but hard to reason about, and that makes AI agent knowledge versioning nearly impossible: you can’t track which version of your data an agent was served, who changed it, or why.
Emerging standards such as Open Knowledge Format (OKF) aim to change that. It treats your knowledge base like versioned code. This approach brings structure to agent context, relying on frontier models’ ability to explore, find and reason about information without forcing it into a complex representation. It is gaining momentum because it makes AI agents more reliable and easier to audit.
OKF gives teams a simple way to package knowledge for humans and agents: Markdown files with a YAML frontmatter.
But retrieval alone does not solve the production questions: which version did the agent use, who changed it, was it reviewed, and can access be governed?
lakeFS provides those missing pieces by adding branches, commits, diffs, tags, rollback, auditability, access control, metadata search, and scalable agentic access patterns.
OKF defines what the agent should know. lakeFS is what makes that knowledge trustworthy: versioning, tagging, and auditability.
In this guide, we’ll build a small-ish OKF repository with the lakeFS high-level Python SDK, tag an immutable release, mount it with lakeFS Mount, and consume it from a Python agent built with Deep Agents. While the example itself might be small, this pattern actually scales to petabyte- scale repositories!
Why AI agent knowledge needs versioning
An OKF bundle is a directory of Markdown files, but without version control, you still don’t have real AI agent knowledge versioning, just files.
Each file is a concept: a metric, table, dashboard, API, playbook, business definition, or anything else an agent should understand.
---
type: Metric
title: Weekly Active Users
description: Unique users with at least one qualifying event in a calendar week.
resource: urn:metric:weekly_active_users
tags: [analytics, engagement, metric]
timestamp: 2026-07-06T00:00:00Z
---
# Weekly Active Users
Weekly Active Users is the count of distinct `user_id` values with at least
one qualifying event in the target calendar week.
## Depends on
- [Events table](/tables/events.md)
- [Freshness playbook](/playbooks/freshness.md)This lines up with the LLM Wiki pattern: keep agent knowledge as plain files, link related concepts, and let agents read the same material humans can review.
But once agents rely on this knowledge, a folder of files is not enough. You need to know who changed it, what changed, which version was approved, which version production used, and how to roll back.
That is the lakeFS part.
The workflow
Let’s get started building this flow.
Setup
python -m venv .venv
source .venv/bin/activate
pip install lakefs pyyaml deepagents langchain-openaiConfigure lakeFS:
lakectl configure # enter your credentials and lakeFS endpoint
export OKF_REPO="okf-knowledge"
export OKF_STORAGE_NAMESPACE="s3://my-bucket/okf-knowledge"To learn more about lakeFS Enterprise, visit our docs or start a free trial.
Publish an OKF bundle
The script below creates a repository if needed, writes a small analytics OKF bundle, validates the required type field, commits the bundle atomically with a lakeFS transaction, and tags the release.
from __future__ import annotations
import os
from datetime import datetime, timezone
import frontmatter
import lakefs
REPO_ID = os.environ.get("OKF_REPO", "okf-knowledge")
STORAGE_NAMESPACE = os.environ["OKF_STORAGE_NAMESPACE"]
BUNDLE = os.environ.get("OKF_BUNDLE", "analytics")
PREFIX = f"okf/{BUNDLE}"
TS = datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
# Docs without frontmatter (index/log) are exempt from OKF metadata validation.
EXEMPT = {"index.md", "log.md"}
DOCS: dict[str, str] = {
"index.md": """\
# Analytics OKF
Start here:
- [Weekly Active Users](metrics/weekly_active_users.md)
- [Events table](tables/events.md)
- [Freshness playbook](playbooks/freshness.md)
""",
"metrics/weekly_active_users.md": f"""\
---
type: Metric
title: Weekly Active Users
description: Unique users with at least one qualifying event in a calendar week.
resource: urn:metric:weekly_active_users
tags: [analytics, engagement, metric]
timestamp: {TS}
owner: analytics-platform
sensitivity: internal
---
# Weekly Active Users
Weekly Active Users, or WAU, is the count of distinct `user_id` values with at
least one qualifying event in the target calendar week.
```sql
select
date_trunc('week', event_ts) as week_start,
count(distinct user_id) as weekly_active_users
from analytics.events
where event_name in ('app_opened', 'page_viewed', 'workflow_completed')
group by 1;
```
## Depends on
- [Events table](/tables/events.md)
- [Freshness playbook](/playbooks/freshness.md)
""",
"tables/events.md": f"""\
---
type: Table
title: analytics.events
description: Canonical event stream table for product analytics.
resource: urn:table:analytics.events
tags: [analytics, events]
timestamp: {TS}
owner: data-platform
sensitivity: internal
---
# analytics.events
Canonical event stream table for product analytics.
Important columns:
| Column | Description |
|---|---|
| `user_id` | Authenticated user identifier |
| `event_name` | Product event name |
| `event_ts` | Event timestamp in UTC |
| `ingested_at` | Warehouse ingestion timestamp |
""",
"playbooks/freshness.md": f"""\
---
type: Playbook
title: Analytics freshness investigation
description: How to investigate stale analytics tables.
resource: urn:playbook:analytics-freshness
tags: [analytics, freshness, incident-response]
timestamp: {TS}
owner: data-platform
sensitivity: internal
---
# Analytics freshness investigation
Use this playbook when an analytics metric is missing data or the latest
partition is older than expected.
1. Check ingestion lag for [analytics.events](/tables/events.md).
2. Compare `event_ts` with `ingested_at`.
3. Verify upstream event delivery.
4. Backfill only after confirming the source window.
""",
"log.md": f"# Analytics OKF log\n\n- {TS}: Initial bundle.\n",
}
def validate(path: str, meta: dict) -> None:
if os.path.basename(path) not in EXEMPT and not meta.get("type"):
raise ValueError(f"{path}: missing required OKF field: type")
def object_metadata(path: str, meta: dict) -> dict[str, str]:
tags = meta.get("tags", [])
tags = [tags] if isinstance(tags, str) else tags
return {
"format": "okf-v0.1",
"okf_bundle": BUNDLE,
"okf_path": path,
"okf_type": str(meta.get("type", "reserved")),
"okf_title": str(meta.get("title", os.path.splitext(os.path.basename(path))[0])),
"okf_tags": ",".join(str(t) for t in tags),
"owner": str(meta.get("owner", "unknown")),
"sensitivity": str(meta.get("sensitivity", "unspecified")),
}
def main() -> None:
repo = lakefs.repository(REPO_ID).create(
storage_namespace=STORAGE_NAMESPACE, exist_ok=True)
parsed = {path: frontmatter.loads(text) for path, text in DOCS.items()}
for path, doc in parsed.items():
validate(path, doc.metadata)
release = os.environ.get(
"OKF_RELEASE_TAG",
f"{BUNDLE}-okf-{datetime.now(timezone.utc):%Y%m%d%H%M%S}")
with repo.branch("main").transact(
commit_message=f"Publish {BUNDLE} OKF bundle",
commit_metadata={"format": "okf-v0.1", "bundle": BUNDLE, "published_at": TS},
tag=release) as tx:
for path, text in DOCS.items():
tx.object(f"{PREFIX}/{path}").upload(
data=text,
content_type="text/markdown; charset=utf-8",
metadata=object_metadata(path, parsed[path].metadata))
commit = repo.ref(release).get_commit()
print(f"published {REPO_ID}/{release} (commit: {commit})")
if __name__ == "__main__":
main()Run it:
python publish_okf.pyYou now have a tagged OKF release:okf-knowledge/analytics-okf-20260706143000/okf/analytics
That tag is the contract between the OKF publisher and the agent.
Mount the OKF release
For retrieval, we’ll use lakeFS Mount (Everest). Since retrieval is our critical path (and is the token-heavy part!)This method allows us to interact with our versioned repository without wasting a single token: no MCP servers or skills to load: our repository will appear as a local directory. Yes, even if it is petabytes in size – and agents are amazingly capable at interacting with local files and folders. (lakeFS Mount is available on lakeFS Cloud and Enterprise, contact us for access to the Everest binary.)
The agent should read files from a mounted release, not from a mutable local directory.
Mount that commit:
export OKF_REPO="okf-knowledge"
export OKF_REF="analytics-okf-20260706143000"
export OKF_PREFIX="okf/analytics"
everest mount \
"lakefs://${OKF_REPO}/${OKF_REF}/${OKF_PREFIX}/" \
./mnt/analytics-okfCheck the mounted bundle:
find ./mnt/analytics-okf -name "*.md"
cat ./mnt/analytics-okf/metrics/weekly_active_users.mdMount resolves a branch or tag to a commit in read-only mode. Passing an immutable tag makes the agent run easier to reproduce.
Use the mounted OKF bundle from an agent
Deep Agents is a good match for OKF because it has built-in filesystem tools such as ls, read_file, glob, and grep. It is built on LangGraph and uses the same model integrations as LangChain.
lakeFS Mount gives the agent a versioned filesystem. Deep Agents gives it the retrieval loop.
# okf_agent.py
from __future__ import annotations
import os
from pathlib import Path
from deepagents import FilesystemPermission, create_deep_agent
from deepagents.backends import CompositeBackend, FilesystemBackend, StateBackend
OKF_MOUNT_DIR = Path(os.environ["OKF_MOUNT_DIR"]).resolve()
system_prompt = f"""
You are an analytics knowledge agent.
Answer only from the mounted OKF bundle under /okf.
lakeFS source:
- repo: {os.environ["OKF_REPO"]}
- release ref: {os.environ["OKF_REF"]}
- commit: {os.environ["OKF_COMMIT"]}
- mount: {OKF_MOUNT_DIR}
Use filesystem tools to find Markdown files, read relevant concepts, and follow
Markdown links. Cite OKF paths in the final answer. If the bundle does not
contain enough evidence, say so.
"""
agent = create_deep_agent(
model=os.environ["OKF_AGENT_MODEL"],
system_prompt=system_prompt,
backend=CompositeBackend(
default=StateBackend(),
routes={
"/okf/": FilesystemBackend(
root_dir=str(OKF_MOUNT_DIR), virtual_mode=True),
},
),
permissions=[
FilesystemPermission(
operations=["read"],
paths=["/okf/**"],
mode="allow"),
FilesystemPermission(
operations=["write"],
paths=["/okf/**"],
mode="deny"),
FilesystemPermission(
operations=["read"],
paths=["/**"],
mode="deny"),
],
)
question = (
"How should I compute weekly active users? "
"Which source table and operational playbook should I check?")
result = agent.invoke({"messages": [{"role": "user", "content": question}]})
print(result["messages"][-1].content)Run it:
export OKF_MOUNT_DIR="./mnt/analytics-okf"
export OKF_AGENT_MODEL="openai:<your-tool-calling-model>"
python okf_agent.pyA good answer should cite files like:/okf/metrics/weekly_active_users.md<br>/okf/tables/events.md<br>/okf/playbooks/freshness.md
Unmount when finished:
everest umount ./mnt/analytics-okfWhere Enterprise matters
lakeFS is the control plane for AI-ready data, built on a highly scalable data version control architecture. That architecture is what makes the capabilities below possible.
Mount lets agents, notebooks, scripts, and services read a versioned OKF release through normal filesystem APIs.
RBAC, SSO and audit logs help govern sensitive internal knowledge: metrics, schemas, runbooks, lineage, owners, and incident procedures.
Metadata Search exposes object metadata as versioned Apache Iceberg tables. Since we uploaded each OKF file with metadata such as okf_type, owner, and sensitivity, platform teams can query the knowledge corpus by release.
SELECT
path,
user_metadata['okf_type'] AS okf_type,
user_metadata['owner'] AS owner,
user_metadata['sensitivity'] AS sensitivity
FROM "okf-knowledge"."analytics-okf-20260706143000".system.object_metadata
WHERE user_metadata['okf_bundle'] = 'analytics';Production pattern
Use this flow as the default:
The key detail is the tag. Agents should not depend on a moving branch like main. They should read a named release or a commit ID.
OKF gives agents a useful knowledge format. lakeFS gives that knowledge a production lifecycle. Together, they make agent context reviewable, reproducible, governable, and easy to consume from normal Python tools.
To learn more about lakeFS Enterprise, visit our docs or start a free trial.
