Skip to content

Introducing the Open Knowledge Format

Authors: Sam McVeety (Tech Lead, Data Analytics, Engineering, Data Cloud, Google Cloud) and Amir Hormati (Tech Lead, BigQuery, Engineering, Data Cloud, Google Cloud) Date: Jun 12, 2026 Original: https://cloud.google.com/blog/products/data-analytics/how-the-open-knowledge-format-can-improve-data-sharing Captured: 2026-07-20

Provenance note: this is a structured capture, not a verbatim copy. The blog post is Google Cloud's copyrighted text; what follows is a factual extraction — the spec's fields, the example bundle layout, the reference implementations, and the named products — plus a small number of short quotes. Consult the original URL for the full prose.

What OKF is

An open specification (v0.1) from Google Cloud that standardises how metadata, context, and curated knowledge are represented for AI systems to consume.

The whole format, stated three ways:

  • Just markdown — readable in any editor, renderable on GitHub, indexable by search tools.
  • Just files — shippable as a tarball, hostable in a git repo, mountable on a filesystem.
  • Just YAML frontmatter — a small set of queryable structured fields on top.

The problem it addresses

Organisational knowledge is scattered across metadata catalogs with proprietary APIs, wikis, shared drives, code comments and docstrings, and senior engineers' heads. Per the post: "Every vendor offers its own catalog, its own SDK, its own knowledge-graph schema, and none of the knowledge is easily portable." An agent answering a question has to assemble it from mutually incompatible surfaces.

Frontmatter schema

---
type: BigQuery Table
title: Orders
description: One row per completed customer order.
resource: https://console.cloud.google.com/bigquery?p=acme&d=sales&t=orders
tags: [sales, revenue]
timestamp: 2026-05-28T14:30:00Z
---
  • Required: type — the only mandatory field in the spec.
  • Standard: title, description, resource, tags, timestamp.
  • Extensible: producers may add custom fields. The post's framing — "The spec defines the interoperability surface, not the content model."

Bundle layout

Directories represent interconnected concepts; each concept is one document with frontmatter plus a markdown body. Markdown links between documents form the graph.

sales/
├── index.md
├── datasets/
│   ├── index.md
│   └── orders_db.md
├── tables/
│   ├── index.md
│   ├── orders.md
│   └── customers.md
└── metrics/
    ├── index.md
    └── weekly_active_users.md

Design principles

  1. Minimally opinionated — only type is required; everything else is flexible.
  2. Producer/consumer independence — a bundle hand-authored by a human can be consumed by an AI agent; one generated by a metadata export pipeline can be browsed in a visualizer; one synthesised by one LLM can be queried by another.
  3. Format, not platform — not tied to a specific cloud, database, or vendor.

Why markdown, for agents

The post leans on Karpathy's argument for LLM-maintained wikis: LLMs "don't get bored, don't forget to update a cross-reference, and can touch 15 files in one pass." The consumption model is a shared markdown library that grows more useful over time, which agents can both read and update autonomously — no SDK involved.

It positions OKF as formalising a pattern people already run ad hoc: Obsidian vaults wired to coding agents, Notion, and Hugo's markdown + frontmatter model.

Reference implementations shipped

  • Enrichment agent — walks BigQuery datasets and generates OKF documents with citations and join paths.
  • Static HTML visualizer — renders a bundle as an interactive graph view, no backend required.
  • Sample bundles — GA4 e-commerce, Stack Overflow, and Bitcoin datasets as conformant examples.

Spec and code: https://github.com/GoogleCloudPlatform/knowledge-catalog/tree/main/okf Knowledge Catalog integration demo: https://github.com/GoogleCloudPlatform/knowledge-catalog/tree/main/toolbox/mdcode/demo

Products named

Product Role in the post
BigQuery Primary data source the reference enrichment agent walks.
Google Cloud Knowledge Catalog Updated to ingest OKF and serve it to agents — the consumer side.
Gemini Enterprise Business Edition Sidebar CTA only; no integration detail in the body.
Obsidian / Notion / Hugo Cited as prior art for the markdown-plus-frontmatter pattern.

Notably absent: no mention of Dataplex, of semantic layers, of data contracts, or of MCP.

Call to action

The spec is open source and invites alternative producers for other data sources, custom consumers (viewers, search indexes, agents), and issues/PRs/extension proposals.