Verifying AI-generated ontologies: Auto Ontology vs Palantir

Verifying AI-generated ontologies: Auto Ontology vs Palantir

Building an ontology with an LLM is now automated. What remains is checking, separately, whether the data is right and whether the meaning is right

We read NVIDIA Auto Ontology's source to see how it generates an ontology with an LLM and how agents answer from it, and compare it with Palantir's ontology. NVIDIA Auto Ontology is open source software that reads a database's schema, keys, query history and sample values, uses an LLM to generate concepts, column mappings, relationships and SQL formulas, and exposes them to agents over MCP, CLI and API.

By · Fred Kim
OntologyAI agentsSemantic layerText-to-SQLMCPPalantir

NVIDIA Auto Ontology generates a draft ontology from database metadata with an LLM, but certifying a definition does not change what the agent retrieves (architecture doc)[2]. Building an ontology is now automated. What remains is checking, separately, whether the data is right and whether the meaning is right. This post traces where those two checks part ways, using the source, docs and evaluation data.

Our sources are the README and architecture doc of the NVIDIA/auto-ontology repository (commit a630ebd, 2026-11-16), the evaluation questions and sample data in the same team's auto-ontology-eval (commit 1b97494), and Palantir's Foundry documentation[1][2][3][4]. We did not install or run the tool. We read its docs, code and sample data. The repository was published in April 2026 and changes almost daily, so later behavior may differ. We build a product that measures how brands appear in AI answers, so we have a stake in how AI links concepts to evidence.

What is an ontology and why do AI agents need one

An ontology is a shared map from the business concepts people ask about to the data behind them. Description logic splits definitions of concepts and relationships (the TBox) from actual entities and facts (the ABox) (Wikipedia: Abox)[6]. In enterprise data, the TBox is concepts such as customer, order and revenue and how they relate, and the ABox is the rows in countless tables. Connecting the two, deciding which column in which table is which concept, is most of the work of building an ontology.

Agents need this map for a plain reason. If an agent guesses meaning from table and column names every time, the same question yields different joins and formulas on different runs. The Auto Ontology README lists six questions an ontology has to answer[1].

QuestionElementIn Auto Ontology
What does it mean?ConceptA term such as Customer or Invoice Line, with description and synonyms
Where is it?MappingColumn attributes tying each term to tables and columns
What's connected?RelationshipDeclared keys plus inferred joins confirmed against data
How do I calculate it?MetricReviewed SQL stored under a term
Where does it come from?LineageLinks to source tables and columns, SQL returned with each answer
Should I use it?CertificationExperts certify names, descriptions and attributes

Table 1. Six questions an ontology answers. Source: NVIDIA Auto Ontology README[1].

The README also says when you do not need one: with one well-modeled source and a fixed set of questions, direct SQL may be enough[1]. An ontology pays off when questions vary, sources multiply, definitions conflict across sources, and many teams and agents must reuse the same meaning.

How to generate an ontology automatically with an LLM

Auto Ontology builds its draft by collecting facts with SQL, writing meaning with an LLM, and confirming again with SQL. The architecture doc marks each step as a model call or as deterministic SQL and rules[2].

[Source DB] schema, keys, views, query history
    |  SQL extract -> (optional) PII tagging -> embed tables and columns
    v
[Per table] sample 1,000 rows, distinct ratios, date formats    <- SQL, rules
    |  FK candidates, junction tables, column descriptions, terms <- LLM
    v
[Across tables] group schema copies into term families          <- rules
    |  link FKs to terms -> store only if a real join returns a row <- embed + LLM + SQL
    |  suggest SQL formulas from query history, describe attributes <- LLM
    v
[Output] Postgres + pgvector, Ossie YAML export
    |
    v
[Certify] test with real questions -> people refine and certify

The per-table pass samples up to 1,000 rows, marks a column unique when at least 95% of sampled values differ, and then has an LLM write foreign key candidates, column descriptions and term names (architecture doc, Per-table pass)[2]. The rule worth noting in the cross-table pass is the storage condition for inferred links. Even a link the LLM picks is stored only if joining the two columns returns at least one row. The step that suggests SQL formulas from query history takes up to 10 expressions per term, and the doc states that usage ranking is not active yet, so these are the first 10 found, not the most used[2].

Models default to NVIDIA Nemotron variants but can be swapped for any OpenAI-compatible endpoint, and the tool connects to ten engines including PostgreSQL, Snowflake and Databricks[1]. Output can be imported and exported as Apache Ossie YAML, which opens a path to share definitions with other semantic model tools.

How AI agents answer from the ontology

Answering is a loop in which a reasoning model picks one tool at a time. The model starts with no schema, uses retrieve to find terms, columns, formulas and join paths in the semantic layer, run_sql to inspect values, and submit to return the final query (architecture doc, Answer a question)[2]. Each question gets 24 tool calls and 10 SQL runs by default.

The loop is model-chosen, but the checks right before execution are deterministic. run_sql and submit compare table and column names against the catalog, reject unknown names, and block tables no retrieve result showed[2]. There is no separate SQL judge model. If no query passes within budget, the agent reports that the question cannot be answered from this data.

Agents connected over MCP get one more tool, check_answerable. It scores the share of entities in a question that have a candidate in the ontology, and in the README example a question about ad spend, which the data lacks, scores 0.5 and the agent asks back[1]. Building a path to say "I don't know" instead of inventing an answer is the practical part of this design. The chat agent, however, does not call this check, and per the doc it does not ask clarifying questions[2].

How to verify an AI-generated ontology

Verification splits into a data layer and a meaning layer, and Auto Ontology automates the first. Machines check whether a join returns rows, whether names exist in the catalog, and whether a query runs. Whether that link is what the asker meant cannot be confirmed from data alone.

The README's first example shows the gap. For "revenue for the sushi roll USB drive in Pennsylvania in June 2015", the agent summed extendedprice on invoice lines and reached the state through the customer's delivery city (README Quickstart)[1]. The query ran and returned $588.80. Yet the evaluation schema in the same project shows the customer table also holds a postal city (PostalCityID) and a bill-to customer (BillToCustomerID) column apart from delivery city (DeliveryCityID) (auto-ontology-eval DDL)[3]. When we checked 5,000 invoice lines in the sample data, extendedprice equaled quantity times unit price plus tax on every row. The example's revenue is tax-inclusive revenue by delivery address.

The evaluation data points at the same spot. Among 41 questions, the reference SQL for an average order value question computes quantity times unit price without tax, and a question on cities with the longest delivery delays carries the note "Uses DELIVERYCITYID instead of city name, which may not match the question's intent" (auto-ontology-eval)[3]. Even within one project, the definitions of amount and location shift by question. That is what people have to certify.

LayerWhat Auto Ontology checks automaticallyWhat is left to people
ExistenceCatalog lookup, rejects unknown tables and columnsNothing
ConnectionStores an inferred join only if a real join returns rowsChoosing the business-correct join among several
CalculationSuggests SQL formulas from query historyMetric definitions such as tax, discount and refund handling
MeaningLLM writes column descriptions and term namesDeciding the basis for region, customer and period
EffectStores a certification markDeciding whether uncertified definitions may feed answers

Table 2. Layers of ontology verification and what remains. Source: our summary of the architecture doc and evaluation data[2][3].

The last row matters most. The architecture doc says certifying marks an element as reviewed in the UI and search results but does not change how the agent retrieves (architecture doc, Output and certification)[2]. When we searched the answer retrieval code (auto_ontology/retrieval), nothing read the certification fields. Human edits to terms and descriptions survive the next compilation, but the doc notes one exception: every compilation re-selects which foreign key each join uses, so a link a person turned off does not stay off. Uncertified drafts and certified definitions enter answers with the same weight.

We chose the opposite for our own company facts. A number with no record of human confirmation is judged "not for external use" at lookup time, so it cannot go into a post. Which choice is right depends on use. For exploratory analysis, using the draft right away is faster. Where numbers leave the building, as in reports or customer replies, treating certification as a gate rather than a label is safer.

NVIDIA Auto Ontology vs Palantir ontology

Auto Ontology defines the path for reading data, while Palantir's ontology also defines the path for changing it. Palantir's docs describe the ontology as semantic elements (objects, properties, links) plus kinetic elements (actions, functions, dynamic security) (Palantir Ontology Overview)[4]. An action is a single transaction that changes one or more objects, and an action type defines the input form, validation of who may run it, and side effects such as notifications (Palantir Action types)[5]. The docs' example is an "Assign Employee" action that only HR staff can take.

AspectNVIDIA Auto OntologyPalantir Foundry ontology
Starting pointDraft generated from existing DB metadataData sources mapped to objects, properties, links
Core questionWhich SQL answers this questionWhich change executes this decision
Agent surface9 MCP tools, all read-onlyThe same action logic called from every user app
WritesPeople edit ontology definitionsActions change objects under permissions and validation
Human approvalCertification mark, no effect on retrievalAction permissions and submission criteria
PortabilityOpen source (Apache 2.0), Ossie YAMLRuns inside the platform

Table 3. Scope of the two ontologies. Sources: Auto Ontology README, architecture doc, Palantir docs[1][2][4][5].

The two sit at different layers more than they compete. Auto Ontology lowers the cost of building an ontology, and Palantir runs operations on top of one. Wiring auto-generated definitions straight into a write path would let definitions whose meaning was never checked trigger real changes. That is why human certification should stand as a gate where the read path meets the write path.

What auto-generated ontologies mean for AI search and brands

AI search engines solve the same problem for brands. Users ask in their own terms, such as "a gentle sunscreen for sensitive skin", and the engine finds matching concepts and evidence in scattered web pages and links them to brands. Just as revenue definitions conflict across enterprise sources, a brand's official site, retailers and reviews state price, ingredients and certifications differently.

The order Auto Ontology follows carries over to brands. First check by machine that facts exist and connect, then let people decide whether those connections match what the asker meant. Our post on brand entity SEO for AI search covers consistent entity information as the first step, and the fake brand entity coherence study shows what AI believes when connections break. We covered a format that records sources and verification status in an agent knowledge base in our Open Knowledge Format post, and how MCP attaches tools to agents in our MCP post.

Limits

This post reflects reading docs, code and sample data. We did not install Auto Ontology or measure its answer quality. The BEAVER #1 and BIRD-Interact #2 badges at the top of the README are placed by the repository itself, and we did not check each leaderboard's conditions or rankings[1]. The examples and evaluation use English column names and Wide World Importers, a US sample database, so we cannot say whether quality holds for non-English column names, abbreviation-heavy legacy tables or questions in other languages. The README states that enabling predictions uploads connected tables to the external KumoRFM service, so a separate data export review is needed before adoption[1]. Because the repository changes almost daily, behaviors described here, such as the effect of certification, are as of the commits listed above.

Frequently asked questions

How is an ontology different from a database schema?

A schema defines how tables and columns are stored. An ontology defines what those columns mean for the business. The same sales.customers table does not say whether a customer's region is the delivery address or the postal address. An ontology records such decisions as concepts, relationships and formulas so that people and agents read the data the same way.

Does Auto Ontology remove the need for data engineers?

It cuts drafting, not decisions. The tool samples tables and uses an LLM to write descriptions, terms and relationships, but business definitions such as whether revenue includes tax or whether region means the delivery address cannot be settled from data alone. The human job moves from writing definitions to testing generated ones with real questions and certifying them.

Can it be used on non-English enterprise data right away?

The published examples all use English column names and US sample data. Whether quality holds for non-English column names, abbreviated legacy columns or questions in other languages is not shown in public material. Plan a separate step to test the draft with dozens of real questions before rollout.

References

  1. [1]NVIDIA, "Auto Ontology" README, GitHub, as of commit a630ebd (2026-11-16)
  2. [2]NVIDIA, "Auto-Ontology architecture", docs/architecture.md, as of commit a630ebd
  3. [3]NVIDIA, "auto-ontology-eval" (Wide World Importers evaluation questions and sample data), as of commit 1b97494
  4. [4]Palantir, "Ontology: Overview", Foundry documentation
  5. [5]Palantir, "Action types: Overview", Foundry documentation
  6. [6]Wikipedia, "Abox" (the TBox and ABox split in description logic)

Summary

  • NVIDIA Auto Ontology is open source software that reads a database's schema, keys, query history and sample values, uses an LLM to generate concepts, column mappings, relationships and SQL formulas, and exposes them to agents over MCP, CLI and API.
  • An inferred relationship is stored only if joining the two columns returns a row, and the answering agent rejects names missing from the catalog and tables retrieval never showed. Data-level checks are tight.
  • Meaning is left to people. The README's Pennsylvania revenue example sums a tax-inclusive amount by delivery city, while the customer table also holds separate postal city and bill-to customer columns.
  • Certification is shown in the UI and search results but does not change how the agent retrieves. Uncertified drafts weigh the same as certified definitions in answers.
  • Palantir's ontology adds a write path on top of concepts and relationships: actions that change data under permissions and validation rules. Auto Ontology's MCP tools are read-only.

Check this topic against your own brand

TRAIL Search measures how ChatGPT and Perplexity answer your customers' questions and finds and fixes where the brand is missing, turning this post into a diagnosis. Start with 10 questions, no card required.

The feature closest to this post is AI slides.

More posts

Keep reading on this topic

This post is part of the Trend category, which collects all 15 posts on the topic. See all posts in Trend