Skip to content

Processing lanes

Knowledge Graph splits ingestion into three lanes, each scheduled independently across your entire corpus. Scheduling is priority-based, not barrier-based: the Structure lane has absolute priority, so a large upload does not wait for entity recognition or embedding before becoming keyword and outline searchable. Between Structure and Vector, the NER lane recognises named entities in each file’s prose with a self-hosted model.

Knowledge Graph ingestion pipelineA workspace file is parsed into markdown and an outline, then indexed into chunks and keyword search. Embed writes a vector for each chunk. Entity recognition extracts entities and relations. Resolve types unknown entities, merges duplicates, and adds relations. Ontology review corrects the entity and relation types.BYTESMARKDOWNPROSECHUNKSENTITIESDECISIONSVECTORSFILESource fileworkspace bytesSTRParsemarkdown · outlineSTRIndexchunks · full text · keywordskeyword searchNEREntity recognitionentity and relation extractionVECEmbedvector embeddingsRSLResolvetypes, merges, and relationsREVOntology reviewcorrects entity and relation typesLEGENDRequiredYou turn these onKeyword search starts at index

The diagram above is ingestion for one file. Parse turns the file into markdown and an outline. Index splits it into chunks and keyword search. Embed writes a vector for each chunk. Entity recognition extracts entities and relations. Resolve types unknown entities, merges duplicates, and adds relations. Ontology review corrects those types.

Under that diagram, the Edges card compares edges that came from parsing with edges that came from extraction.

A question follows a separate path. The assistant searches with full text, embeddings, and a Cypher graph hop, then fuses them by rank. Co-annotation is an extra lane on that search. See Retrieval.

Ingest pipeline and question searchA source file enters the ingestion pipeline: parse, index, and vector embeddings, then entity and relation extraction and resolve. The Edges card compares parse edges with extract edges. When you ask a question, the assistant runs Knowledge Graph search: full text, embeddings, a Cypher hop, and rank fusion. Co-annotation finds chunks that name two entities, and the span linker attaches an unnamed phrase. The response labels entities bridge, filter, or ignore.FILEREADSNAMESASKSMCPJOINSINGESTWHEN YOU ASKFILESource fileone documentINIngestion pipelineparse · index · embedentities, relations, resolvecompare parse and extract edgesONTPublished ontologyentity and relation namesYOUYouask in chatAIAssistantcalls MCP searchKGKnowledge Graphfull text · embeddingsCypher · rank fusionOPTCo-annotationchunks naming two entitiesspan linker for an unnamed phraseLEGENDAlwaysWhen two entities are namedWhat the assistant calls

Each file moves through readiness stages as the lanes complete their work. These are not a single binary “processing” flag: the workspace UI shows partial progress so you know exactly what is queryable right now.

Stage What becomes queryable Blocks on
PARSE Markdown text and a heading outline skeleton Document parsing (anydoc, Docling OCR fallback, JSON or JSONL, or plain text)
INDEX (searchable_partial) Chunk full-text search, the outline tree, and ontology keyword mentions A published ontology, for keyword matching
NER (searchable_partial) Typed entities and mentions from prose, and sentence co-occurrence The NER service; runs when the namespace’s entity recognition is on
RESOLVE (searchable_partial) Types for unknown entities, merges of duplicate pairs, relations for strong co-occurrences, verdicts on mined candidate types TypeSafe System One, when the namespace’s entity resolution is on; metered against the namespace budget
EMBED (searchable_embedded) Vector hybrid search, and optional entity similarity candidates The platform embedding profile on Models → Embeddings

You configure each stage per namespace, with folder-level overrides for parse and index and for embeddings.

Setting Values Meaning
Parse and index On / Off On parses and indexes each file when it arrives. Off leaves the file until you reprocess it
Embeddings On / Off On embeds after indexing. Off embeds only when you choose Embed now on a file
Entity recognition (NER) On / Off On recognises people, organisations, places and similar entities in each file’s prose with the entity recognition service installed on your platform. Off skips that step
Entity resolution (System One) On / Off After NER, hands each file’s open questions to TypeSafe System One for calibrated closed-set decisions; only confident answers are written, the rest waits for the ontology review; charged to the namespace budget
Ontology mining On / Off On proposes types from table headers, headings and acronyms while files are indexed. Off does not
Ontology agent Any eligible agent The agent that reviews and corrects this namespace’s ontology. One choice per namespace
Budget cap Dollar cap A hard stop on language-model spend for this namespace’s ontology review

Entities of the same type whose names are nearly identical are merged automatically when the Vector lane embeds them, so duplicates do not accumulate as a corpus grows.

You can stand up a namespace and load its ontology before any file exists, or upload data first and let algorithmic mining and discovery propose a schema afterward. Neither order is privileged: the Structure lane gazetteer-matches from whichever file arrives once a published ontology exists.

Fontana shows a forecast before you commit a corpus, and a live ledger while ingest runs:

  • Pre-parse estimate (at upload time, before any cost): a token forecast per stage from file bytes, format, and page count, calibrated from your workspace’s own historical ingest costs.
  • Post-parse refinement (after PARSE): once markdown exists, chunk and token counts are exact, so embedding forecasts sharpen into a near-exact estimate.
  • Budget governance: a namespace budget cap hard-stops language-model spend when the running ledger crosses it. Ingestion itself is unaffected — files continue to parse, index, recognise entities and embed.

Knowledge Graph never rebuilds a whole namespace for one file change:

  • A content-hash check skips PARSE entirely when a file’s bytes have not changed.
  • Reprocessing one file replaces only that file’s lexical and structural subgraph (its chunks, outline, and mentions); shared entities stay in place unless their last evidence disappears.
  • If PARSE and INDEX succeed but EMBED fails, the file stays searchable_partial and retries EMBED without re-parsing.
  • Publishing a new ontology re-runs keyword matching for affected files. Full-text, outline, and vector search remain queryable throughout.