Build the graph in validated stages
The graph is treated as an engineered representation, not as an automatic LLM by-product. Each intermediate stage is inspected before the next one is trusted.
Let the corpus suggest the ontology, then review it manually
Candidate concepts were collected from the policy corpus, embedded with Nomic, projected with PCA, clustered with HDBSCAN, and manually reviewed into a domain-specific schema.
PCA view of candidate concepts
Cluster-size summary
Reviewed entity schema · 25 types
Extract candidate spans with a deliberately low validation floor
GLiNER receives the 25 reviewed labels and the original chunks. Each stored prediction preserves the exact span, canonical type, confidence, and character offsets.
Current extraction metadata
| Documents | 5 |
|---|---|
| Chunks | 20 |
| Chunk size / overlap | 700 / 120 |
| Schema size | 25 types |
| Stored detections | 361 |
| Unique entity/type pairs | 198 |
| Extraction floor | 0.20 |
| Confidence range | 0.202 – 0.987 |
| Mean confidence | 0.586 |
Threshold experiment
The 0.20 threshold is an extraction floor for validation, not the final acceptance threshold. Lower-confidence candidates are intentionally retained so they can be inspected instead of discarded upstream.
Detections by entity type
Detections by source policy
Counts are mention-level detections in the current ≥0.20 JSON and include repeated entities across chunks/context.
Inspect every entity in its original context.
The standalone viewer contains the entire chunked corpus and allows filtering by source document, entity type, and confidence. It is used to check coverage, span boundaries, label quality, and the effect of moving the confidence threshold before relations are trusted.
Find relation evidence before naming graph edges.
The first relation pass stays grounded in the policy text: detected entities are paired inside each sentence, the exact text between them is retained as evidence, and compact adjacent pairs are prioritized before clustering or semantic normalization.
Initial candidate finding
For each detected entity pair, the relation-candidate record keeps the source sentence, left and right entity spans and types, the exact text between them, token gap, number of intervening entities, adjacency flag, and pair confidence. This produced 372 unique candidates.
The first precision-oriented view keeps pairs with no intervening detected entity. That leaves 161 adjacent pairs (43.3% of the candidate pool), and 159 of 161 of those have eight or fewer between-text tokens. The relation viewer exposes the full pool while defaulting to the compact adjacent set.
What the evidence looks like
The between-text already exposes recurring typed structures that can later be consolidated into a smaller relation vocabulary. Examples from the current corpus include:
These are evidence patterns, not final Neo4j relationship labels. Pair confidence is inherited from entity detection; it is not yet a learned relation-confidence score.