Three Broken Grammars: Teaching CodeGraph to Read My Infrastructure
CodeGraph indexes a codebase with tree-sitter and hands the resulting symbol graph to an agent over MCP, so questions like “what calls this” get answered from an index instead of from twenty file reads. It works well on my application repos. It was completely blind to the infrastructure they run on.
Two repos sat at exactly zero nodes. JeakylJenkins holds 890 lines of Groovy across seven shared-library files plus five pipeline templates; JeakylK8s holds 65 YAML files, split between the Kubernetes manifests behind the cluster rebuild and an Ansible bootstrap. Both had been indexed. Both produced nothing. My own workspace notes said “empty by design” and told me to fall back to grep, which is the sort of documentation that starts as an accurate observation and quietly becomes an excuse.
The cause was three separate gaps, and fixing them took a day of mostly unglamorous discovery. The grammars I needed were broken in three different ways, a scan gate rejected half the files before any grammar saw them, and one of my own tests found a secret leak that had been there before I started.
The gate that runs before the grammar
Jenkinsfile has no extension. Neither does Dockerfile.
CodeGraph decides whether to index a file with a function called isSourceFile, and its third line is if (dot < 0) return false. No dot, no extension, no indexing. This is why convertx and JeakylLogs were indexed perfectly well while their Jenkinsfile and Dockerfile sat invisible in the same directory; the language detection would have handled them fine if anything had ever asked it to.
There’s no filename table to add an entry to. The established pattern is a predicate function wired into two call sites, which is how Play Framework’s extensionless conf/routes gets in. Miss the second call site and you get the failure that wastes an afternoon: detection reports the right language, and the scanner still never hands it a file.
Worth noting how the suffixed spellings behave, because they’re the reason a basename check is the only thing that works. Jenkinsfile.release has a last-dot extension of .release; python-uv.Jenkinsfile has one of .Jenkinsfile. Neither belongs in a general extension map.
Every grammar I needed was broken
CodeGraph ships grammars as WebAssembly, mostly from the tree-sitter-wasms package. I needed Groovy, YAML and bash. The package has two of those, which felt like a good start for about four minutes.
The repo has a health-check script for exactly this reason, and its comments explain the failure mode it was written for: an old-ABI grammar can corrupt the shared WASM heap and silently drop nodes on every file after the first. I ran it against YAML and got something worse than degraded output.
TypeError: resolved is not a function
at stubs.<computed> (node_modules/web-tree-sitter/tree-sitter.js:2947:24)
The shipped YAML grammar is ABI 13 and throws on parse. Bash was ABI 14 and looked healthy against one script, then killed the Node process outright on another. I narrowed that to a single construct in a build script: ${TARGET:+ (target $TARGET)}, an alternate-value expansion containing unquoted parentheses. Not a bad parse; a hard crash of the runtime, mid-index.
Groovy wasn’t in the package at all. The one published wasm, tree-sitter-groovy 0.1.2, parses neither field modifiers nor constructors nor def, and on a real Jenkins pipeline it starts erroring at line 1. A grammar that can’t handle private def script isn’t a Groovy grammar in any useful sense.
So all three had to be vendored. YAML and bash have prebuilt ABI 15 wasms published under different package names, which is a five-minute fix once you know to look. Groovy needed building from source, and the two maintained forks ship a grammar but no wasm, so that meant tree-sitter build --wasm, which needs emscripten or Docker.
Three things went wrong in that build, in ascending order of interest. Docker Desktop wasn’t running. The npm tarball omits the tree-sitter.json the 0.25 CLI requires, so I hand-wrote one. And a local shell shim on my machine rewrites npx to npm, which silently ate the -p flag and produced Unknown command: "tree-sitter-cli@0.25.10" twice before I stopped blaming the CLI and looked at my own tooling.
The result was worth it. Built from @bootswithdefer/tree-sitter-groovy 0.2.0 with tree-sitter-cli 0.25.10, the grammar comes out at ABI 15 and parses the shared library, the pipeline templates and the Jenkinsfiles with hasError=false on every one.
The vendored bash still can’t parse that one exotic expansion. It produces a localised error node and recovers, which is the ordinary tree-sitter behaviour and entirely liveable; I mention it because “we fixed the grammar” would be a nicer sentence than the true one.
Indexing YAML without ruining everyone else’s graph
Here’s the design question that took longer to settle than any of the code. YAML is data. A Kubernetes manifest describes a system’s structure and genuinely belongs in a graph; a Jekyll _config.yml is a bag of settings and does not. Turning on YAML symbol extraction for everybody would flood every existing user’s index with config keys pretending to be symbols.
So the extractor is gated on shape, not on file type. YAML stays file-level-only by default, exactly as before, and only three recognised shapes divert to the new extractor: a document with apiVersion and kind, an Ansible role directory or playbook, a Compose file with a services: root. Everything else falls through untouched. Half the tests I wrote for this are negative cases, asserting that a CI workflow and an app config still emit precisely nothing.
Detection by shape has one trap I walked into. My first playbook check looked for - hosts: at the start of a sequence item, which is a perfectly reasonable reading of the format and matched none of my playbooks, because they all lead with - name: and put hosts: on the following line. The fix is to match the sequence and the key independently.
I also couldn’t invent new node kinds to model any of this. The NodeKind list is part of the native Rust kernel’s wire contract, verified by byte equality when the binary loads, so adding to it means shipping a new kernel across six platform triples. Everything reuses existing kinds instead, following what the Terraform extractor already does: a Kubernetes resource is a class, an Ansible role is a module, a task is a function, a pipeline stage is a component. Slightly odd names, no rebuild, and the queries read fine.
The convention that makes a shared library work
The most valuable edge in the whole exercise is also the one static parsing can never find on its own.
Jenkins shared libraries work by filename. A file at vars/jekyllAws.groovy defines a global pipeline step called jekyllAws, and that name appears nowhere in the file’s contents. That indirection is the entire point of putting the build logic in a shared library, and it’s also what makes the code invisible: my blog’s own Jenkinsfile is four meaningful lines that call jekyllAws(siteDir: ...), and no amount of grammar work will connect that call to the 102 lines of Groovy that actually build and deploy this site.
That’s what CodeGraph’s framework resolvers exist for, so the Jenkins one synthesises a function node per vars/ file and resolves bare step calls to it. Now callers of jekyllAws returns the pipelines using it.
One limit I can’t design away: indexes are per repo. The templates inside JeakylJenkins resolve; my blog’s call site can’t reach across the repo boundary to the library, because the two are separate git repos with separate indexes. I’d rather say that plainly than pretend the feature is complete.
The test that found a leak
I wrote a test asserting that a sops-encrypted Kubernetes Secret gets indexed as a resource while its values never reach an agent. It failed, and it failed on code I hadn’t touched.
CodeGraph already had this protection. It withholds config file source so that a Spring application.yml password can’t be dumped into an agent’s context, and the check is kind === 'constant' && language is yaml or properties. My Kubernetes resources are class nodes. They walked straight past a kind-based filter and codegraph_node with includeCode returned the manifest body verbatim, stringData and all.
The tempting fix is to widen the existing predicate to any YAML node. That would also have broken the feature I’d just built, because the same predicate is used by the name matcher to keep config keys out of symbol resolution; widening it there would have excluded every Kubernetes resource from matching and killed the resource-to-resource references. So it needed a second, separate predicate for the source-dump path, and both read paths gated on it. There were two, and I’d only patched the fallback one on the first attempt.
The lesson isn’t that the original protection was wrong. It was correct for the node kinds that existed when it was written. Adding a new shape of node to a file type that had a security rule attached to it silently moved that rule out from under the thing it protected, and only an assertion written in the same session caught it.
Where it landed
JeakylJenkins went from 0 to 104 nodes: six shared-library steps, the Groovy classes and methods, pipeline stages, the shell scripts, both Dockerfiles, and the Compose services. JeakylK8s went from 0 to 183, covering 33 Kubernetes resources with their namespace and kind, five Ansible roles, and 46 tasks and handlers wired by notify:.
The control mattered more to me than either. Re-indexing convertx moved it from 313 nodes to 325, and the twelve new ones are exactly its Jenkinsfile, its Dockerfile and two Compose services. Nothing else shifted, which is what I needed to see before believing the YAML gate holds.
Asking about the logs stack now returns the Deployment, the StatefulSet, and the HTTPRoute pointing at the Service, without opening a file. The Secret references come back unresolved, and correctly so: that repo’s codegraph.json excludes *secret*.yaml from indexing entirely, so there’s nothing for them to point at. The reference is real, the target is deliberately absent, and I’d rather see the dangling edge than have it quietly dropped.
One repo stayed at zero. NUC-Rog contains a Markdown file, a .gitignore and two SSH keys, so there is genuinely nothing to index; Ansible support will pick it up when playbooks land there and not a moment sooner. Its entry in my notes still says the index is empty. That one’s still true.