C-xrefactory - a C/Yacc refactoring and code browsing tool - Design Documentation

A user can navigate all references of a symbol, limited to the semantic scope of that symbol, by "Goto Definition" and then navigate using "Next/previous Reference". This is a fast way to inspect where a symbol is used.

It is also possible to search for definitions or symbols containing a string, including possible wildcards. This will present various forms of "menues" where filters can be applied and a found symbol be inspected.

As c-xrefactory have information about symbols and their semantic scope, it can also provide semantically informed completions and suggestions.

In his book "Refactoring" Martin Fowler describes a large number of refactorings, changes to source code that does not change the behaviour, but improves it structure and readability. For each refactoring they describe step by step which edits to make to apply the refactoring manually.

The natural next step was of course to attempt to automate this in editors or IDEs, which started to happen.

In an article from 2001 Martin pronounces Xref, the ancestor to c-xrefactory, to be the first tool to cross "Refactorings Rubicon", being able to extract a function semantically correct.

The term "automated" means that some software can examine the source code and quickly and safely modify it using patterns from the list of possible refactorings, without user interaction. Many refactorings in the book, and on the website, are applicable mostly for OO-languages, but many also apply to C. c-xrefactory can perform some of them. More are considered for implementation.

Using these automated refactorings it is much easier and safer to continuously maintain and improve the quality of any code base.

The LSP protocol was designed for single-shot, non-interactive operations. This creates constraints for `c-xrefactory’s advanced refactorings:

Interactive Refactorings: c-xrefactory’s extract/parameter operations require multi-step user input (names, positions, declarations). LSP’s `textDocument/codeAction can offer a refactoring, but has no standard way to ask the user for input while it runs.

Symbol Browsing: c-xrefactory provides interactive symbol browsers with filtering and keyboard navigation. LSP returns flat reference lists with no standard for interactive UI.

How much of this can still reach LSP editors, and what each editor would need in its own extension, is covered in Roadmap: Reaching refactorings.

scanCxFile() is the only entry point that opens a .cx file for reading. It asserts that snapshotLoadComplete (a static flag in cxfile.c set at the end of loadSnapshotFromStore()) is still false. Any later call aborts with a message pointing here.

Writes — periodic save, save on exit — are not affected; the tripwire is on the read path only. A future feature that requires post-startup disk access means this principle has to be revisited.

cxrefProgram is the core container. It does all the work when it comes to finding and reporting references to symbols, communicating refactoring requests as well as storing reference information for longer term storage and caching.

Although c-xref can be used as a command line tool, which can be handy when debugging or exploring, it is normally used in "server" mode. In server mode the communication between the editor extension and the 'cxrefProgram` container is a back-and-forth communication using a non-standard protocol over standard pipes.

The responsibilities of cxrefProgram can largely be divided into

At this point it seems like refactorings are performed as separate invocations of c-xref rather than through the server interface.

The EditorExtension container is responsible for plugging into an editor of choice and handle the user interface, buffer management and executing the refactoring edit operations.

Currently there is only one such extension supported, for Emacs, although there existed code, still available in the repo history, for an extension for jEdit which hasn’t been updated, modified or checked for a long time and no longer is a part of this project.

The References database stores crossreferencing information for symbols visible outside the module it is defined in. Information about local/static symbols are not stored but gathered by parsing that particular source file on demand.

Currently this information is stored in a somewhat cryptic, optimized text format.

This storage can be divided into multiple files, probably for faster access. Symbols are then hashed to know which of the "database" files it is stored in. As all crossreferencing information for a symbol is stored in the same "record", this allows reading only a single file when a symbol is looked up.

Full parsing (running the C parser with preprocessor, semantic actions, and reference creation) is expensive — seconds per file on complex codebases. On a cold start with thousands of CUs, parsing everything up front would block the user for minutes.

The include structure, however, can be determined by simple text scanning: find lines matching #include "…​" and resolve the paths. This is orders of magnitude faster (e.g. ffmpeg: 2619 CUs scanned in under 1 second vs. minutes for full parsing).

resolveIncludePath() tries directories in order:

This matches standard C compiler behavior for quoted includes.

The scan populates the same data structures that full parsing would:

The key insight is that the scan produces the same TypeCppInclude references that full parsing creates. Code that queries the include graph (like collectIncludersOfStaleHeader() in Entry Refresh) doesn’t care whether the references came from scanning or parsing.

The scan only captures #include "…​" (quoted includes), not #include <…​> (angle-bracket includes). This is deliberate: without the header-filtered sibling optimization (see Entry Refresh Pass 3 limitations), scanning <string.h> and similar ubiquitous system headers would cause near-full-project sibling explosions.

Full parsing does discover <…​> includes. On the first request after cold start, Pass 1 reparses the request CU with the full parser, which finds the <…​> edges. Pass 3 may then discover additional siblings via those edges — this is the cause of the "double progress bar" on cold start.

After scanning, markMissingFilesAsDeleted() iterates the file table and marks entries whose files no longer exist on disk. This handles the case where a user deletes a source file between sessions. Marked files have their references purged via removeReferenceableItemsForFile() so that navigation doesn’t point to deleted files.

Only CUs are checked (they are glob-discoverable). Header files are not checked because they might legitimately be absent if they’re generated or live outside the project tree.

The three passes, by what they reparse: Pass 1 (stale edited CUs), Pass 2 (includers of stale edited headers) and Pass 3 (unparsed sibling CUs). Stale here means changed in the editor, i.e. sent by the client as a preload. A file changed behind the server’s back is not seen by Pass 1 or 2. When a file held as an edited buffer also changes on disk, the buffer wins, see Principles (Editor Buffers Are the Content Source).

The refresh covers two distinct concerns:

Both run before the operation dispatches, but they’re answering different questions and live in separate calls. Within buffer-driven refresh, every strip comes before any build; see Pass 2 for the rationale.

Between them, the passes and the cleaning concern below cover the ways the codebase can change from one request to the next:

Every row but the last fits the invalidate-then-rebuild model cleanly. The last is the gap the next section addresses.

The simplest case: the user edited a .c file, or the client sent a -preload with newer content. It is also the case when a header the CU includes changed on disk.

Detection: collectOutOfDateHeldCUs() takes each held CU for which knowledgeIsOutOfDate() is true, i.e. it has no knowledge, an input is gone, or an input changed after its knowledge was built (ADR-0032, ADR-0035). A held CU that is gone from disk is marked as deleted instead.

Action: none yet. The CU is put on the list, and is built after Pass 2, together with the includers.

Side effect: Building a CU also refreshes its TypeCppInclude references (the include edges).

If the user edited a header file, all CUs that include it (transitively) have stale references — even though those CUs themselves haven’t changed.

Detection: stripChangedHeldHeaders() takes each held header for which fileNumberIsStale() is true, i.e. its buffer is newer than when it was last parsed.

Discovery: collectCUsIncluding() walks the reverse include graph:

The header’s own references are then stripped, and building the includers puts them back. The includers are added to the list from Pass 1, without duplicates, so a CU that is both out of date and an includer is built once.

Why every strip comes before any build: A CU built before a header it includes is stripped would lose its part of that header. The include edges used are the ones from before the request. When a user adds a new #include "foo.h" to bar.c, the bar.c → foo.h edge is not there yet, but bar.c is held and out of date, so it is built anyway.

On cold start (no snapshot, or snapshot doesn’t cover everything), many CUs have never been parsed. If the user navigates to a symbol defined in a header, the server needs references from other CUs that include that header — otherwise navigation only shows the definition, not all usage sites.

Discovery: parseUnparsedSiblingCUs() uses a forward-include lookup:

Scope control: Only CUs sharing headers with the request file are parsed — not the entire project. This keeps cold-start latency proportional to the local neighborhood of the file being worked on. That relies on every file that references a symbol including the header that declares it; an extern written directly in a .c file is deliberately not followed (ADR-0013).

Current limitation: The sibling lookup is based on all shared headers, not just the header containing the navigated symbol. This can over-parse on projects with widely-shared utility headers (e.g. avutil.h in ffmpeg pulls in ~978 siblings). A future optimization would filter to sibling CUs sharing the specific header that defines the symbol being navigated to.

Pass 3 follows includes from a position. Some operations have no position to start from, or are about the whole project by definition, so no include-based subset can make their answer complete. For those, after Pass 3, the server counts the CUs that are out of date. Within the cap (MAX_CUS_TO_REPARSE) it parses them without asking. Above it, it first asks "N of M files need analysing. Analyse all before …​?" (see Blocking confirmation under Interactive Dialogs below). Before parsing, it strips each header whose includers are all among them. The snapshot is saved after every 100 CUs, so that progress survives an interrupt or a crash. That is a save interval, not a limit.

The operations are search, unused globals and push by name (needsWholeProjectParsed()).

A related consequence: on a cold start the symbol menu at filter level 0 (all symbols with the name) only lists symbols from CUs parsed so far. A same-named static in a file not connected through includes appears only once that file has been parsed. This is left as is: the menu offers other symbols with the same name, so a missing entry does not make the selected symbol’s references wrong, and a menu filter change is too cheap an operation to put a parse-all dialog behind.

Operations that do not need every CU, and why:

Refactoring operations (rename, move-function, etc.) modify editor buffers in memory. The disk files and their mtimes are unchanged, so the normal mtime-based staleness detection doesn’t trigger.

After serverPerformRefactoring() completes, markModifiedEditorBuffersAsStale() iterates all editor buffers with modifiedSinceLastQuasiSave == true and sets their file’s lastParsedMtime = 0. This causes fileNumberIsStale() to return true on the next request, triggering a reparse via Pass 1.

This is called before quasiSaveModifiedEditorBuffers(), which clears the modifiedSinceLastQuasiSave flag and updates the buffer’s modificationTime.

Reparsing a CU strips its old references and adds its current ones — but "old" means only references at position.file == CU. removeReferenceableItemsForFile is position-scoped. For everything a CU emits into itself that is complete: stripped, then rebuilt.

A CU also emits references into the headers it includes — the declarations it records at header positions while walking the included text — and which declarations it emits can depend on the CU’s own preprocessor state. The position-scoped strip does not reach those.

For example, a header declares a function under #ifdef FEATURE. A CU that defines FEATURE records that declaration at the header’s position. If the CU later removes the #define and is reparsed, the strip clears only the CU’s own positions; the header was never edited, so nothing clears the now-orphaned declaration, and the reparse gates it out rather than re-emitting it. The reference lingers.

This gap is real and confirmed (test_preprocess_edit_removes_ifdef_define), and at present open: the orphan survives until the header is reparsed for some other reason. The chosen fix — not yet implemented — is to reparse by include structure, stripping and rebuilding the affected header’s contributions, gated on a cheap per-CU fingerprint of the CU’s header emissions so the cost is paid only when an edit actually changes what a CU emits. Per-CU provenance (recording which CU emitted each header-position reference) was considered and rejected as too costly at scale; see ADR-0025.

The Duality of Source Code

When a user edits code in Emacs, the source code exists in two forms:

For code analysis to be useful during active editing, c-xrefactory must treat editor buffers as the source of truth when they exist.

The Preloading Mechanism

When the Emacs client sends a command to the c-xref server (like PUSH or NEXT), it uses the -preload option to transmit modified buffers:

For example:

The process:

This ensures that the server analyzes what the user sees in the editor, not the potentially stale disk file.

EditorBuffer Lifecycle

Editor buffers have different origins and lifetimes:

The Emacs client sends -preload for all modified buffers on every request. This is not an optimization — it is a protocol invariant that the server depends on:

The server uses this to detect when a user saves or closes a file:

Without ghost buffer pruning, a stale preloaded buffer would persist indefinitely. The server would keep parsing from the old content even though the user saved the file and the disk version is newer.

Reference cleanup is essential: Simply closing the buffer is not enough. The reference table still contains entries from the stale parse. removeReferenceableItemsForFile() must be called before closing the buffer, so that the next parse (from the disk file) populates the table with fresh references without duplicates.

Navigation sessions are safe: Sessions store Reference structs (position-based), not EditorMarker pointers. Closing a buffer does not create dangling pointers in the browsing stack.

To implement incremental updates, c-xrefactory tracks when each file/buffer was last parsed.

The Fields

Each FileItem in the file table has:

It is a FileTimestamp value (struct timespec with nanosecond precision).

Dual Semantics

The lastParsedMtime field has dual semantics depending on context:

This works because editorFileModificationTime() abstracts over both:

The abstraction is seamless: code can check "has this file changed?" without caring whether it’s a disk file or editor buffer.

Change Detection

To determine if a file/buffer needs reparsing:

References flow through multiple stages in c-xrefactory, with different storage locations and ownership models at each stage.

Stage 1: Parsing

When a file is parsed:

Storage: cxMemory (a custom arena allocator)

Lifetime: Lives until the next update that reparses that file

Stage 2: The ReferenceableItemTable

The canonical storage for all references.

Key properties:

Stage 3: Session Stacks

When a user performs a PUSH operation (browsing a symbol), a session is created:

Key properties:

Why Separate Storage?

Memory ownership:

If sessions pointed directly to table references, we’d have:

Snapshots vs. live data:

The Staleness Problem

The separation causes a problem: session references can become stale.

Scenario:

Solution: needsBrowsingStackRefresh flag + updateSessionReferencesForFile()

When entry refresh reparses a file, it sets needsBrowsingStackRefresh = true on that file’s FileItem. When NEXT/PREVIOUS operations occur, gotoNextReference() checks this flag on the current and next reference’s file:

This keeps navigation working correctly with live-edited code.

Trade-offs of the Incremental Approach

Advantages:

Limitations:

For typical usage (navigating within files being actively edited), these limitations rarely matter. A fresh PUSH creates a new session with fresh references.

The key insights:

Some operations involve interactive dialogs with the client:

Given a region of C source inside a function body, the tool must:

All three happen during a single parse pass in PARSE_TO_EXTRACT mode.

The editor tells the server two byte offsets (mark and cursor). When the lexer reads past those offsets it injects synthetic tokens that turn the selection into a nested compound statement: an opening boundary before the region and a closing boundary after. An OL_MARKER_TOKEN sits inside each boundary — it is consumed by the lexer, invisible to the parser, and triggers the extract code to remember where the region begins and ends.

The synthetic braces serve two roles: they make the region parse as a proper block (so declarations at its top are legal), and they give the extract code a structural invariant to verify — the block boundaries must live inside the same enclosing function body. Violating it aborts extraction with Region / program structure mismatch.

A side effect of the synthetic block is discussed under Subtlety below.

Every reference emitted during the parse — variable uses, control labels, the synthetic block boundaries — becomes a node in a linked list ordered by source position. Each node carries:

This is the substrate for classification. The graph is not a syntactic AST; it is a flat sequence capturing the order in which the compiler would touch each reference.

For each local variable the extract code performs a small dataflow analysis over the graph, tracking whether the variable holds a value coming from outside the region, inside it, or both. From the resulting state at each of the variable’s references it assigns one role:

A second role, local result value, covers variables declared inside the region and returned from the function — see Subtlety below.

Classification in hand, the generator walks the program graph three times to emit:

The synthetic braces carry a side effect that matters for one specific case.

If a variable is declared inside the region and read after it, the synthetic closing brace pops the symbol from the parser’s scope before the post-region code is parsed. The later use then fails to resolve and the C parser falls back to K&R implicit declaration — treating the name as an extern int function call. The reference for the outside use is recorded against that implicit-function symbol, not against the region-local.

Without intervention, the classifier sees the region-local with only inside uses and drops it from the extracted function’s signature, leaving the call site broken.

The extract pass handles this by scanning the reference table after graph construction for the implicit-function shadows, matching them by name to region-locals, and splicing the shadow’s references back into the graph as outside-block nodes pointing at the region-local’s ReferenceableItem. The existing classifier then sees the true outside uses and classifies the variable correctly.

This is a deliberate post-parse reconstruction: the synthetic scope is structurally necessary for the parse to work, so the information lost by scope-popping is recovered from the forensic trail the parser leaves behind.

The server maintains a global editorUndo linked list. Every replaceStringInEditorBuffer call pushes an undo entry recording the buffer, offset, and original text. This is used internally during refactoring — not for user-facing undo.

The core pattern is silent apply, then replay with output:

The replay is necessary because the server edits its in-memory editor buffers directly, but the client needs protocol commands to apply the same edits to the Emacs buffers. By undoing and replaying, the server generates output for exactly the edits that survived safety checks.

The Emacs client layers multi-buffer undo on top of Emacs' native per-buffer buffer-undo-list. This lets the user revert a completed refactoring that touched multiple files.

Checkpointing (before refactoring):

c-xref-multifile-undo-set-buffer-switch-point snapshots every open buffer’s undo position:

This bookmark captures where each buffer’s undo history was before the refactoring.

During the refactoring:

The server sends edit commands via the protocol (the GEN_FULL_OUTPUT replay from above). The client applies them to the appropriate Emacs buffers. Emacs automatically records each edit in the buffer’s buffer-undo-list. Files not yet open are opened by the client to apply the edits.

Undo last refactoring:

c-xref-undo-last-refactoring pops the most recent snapshot and undoes each buffer back to its bookmarked position:

The two-pass ordering processes buffers without bookmarks first — these were opened by the refactoring and should be fully reverted.

Known limitation: There is a known issue where undo of files that were not open before the refactoring does not always work correctly — under investigation.

When currentInput runs out of lexems (read >= write), refillInputIfEmpty() uses the stream type to decide what to do:

Stream type must match buffer allocation:

Violating this invariant causes fatal errors when trying to free buffers from the wrong arena.

Pushing a NORMAL_STREAM onto the macro stack, then trying to free it as if it were MACRO_STREAM:

Concrete cases met along the way, each with the step it belongs to. A line goes when the row above that fixes it lands.

Two different numbers, on the same tree — ffmpeg in tests/test_ffmpeg, 2615 compilation units, 1.38M lines of .c and .h, on the maintainer’s Mac:

The first is what a batch prime costs and what a progress estimate would divide; the second is what a user waits for before anything appears.

The cold-start PUSH again, for AudioFIRContext from af_afir.c, on the WSL machine (session d25255b8), with the development build and the fixture’s c-xrefrc.tpl, from no snapshot. Three runs, within 0.15s of each other:

There is one round now, not two. What is to be built is collected once per request, before anything is built, so includes found while building wait for the next request, as do the 1439 files above the cap. The March number does not say which machine or build it came from, so the faster parse per compilation unit cannot be put down to a change.

CI cannot supply either number. Travis times script lines, not tests, and make ci runs the suite as make -j with no limit, so its total is build plus the longest test with everything else overlapped — an upper bound, not a measurement. Getting a series out of CI would mean printing elapsed time per test in tests/Makefile.

The Emacs protocol is interactive and LSP is not, but most of that gap can be closed in the server rather than in each editor. Every LSP editor gets what the server exposes through standard requests. Anything beyond that needs client code, written once per editor. So each operation should live in the lowest tier that can carry it:

The aim is to keep tier 3 small. The more an operation’s questions can be settled before it starts, or turned into a choice between buttons, the more editors get it for free.

This also shapes how the server is packaged. The server itself is editor-neutral. Each editor gets a thin extension that launches it and implements tier 3: a Node extension in VS Code, a Lua plugin in Neovim, eglot or lsp-mode in Emacs, a Python plugin in Sublime, a plugin (native LSP API or LSP4IJ) in JetBrains IDEs. Editors with little or no extension API, such as Zed and Helix, get tiers 1 and 2 only.

Reorganize code by moving function definitions between source files while maintaining correctness. Essential for refactoring large codebases into better module structure.

Example: Extract a utility function from main.c to utils.c, automatically handling visibility changes and header declarations.

A comment on the same line after a moved function always moves with it. If there is a comment above it, or above something that moves along, Move Function asks whether to move the comments too, up to a blank line.

Remove extern declaration: Remove the moved functions extern declaration from the header file for the source, if the function was not static.

Remove the forward declaration: Remove a declaration of the moved function from the source file. A static one left behind breaks the build.

Include management: Add the #include directives the moved function needs to the target.

Move Function to New Module: Create the target .c and the includes the function needs, then make the ordinary move, which creates the header, and add the module to the build (and a mock for the unit tests).

A static shared with what stays: Today it makes the move refuse. Ask whether to make it extern where it is (its declaration goes in the source’s header, which the target then includes) or to move it and make it extern from the target. Either way it is a dependency between the modules, so the user decides. The references in the function’s range are known when the move is made, since finding its boundaries parses the source.

Macros and types defined in the source .c have the same problem, and belong with include management.

Smarter header placement: Automatically find the right location in header files based on existing declarations and dependencies.

Preview: Show what will be changed before applying the refactoring.

Delete Function: Remove a function definition and its declaration. Safety check warns if callers exist. Reuses the function boundary detection from Move Function and the unused detection logic to verify no references remain.

Language Server Protocol (LSP) integration enables c-xrefactory to work with modern editors and IDEs (VS Code, vim/neovim with LSP plugins, etc.) beyond Emacs. This opens c-xrefactory’s refactoring and navigation capabilities to a much wider audience.

Go to Definition (textDocument/definition):

File Parsing:

Single-file scope: Only symbols in the currently opened file are accessible. To navigate to a function in another file, you must first open that file.

No local variables: Go-to-definition doesn’t work for local variables or function parameters. This works in Emacs mode but requires architectural changes for LSP.

Missing LSP features: Only textDocument/definition is implemented. Coming features:

See the README for LSP client configuration examples. Basic setup:

The LSP path reuses the native architecture rather than inventing its own — document events are entry-refresh sync points (Algorithms). The steps:

Exposing refactorings follows the three tiers in Roadmap: Reaching refactorings: standard requests first, then code actions and workspace/executeCommand (stubs exist, with codeActionProvider still disabled in src/lsp_handler.c), and custom methods only where an operation needs interaction during the operation itself.

When no .c-xrefrc covers the file, the server answers PPC_NO_PROJECT and the client asks "No project covers <file>, create new project?". Answering yes runs c-xref-project-new, which writes a .c-xrefrc in the project root, auto-detected from .git.

The operation that asked the question is then thrown away. c-xref-server-dispatch-no-project ends with (error "Project created") — flow control by error, as the comment there says — so the goto-definition or completion the user asked for never happens, and they have to repeat it.

It is a small annoyance today. It becomes the normal first contact with a new project once auto-discovery is the only way in, so it is worth fixing then.

Keep the dialog and re-send the original request once the config is written, rather than unwinding. The request’s dispatch data is still in hand at that point, so this should not need new plumbing — worth confirming before committing to it.

The server parses a compilation unit when a request needs it. An operation that is project-wide asks before parsing the rest, and entry refresh reparses what has changed before every operation. Parsing is therefore spread through a session instead of happening once. During this parsing, warnings and errors may occur (e.g., missing include files, syntax errors from missing -D defines). These are expected and harmless — c-xrefactory continues best-effort — but users benefit from seeing them to tune their .c-xrefrc (adding -I paths, -D defines, etc.).

What did not parse is collected in the buffer c-xref-diagnostics, with its position. The buffer is in a mode derived from compilation-mode, so RET in it, and M-g n and M-g p from anywhere, go to the place. The messages are grouped by the compilation unit whose parse found them, under a line Analysing <file>, as compile output is. When a unit is parsed again its group is replaced and moved last, and a unit that parses clean has no group. The mode line shows how many messages there are, e.g. C-xref:2, and mouse-1 on it shows the buffer. Notes are not counted, and with no messages nothing is shown.

The server sends these messages as <log> records, after an <analysing> record for the unit. Every other message record is modal, or a line in the minibuffer, and <error> also abandons the operation. A server’s stdout and stderr cannot be used either, since they are the synchronisation stream, and the client stops with SYNCHRO PROBLEM on anything else there.

In the server, a message about the code goes through parsingMessage(), which gives the position of the lexem the parser got last. The lexer and the preprocessor are ahead of the parser, so they use parsingMessageAt() with a position of their own, e.g. that of the #include. They send nothing unless the server was started with -errors, which the client does, and nothing in the parses that only refresh other files. So what is reported is what the request’s own parse found. A unit that is only reparsed to refresh it keeps its old group until it is parsed for a request again. warningMessage() and errorMessage() stay for what an operation itself has to say, and those are modal.

Distinct unresolved include names, each with a count of how many places wanted it. The set of names is short even in a badly configured project.

Two triggers for the same report: after a whole-project parse, and on demand, which is how a user asks whether the project is configured correctly.

An include that is not found often has a file with that name somewhere in the project. The report could then ask whether that is the one, and offer to add the -I for it to the config, i.e. "`foo/bar.h` was not found, there is one under lib/include/. Add -Ilib/include?". After that the CUs that wanted it are out of date, since the config changed, and get parsed again when needed. More than one candidate is a menu, and none is just the report.

The "Browse Global Unused" feature reports symbols that are defined but never used. In practice, certain files produce noise that hides genuinely unused symbols:

The noise also goes the other way. A call from a unit test counts as a use, so a function that only its tests call is not reported. When removing XrefMode, xref() was unused in production but did not show up, because xref_tests.c still called it.

Support exclude patterns in the project configuration to suppress unused detection for matching files:

Symbols defined in excluded files would be skipped during unused detection. Navigation and refactoring would be unaffected — only the unused detection filter applies.

References from excluded files should probably not count as uses either. Then a function called only from *_tests.c would be reported as unused in the production code.

The exclude patterns would be stored as a StringList in ProjectConfig (or options) and checked during the mapAddGlobalUnusedSymbolsToHkSelection iteration. The check would match the filename of the definition reference against the exclude patterns using the existing shell-glob matching logic.

A variable that is assigned but never read is as unused as one that is never mentioned. Unused detection does not report it, since the assignments count as uses.

A close relative is a variable that is read, but only ever assigned the value it starts with. fileAbortEnabled in c-xrefactory’s own code was such a global: a bool, false from the start, that was only ever set to false, so its one check could never change anything.

Unused Symbols also reports a variable whose only references, apart from its definition and declarations, are assignments. The parser already records an assignment as UsageLvalUsed, so this should be a filter on the usages that are already there. A variable whose address is taken (UsageAddrUsed) might be read through the pointer, so it is not reported.

The relative needs more, the value of each assignment, which the references do not carry. That is probably a later step, if at all.

The #include preprocessor statments are referenceable items just like typenames and variables. That means it is logical to expect that placing the cursor on an #include line and navigation would follow the same UX as for a variable: PUSH moves cursor to the definition (in this case the file itself) and NEXT to the next "reference", which in this case would be the next #include statement for that file.

This does not happen.

This needs to be investigated and understood before an attempt to design a solution can be made.

Files discovered through -I include paths are currently treated the same as project-owned files for navigation purposes. However, these files typically belong to external libraries or shared code that the user should browse but not modify through refactoring operations.

For example, when a project uses -I /shared/lib/include, the user can navigate into headers from that library. If a rename refactoring is performed on a symbol defined in the project but also referenced in those external headers, the refactoring should not propagate changes into the external files.

The legacy c-xrefactory treated all parsed files equally — there was no distinction between "my code" and "code I depend on." This was acceptable when projects were self-contained, but with the modern include-path-based project scope (see Principles, Project Scope) the distinction matters.

Files discovered through -I paths (but not listed as source directories in the configuration file) would be marked as read-only for refactoring purposes:

This requires a per-file or per-directory ownership flag in the file table, checked by the refactoring engine before applying edits. The flag would be set during the lightweight scan based on whether the file was discovered under a source directory (owned) or an include path (read-only).

The exact mechanism and UX (warning, confirmation, or silent skip) need further design.

_Generic selects an expression by the type of a controlling expression, most often to pick one of several functions, as in <tgmath.h>:

Today _Generic is neither a keyword nor defined away, so it parses as an identifier and every association is a syntax error. Error recovery keeps some references by chance but loses others, such as the default: branch and a typedef used as an association type.

Parse every association and record the references in all of them. Do not select the branch the controlling expression’s type would pick.

The grammar needs a _Generic keyword and a generic-selection rule with associations of the form type-name : assignment-expression and default : assignment-expression. Branch selection would only matter for navigation or completion on the type of the result (_Generic(…​).field), which is rare.

There is no way to inline a function or a macro. The case that asked for it was two wrappers in startup.c, saveMemoryCheckPoint() and restoreMemoryCheckPoint(), that only called saveCheckpoint() and restoreCheckpoint() with a fixed argument. Removing them meant rewriting each call by hand, and deleting the declaration in startup.h and the stub in startup.mock.

With the cursor on a function, or on a call to it, Inline Function replaces each call with the body, the arguments substituted for the parameters, and removes the definition. The declaration in a header and a stub in a .mock file go too. Inline Macro does the same for a macro.

A body that is a single statement, or returns a single expression, is probably the case to do first. Anything more needs more than text substitution. An argument with side effects must not be evaluated twice, and a name in the body can collide with one at the call site. Refusing with a reason is fine there.

A body that is one statement, or return expr;, where no parameter is assigned or has its address taken. Each argument replaces its parameter, in parentheses. The parameters' references tell if they are assigned, so this needs no data flow analysis. Both cases so far fit, the wrappers in startup.c and displayingErrorMessages(), which only returns options.errors.

Extract Function copies the region verbatim and puts what is needed at the edges, e.g. int b = *_b; first in the new function and *_b = b; last. Inline can do the same the other way round. Each parameter becomes a local in a block, initialised with its argument, and the body is copied verbatim:

Each argument is then evaluated once, and a body that assigns to a parameter, or takes its address, still works. Extract already makes such declarations from a local’s link name (getLocalVariableDeclarationFromLinkName()), and a parameter is a local. A parameter that is never assigned, with an argument that is a name or a constant, can still be replaced directly instead.

What is left is the return. A return e; last in the body becomes x = e;, or e; if the value is not used. A call inside an expression has no room for a block, so there only a body that is return expr; can be inlined. More than one return is refused.

Names can collide both ways. An argument can use a name that an earlier parameter’s local hides (int p = p + 1;). The body can use a global that a local at the call site hides. Both show in the references, which say what each name resolves to. A body with a static local, a recursive or variadic function, and a goto into the body are refused.

The parts that exist are finding every call (as for Rename), the extent of a function definition (Move Function) and the position of an argument at a call (the parameter refactorings).

Inline Macro is closer, because a macro means its text. The arguments replace the parameters verbatim, and only and # need care.

Extract Function only works on statements. Marking part of an expression, e.g. one operand of || in an if condition, gives a void function, which is not what you wanted. The case that asked for it was isPreloaded(buffer) || buffer→modified in closeEditorBuffersNoLongerHeld(), which had to be extracted by hand.

When the marked text is an expression, extract a function that returns it, with the type of the expression, and replace the text with a call.

If finding the type of the marked text proves hard, an explicit Extract Predicate could be an alternative. The user then says it is a condition, and the function is always a bool one. That probably covers most of the cases that come up.

Old C code often has K&R function definitions, where the parameter types are declared between the parameter list and the body. There is no way to turn such a definition into one with a prototype.

With the cursor on a K&R function definition, Modernize Function (or Prototype Function) moves the parameter declarations into the parameter list. A declaration without parameters elsewhere, e.g. int f(); in a header, gets the same parameter list. An implicit int, for the return type or for a parameter that is not declared, should probably be made explicit at the same time.

A parameter declared char, short or float is a catch. A caller without a prototype in scope passes it promoted, to int or double, so adding the prototype changes how it is called. Refusing, or at least warning, is probably right there.

A parameter’s name is given again in every declaration of its function, i.e. in the header, in a forward declaration, in the definition and in a mock. Each is a symbol of its own, so Rename changes one of them. Giving a parameter the same new name everywhere means one Rename for each definition, real and mock, and editing the declarations by hand.

With the cursor on a parameter, Unify Parameter Name asks for a name and gives the parameter that name in every declaration and definition of the function. In a definition that is a rename of the parameter, uses included. In a declaration it is only the name in the parameter list. A declaration that has no name for the parameter could get one, or be left as it is.

Extract Function gives one parameter for each local the region uses, and an in/out pointer for each one it changes. When the region works on a group of locals that always go together, the parameter list gets long. The case that asked for it was the loop in parseUnparsedSiblingCUs() (server.c) that collects the sibling CUs. Extracting it gave seven parameters. Four of them were the selection being built, i.e. the array of file numbers, its count and two counters for what was skipped.

With some locals marked, Introduce Parameter Object (or Group Locals into Struct) asks for a type name, declares a struct with those locals as fields, and replaces them with one local of that type. Every use becomes a field access, e.g. selection.count. An Extract Function after that passes one pointer instead of four.

The same for a group of parameters that always go together is Fowler’s original. With the cursor in a parameter list and some parameters marked, it asks for a type name, declares a struct with those parameters as fields, and replaces them with one parameter of that type. In the function body every use becomes a field access. The simplest thing at the call sites is to pack the arguments there, since they are probably separate variables on the outside, e.g. collectCUsIncluding(fileNumber, (CUList){cuFileNumbers, cuCount}, maxCUs). Turning the variables at a call site into one struct as well is then the locals variant above.

A second case was cuFileNumbers and cuCount, which go together through collectOutOfDateHeldCUs(), stripChangedHeldHeaders() and collectCUsIncluding() (server.c). There the count also comes back as the return value, so packing a copy at the call site is not enough. Passing a pointer to the packed struct, and reading the count back after the call, would work, but is not pretty. How the UI marks the parameters, and what to do with such a count, is not decided.

Extract Function refuses a region that a return, break, continue or goto leaves ("There are jumps in or out of region", makeExtraction() in extract.c). Only Extract Macro lets them through, since a macro is text and the jump lands where it was. The case was the loop in __has_include(<…​>) (yylex.c), which ends with a break at the > and a return 0 at the end of the line.

The program graph already has what it takes to find them. A return is a reference to the internal label -1, break and continue to the label of the innermost loop, and goto to the user’s label (jump_statement in c_parser.y). Each such node knows its target, which side of the region both are on, and its own position. What the graph does not have is the value of a return, since a constant is no reference, and where the statement ends. Both can be read from the text, from the position to the ;.

The new function returns how it ended, and the call site acts on it:

It has to be if and not switch, since a break in a switch leaves the switch and not the loop.

The first case to do is probably a region that is only left by return <constant>;, or return; in a void function, and has no result of its own. The new function then returns a bool, and the call is if (newFunction_(…​)) return 0;. A break out of a loop around the region comes next. A region that already returns a value, or a return of a local’s value, needs an out parameter as well, and could be refused to start with.

For each local the region changes, Extract Function copies it in from a pointer parameter at the start of the new function and back at the end, e.g. int cuCount = *_cuCount; and *_cuCount = cuCount;. That is safe, but the copies are usually the first thing you want gone.

With the cursor on such a parameter, Inline Output Parameter replaces every read of the local with *_cuCount, every write with an assignment through the pointer, and removes both copies. The parameter then takes the local’s name, cuCount, which is free now. A read that is an operand gets parentheses, so count` becomes `(*count) and not *count++. If the local was declared together with others, e.g. char *name, type;, only its declarator goes. It should refuse if the address of the local is taken, or if not every return writes it back.

The steps for one parameter have to be applied together. Done one at a time, the local is set but never read in between, and -Werror stops the build.

Turning a single in/out parameter into the return value is the natural next step, and could be a refactoring of its own.

Extract Variable replaces the marked expression with a new variable and declares it, initialized to the expression. The declaration is always suggested as int, since the type of the expression is not known (generateNewVariableDeclaration() in extract.c). You can change it, since the client asks for the declaration of the new variable, as Add Parameter does. It goes in on the line where the expression starts, not at the start of the statement that held it. So it lands in the wrong place when the statement started on an earlier line, and when the statement is the body of an if without braces or follows } else if.

Even so it was an immense help when generalizing a function. Extracting the varying parts of a function body as variables first, and then extracting the function, gives them as parameters. The case was handle_invalid_request() and handle_method_not_found() in lsp_handler.c, which differed only in an error code and a message.

The declaration gets the type of the expression, and goes in before the statement that held it. This is the same type that Extract an Expression as a Function needs, so the two probably share it.

The generalization above ended with inlining the extracted variables again at the first call site, by hand.

With the cursor on a local that is assigned once, Inline Variable replaces each use with its initializer and removes the declaration. It should refuse if the variable is assigned again, or if its address is taken. An initializer with side effects, or one that reads something changed between the declaration and a use, makes the result differ, and is probably reason to refuse too.

A literal that means something, e.g. an exit code the protocol prescribes, is better as a name. Today that is a #define, or an enum member, written by hand and then a search for the other occurrences.

With the cursor on a literal, Name and Define Constant asks for a name, defines it, and replaces the literal with the name. Other occurrences of the same literal could be offered, but only the user can tell which of them mean the same thing. The type does not tell either, e.g. two functions that return LexemCode can both return 0 for different reasons. Where the definition goes, i.e. in the file or in a header, and as a #define or an enum member, is not decided.

The server only works when a request arrives, and then the user waits. Out-of-date knowledge outside every request’s goal is never brought up to date, and on a large project the first request carries the whole initialization. With the sweep, bringing knowledge up to date a part at a time is safe, since each step removes only what is shown to be false.

When no request is waiting, the server does one unit of work, e.g. builds the knowledge of one out-of-date CU, and then checks for a request again. Answers fill in while you work, and a cold start answers what it knows at once. Other candidates are the sweep itself, saving the snapshot, and checking the change times of inputs.

The server has no threads and blocks in readOptionsFromPipe() until the next request. With all its global state, threads are not an option, so the work would be cooperative: wait for input with a timeout, and work when it expires. A parse cannot be interrupted, so a request may wait for one CU to finish.

Parse C and Yacc source files, producing symbol references and semantic information for the reference database and for feature-specific operations (completion, extraction, refactoring).

The parsing component consists of several internal modules:

The integrated preprocessor is a key architectural choice: by implementing its own preprocessor rather than using the system’s, c-xrefactory can navigate to macro definitions, show macro usage, refactor macro names, and complete macro identifiers. The trade-off is imperfect compatibility with all compiler-specific preprocessor extensions.

The parser’s behavior is configured through ParserOperation, decoupling parsing from server-level concerns:

The server maps its ServerOperation to a ParserOperation via getParserOperation(), so the parser never needs to know about server-level operation enums.

The parsers are generated using a patched Berkeley yacc (byacc-1.9). The patch modifies the skeleton to support error recovery and a recursive parsing feature (originally for Java). CPP macros rename parser data structures so that multiple parsers can coexist in the same executable. The Makefile generates and renames the parser output files.

Key entry points (see parsing.h):

parseToCreateReferences() is the clean entry point used by LSP mode, the entry-point reparse loop, and navigation refresh. It takes a filename, determines the language, sets up configuration, and parses.

callParser() is the lower-level dispatch used by server mode’s parseInputFile(), where configuration is set up separately with cursor position and operation-specific state.

Serve the editor extension by processing requests in an infinite loop. Each request is a set of command-line-style options received over stdin. The server:

The server runs as a long-lived process started by the editor (c-xref -server). Communication is over stdin/stdout using a text protocol where commands look like command-line options.

Before dispatching any operation, the server reparses stale preloaded files so that all operations see fresh in-memory references. This separates the concern of "keeping data fresh" from individual operation logic.

The entry-point reparse loop:

The needsBrowsingStackRefresh flag bridges the entry-point (which handles parsing) and the navigation module (which handles browsing stack updates). Navigation checks this flag instead of fileNumberIsStale() and calls updateSessionReferencesForFile() to update only the browsing stack without re-parsing.

After a browsing stack refresh, the server restores its position by finding the nearest reference to where it was, then advances (NEXT) or retreats (PREVIOUS) in list order. This respects the definition-first navigation ordering without assuming any particular sort order.

The server can only reparse and navigate files it knows about. "Knowing" a file means it has an entry in the file table — a file number and metadata (FileItem). But being known does not mean being parsed: a file in the table may have no references in the referenceableItemTable yet. This distinction matters because the entry-point reparse loop, the reverse-include graph walk, and navigation all operate only on known files.

Files enter the file table through three mechanisms:

Disk database (loadFileNumbersFromStore()). On the first request that triggers project initialization, the server loads file numbers from the .cx database. This is why -create was historically required before any interactive use: it populated the persistent database that seeds the file table. Without it, the server would only know about the single file in the current request.

Editor preloads. When the editor sends -preload <file> <tmpfile>, the file enters the file table via the editor buffer mechanism. Files created during a session — new source files the user opens — are discovered this way without any explicit rescan.

Project directory glob (scanProjectForFilesAndIncludes() → listFilesInDirectory()). At startup, after project context initialization, the server walks the project root recursively, filtering by known source suffixes (.c, .y, etc.) and honoring -prune paths (applied in listFilesInDirectory against the project-root-relative -prune entries). This discovers all compilation units in the project directory without requiring a prior -create invocation.

The three mechanisms are complementary: the disk database provides historical knowledge, preloads provide real-time updates during the session, and the project directory glob provides a fresh scan of what actually exists on disk at startup.

Design boundary: Files that appear on disk but are not in the file table — because they weren’t in the disk database, weren’t opened in the editor, and weren’t present at startup — are invisible to the server until the next session. The startup glob combined with editor preloads covers the primary use cases: all project files are discovered at startup, and new files created during the session enter through the editor.

Operations are classified by what they need:

Implement the Language Server Protocol interface, allowing LSP-capable editors (VS Code, Emacs lsp-mode, etc.) to use c-xrefactory’s parsing and navigation capabilities. Currently at proof-of-concept stage with partial `textDocument/definition support.

The LSP adapter is a self-contained subsystem with clear internal layering:

The LSP adapter takes a fundamentally different approach from the legacy editor server:

The LSP adapter is a proof-of-concept. Key gaps:

Coordinate refactoring operations: rename, move, extract, and argument manipulation. The refactory component receives a refactoring request, uses the parser and reference database to analyze the code, performs safety checks, and produces a sequence of edits for the editor to apply.

Refactoring operations run as server operations (OP_REFACTORY) within the long-lived editor server process. The editor sends the refactoring request (e.g. -rfct-rename, -rfct-extract-function) as a server command, and the server responds with edit instructions via the protocol.

The refactoring code saves options into refactoringOptions to preserve the refactoring intent while the server’s entry refresh and parsing machinery runs with its own options.

A typical refactoring (e.g. rename):

Before applying a refactoring, the refactory performs safety checks (OP_INTERNAL_SAFETY_CHECK). These use the reference database to verify that the transformation is semantically valid — for example, that a rename won’t collide with an existing symbol in scope.

Analyze control flow for extract-function/macro/variable refactoring. The extract component operates in two phases: collection during parsing (registers synthetic labels, gathers references) and analysis after parsing (classifies variables, generates output).

Extraction uses a specialized parse operation (PARSE_TO_EXTRACT) that tracks:

After parsing, it classifies variables as inputs (passed as parameters), outputs (returned), or local (moved into the extracted function), and generates the function signature, call site, and body.

Manage the in-memory symbol reference tables and the browsing session stack. The cxref component is the runtime engine for symbol lookup: it loads references from the on-disk database (via cxfile), merges them with freshly parsed references, and provides the data structures that navigation and refactoring operations query.

`c-xrefactory’s core functionality relies on a symbol database that stores cross-references, definitions, and usage information for all symbols in a project. The database has two forms:

Browsing Stack (sessionData.browsingStack)

The browsing stack is the runtime data structure for symbol navigation. Each push operation (triggered by a navigation request) creates a new session entry containing the references for the symbol under the cursor.

Referenceable Items

A ReferenceableItem represents a symbol (function, variable, macro, type, etc.) with its attributes: link name, type, storage class, scope, and visibility. Each referenceable item has a linked list of Reference entries recording every usage position.

References

A Reference records a single occurrence of a symbol: file, line, column, and usage kind (defined, declared, used, etc.). References are the fundamental unit that navigation, unused-symbol detection, and refactoring operate on.

When the user requests "go to definition" for a symbol:

The Memory module provides arena-based allocation for performance-critical and request-scoped operations:

Historical Context

In the 1990s when c-xrefactory originated, memory was scarce. The design had to:

Most memory arenas use statically allocated areas. Only cxMemory supports dynamic resizing to handle out-of-memory situations by discarding, flushing and reusing memory. This forced implementation of a complex caching strategy since overflow could happen mid-file.

Modern Benefits

Even with abundant modern memory, arena allocators provide:

Marker-Based Cleanup

Functions save a marker before temporary allocations:

Buffer Growth Pattern

Long-lived buffers that may need to grow:

Overflow Handling

The cxMemory arena supports dynamic resizing:

When overflow occurs, handler can:

Key functions (see memory.h):

See the "Arena Allocator Lifetime Violations" section in the Development Environment chapter for:

Modern systems have abundant virtual memory. Possible improvements:

The experimental FlushableMemory type explores some of these ideas but hasn’t replaced current implementation.

Content buffers provide an in-memory cache of file content that transparently overrides disk file reading during parsing. When a content buffer exists for a file, the parser uses its in-memory content instead of reading from disk.

Content buffers (the EditorBuffer struct) serve three distinct roles:

The lexer reads from a character buffer (currentFile.characterBuffer), which is the common layer regardless of content source. Where the character buffer gets its data depends on whether a content buffer exists:

When the lexer processes an #include, it calls findOrCreateAndLoadEditorBufferForFile() for the included file. This function:

If a content buffer is found, the character buffer is set up to read from it. If not (file doesn’t exist or is a directory), the fallback openFile() path reads from disk via a FILE handle. This means preloaded editor content transparently overrides disk content — the parser doesn’t know or care where the content came from.

Buffer lifetime depends on the server mode and buffer role:

The existing bufferIsCloseable() predicate distinguishes these cases: it returns false for preloaded buffers (preserving editor content) and true for loaded, unmodified, marker-free buffers (allowing cache cleanup).

For preloaded buffers, staleness is detected by comparing fileItem→lastParsedMtime with buffer→modificationTime. After parsing a preloaded buffer, lastParsedMtime is set to the buffer’s modification time. If the buffer is re-sent with a new tmp file (new mtime), the file appears stale and is reparsed.

See the Server component’s Entry-Point Reparse section for how staleness drives the sync phase.

Read and write the CXref database (.cx files) in a compact text format. Cxfile is the persistence layer: it serializes symbol references to disk after parsing and loads them back during navigation and refactoring operations.

The database is a single file, .c-xref/db in the project root. It holds the file metadata and paths first, then the symbol records. It is a startup snapshot, loaded whole and never consulted again, and written from memory on exit. Without a project root there is no snapshot.

It used to be split into files by a hash of the link name, sized by -refnum, so that a single file read could look up a symbol. Nothing reads it that way any more, so the split and -refs were removed, see ADR-0028. The snapshot still records the partition count, always as 1.

Records use the general format <number><key>[<value>]. The encoding uses single-character markers listed at the top of cxfile.c.

The coding often starts with a number followed by a character key: 4l means line 4, 23c means column 23. References are optimized to avoid repeating fields that haven’t changed — so 15l3cr7cr means two references on line 15, one at column 3, the other at column 7 (using fsulc fields: file, symbol index, usage, line, column).

Some fields carry a length prefix: filenames use <length>:<path> (e.g. 84:/home/…​/file.c), version strings use <length>v.

The snapshot starts with its format version, C_XREF_FILE_FORMAT_VERSION in cxfile.c. A snapshot of any other version is ignored like no snapshot, and rewritten on exit, so a bump costs one cold start.

Removing or adding an integer record needs no bump. The snapshot’s key list (@) names the integer records it uses, and the reader stores a listed key’s value whether or not it has a handler. So an old snapshot’s extra record is stored and never read, and a new record is 0 when an old snapshot lacks it, so a 0 must mean "unknown". Older binaries read newer snapshots the same way. A string record must stay out of the key list: the reader skips an unlisted key’s text by its length, but stores only the length of a listed one, and misreads the rest.

The version is a promise about what the snapshot means, not only about its syntax. Bump it whenever a snapshot written before a change could be read after it and give a wrong answer, also when every record still parses:

Each bump comes with a test_snapshot_from_<version>_is_ignored.

Example file information line:

Reading is controlled by scanFunctionTable arrays. Each entry maps a record key to a handler function. As the reader encounters a key in the file, it looks up the handler and calls it. This table-driven approach allows different consumers to register for different record types — for example, loading only symbol names vs. loading full reference lists.

The Emacs client extension provides the user-facing interface for navigation, refactoring, and completion. It starts the c-xref process on the first user interaction and communicates with the server process over stdin/stdout using a text protocol.

To give the server access to unsaved content the client sends -preload <file> <tmpfile> for modified editor. On PUSH, NEXT, and PREVIOUS, all modified buffers (including the current one if modified) are preloaded. Unmodified buffers are not preloaded — the server reads the disk file directly.

The preload mechanism writes the buffer content to a temporary file and passes both the original filename and the temp file path. The server creates an EditorBuffer with the content and the temp file’s modification time. Preloaded buffers persist across requests as long as the client keeps sending them. On each preload, the server compares the new temp file’s mtime against the existing buffer’s; if advanced, the buffer is reloaded with the new content, and the per-request entry refresh reparses it before the operation runs. When the client stops sending a preload (because the user saved or closed the file), the server detects this and prunes the now-stale buffer (ghost-buffer cleanup).

A preload means that the editor has newer content than the file. The temp file is written when the content changes, so it is newer than the file on disk and newer than anything the server knew before the request that carries it. The server relies on this when it decides whether its knowledge is out of date (ADR-0032). A client that sends an older temp file has a bug. == Data Structures

There are a lot of different data structures used in c-xrefactory. This is a first step towards visualising some of them.

A ReferenceableItem represents a referenceable entity in the codebase - something that can be referenced from multiple locations:

Each ReferenceableItem contains:

ReferenceableItems are stored in the referenceableItemTable (hash table) and persisted to .cx files.

A Reference represents a single occurrence of a ReferenceableItem at a specific location:

Each ReferenceableItem maintains a linked list of all its References, allowing you to find every place that entity appears in the codebase.

Note: The term "Reference" in this context means "occurrence" - one specific use of an entity at one location. This is distinct from C++ references or reference semantics.

Options has its own allocation using optAlloc which allocates in a separate area, currently part of the options structure and utilizing "dynamic allocation" (dm_ functions on the Memory structure).

The Options structure are copied multiple times during a session, both as a backup (savedOptions) and into a separate options structure used by the Refactorer (refactoringOptions).

Since the options memory is then also copied, all pointers into the options memory need to be updated. To be able to do this, the options structure contains lists of addresses that needs to by "shifted".

When an option with a string or string list value is modified the option is registered in either the list of string valued options or the list of string list valued options. When an options structure is copied it must be performed using a deep copy function which "shifts" those options and their values (areas in the options memory) in the copy so that they point into the memory area of the copy, not the original.

After the deep copy the following point into the option memory of the copy

Arena allocators use bump pointer allocation:

This is extremely fast (O(1) allocation) but requires stack-like discipline for deallocation.

Arenas follow LIFO (last-in-first-out) cleanup:

Only the most recent allocation can be resized:

This constraint is enforced by guards in memory.c (see Development Environment chapter).

c-xrefactory uses specialized arenas for different purposes (see Components chapter for details):

When an editor buffer is modified but not yet saved:

Without preload, the server would only see the last saved version of the file. The preload mechanism ensures that:

When a file is preloaded, the server must handle reference updates carefully:

The browser stack is a linked list of OlcxReferences entries, where each entry represents a symbol lookup session:

The browser stack is populated in two stages:

This separation means that browser stack entries can become stale if files are reparsed (e.g., with preloaded content) without refreshing the stack. Users typically need to pop and re-push to get fresh reference lists after edits.

Each BrowsingMenu entry is a menu item wrapping a ReferenceableItem with UI presentation state:

Key insight: BrowsingMenu is not just a menu - it’s a menu item. A collection of BrowserMenu items forms the actual menu shown to the user.

A single browser stack entry (OlcxReferences) can contain multiple BrowsingMenu items:

This allows users to:

Browser menus are populated by scanning the referenceableItemTable:

Result: A persistent database mapping each entity (ReferenceableItem) to all its occurrences (References) across the entire codebase.

Result: Interactive navigation through the cross-reference database with selection and filtering.

This architecture separates concerns:

Nothing is scheduled. A request names one file, the editor’s current buffer, as the only bare word on its command line, and initServer() enters it in the file table with addRequestFileToFileTable(). The file need not exist on disk yet. Menu and filter operations name none. prepareInputFileForRequest() makes it the request’s requestFileNumber.

The rest of the project is known without being named: the structure scan (scanProjectForFilesAndIncludes()) finds the compilation units and their includes, and the snapshot holds what was parsed before. See Unified Server Flow (ADR 22).

The startup command line carries no file: c-xref -server file.c is refused before the server starts.

SINGLE FILE PER REQUEST - Server processes one file per request.

The entry refresh (see Terminology in Principles) ensures the reference table is up-to-date before each operation. Pass ordering is critical: Pass 1 reparses stale CUs, which updates their TypeCppInclude references. Pass 2 then queries those freshened references to find which CUs include stale headers. Pass 3 uses the include structure from the lightweight scan to find CUs sharing headers with the request file that have never been parsed. A user cannot create a new include edge without editing the includer, so the includer is always stale when the edge is new.

The per-request dispatch still follows the legacy pattern:

Assignments to `inputFileName`

Server Mode: Set once per file processed

`requestFileNumber` Only Used in Server Mode

Confusion: THREE Different File Number Globals!

currentFileNumber is defined in parsing.c:

The comment reveals the distinction: currentFileNumber can change DURING parsing when entering #include files, while requestFileNumber stays constant as "the file we were asked to process."

`initializeFileProcessing` is Heavy Orchestration

This 500-line function does five major phases:

The firstPass parameter gates Phase 4’s memory checkpoint save/restore.

Naming Inconsistency

If they represent the same concept (the file being processed), they should have parallel names.

Server Mode - Single File Per Request

LSP Mode - Single File Per Request

The auto-detect path uses a single initialization via initializeProjectContext and treats the in-memory reference table as authoritative. The legacy path (initializeFileProcessing) runs when no project root was detected. The flow below describes the auto-detect path.

Key principles:

This design eliminates the need for -create (replaced by the scan) and for callXref() before refactoring (replaced by scan + incremental reparse of stale files). It also makes the server viable for LSP, where blocking on a full project parse at startup is not acceptable.

Lexing/scanning is performed in two layers. lexer.c does the actual lexing into lexems, which are written into a lexembuffer — a sequence of encoded and compressed symbols, each starting with a LexemCode followed by a fixed set of attributes (e.g. Position).

The higher-level "scanning" is performed, as usual, by yylex.c. lexembuffer defines functions to put and get lexems, chars, identifiers, integers, and positions.

The put/get lexem functions currently take a pointer to a pointer to chars — the lexem stream inside the lexembuffer — which they also advance. This requires the caller to manage the LexemBuffer’s internal pointers and finally set them right when done. The direction is to pass a LexemBuffer directly; macro and define handling are the remaining cases where lexems are not put in a LexemBuffer.

Refactorings need semantic information beyond what the in-memory reference table holds — for example, byte offsets of block and function boundaries needed to verify that an editor region selected for "Extract Function" does not cross a block break. This information is collected when the refactoring entry calls parseBufferUsingServer() to drive a full parse over the buffer of interest; the parser’s semantic actions populate dedicated structures.

Two structures currently hold this information; the main one is parsedInfo.

The server tracks editor-buffer state via the -preload <filename> <tmpfile> option: the editor writes the in-memory buffer content to <tmpfile> and tells the server to treat that file’s content (and modification time) as the authoritative version of <filename>. The server creates an EditorBuffer accordingly and uses it in place of the on-disk file for parsing and reference creation.

This is essential during refactoring: the buffer the user is editing typically diverges from disk content, and the server must work against the buffer to produce edits that match what the user sees.

The client only preloads modified buffers — unmodified files are not preloaded even if they are open in the editor. The server then instead reads the files directly from disk. Preloaded buffers, meaning modified files open in the editor, are persisted in the server across requests until the client stops sending them. See "Preload" in Principles for the full lifecycle, including the per-preload mtime check and the connection to entry refresh.

The initial invocation of the edit server creates a process with which communication is over stdin/stdout using a protocol which from the editor is basically a version of the command line options.

When the editor has delivered all information to the server it sends 'end-of-option' as a command and the edit server processes whatever it has and responds with <sync> which means that the editor can fetch the result in the file it named as the output file using the '-o' option.

Setting the emacs variable c-xref-debug-mode forces the editor to copy the content of such an output file to a separate temporary file before re-using it.

The c-xref server is long-lived: the same process persists across requests for a project. The editor starts a new process on first contact with a project, and on project switch (c-xref-kill-xref-process sends -exit before killing the old process).

Communication between the editor and the server is performed using text through standard input/output to/from c-xref. The protocol is defined in src/protocol.tc and must match editor/emacs/c-xrefprotocol.el.

The definition of the protocol only caters for the server→editor part, the editor→server part consists of command lines resembling the command line options and arguments, and actually is handled by the same code.

The file protocol.tc is included in protocol.h and protocol.c which generates definitions and declarations for the elements through using some macros.

There is a similar structure with c-xrefprotocol.elt which includes protocol.tc to wrap the PROTOCOL_ITEMs into defvars.

There is also some Makefile trickery that ensures that the C and elisp impementations are in sync.

One noteable detail of the protocol is that it carries strings in their native format, utf-8. This means that lengths need to indicate characters not bytes.

The editor fires up a server and keeps talking over the established channel (elisp function 'c-xref-start-server-process'). This probably puts extra demands on the memory management in the server, since it might need to handle multiple information sets and options (as read from a .cxrefrc-file) for multiple projects simultaneously over a longer period of time. (E.g. if the user enters the editor starting with one project and then continues to work on another then new project options need to be read, and new reference information be generated, read and cached.)

FINDINGS:

The editor server is started using the appropriate command line option and then it keeps the communication over stdin/stdout open.

The editor part sends command line options to the server, which looks something like (from the read_xrefs test case):

In this case the "-browse-push" is the operative command which results in the following output

As we can see from this interaction, the server will handle input as command lines and manage the options as if it was a command line invocation.

This explains the intricate interactions between the main program and the option handling.

The refactoring protocol messages exchanged between the refactory process and the editor:

Rename Example

Invocation: -rfct-rename -renameto=NEW_NAME -cursor=POSITION FILE

Result: A sequence of precheck/replacement pairs:

followed by:

Extraction Example

Extraction (-rfct-extract-function) returns an <extraction-dialog> with three <str> parts: the call site replacement, the text to insert before the region (function header), and the text to insert after (closing brace). The editor applies these and then initiates a rename for the new function name.

Protocol Messages

Memory allocation is managed through the Memory structure, which implements an arena/bump allocator. Different memory arenas serve different purposes:

See Components chapter for detailed description of each arena’s purpose and lifetime.

The optMemory arena requires special handling because Options structures are copied during operation. When copying, all pointers into option memory must be adjusted to point into the target structure’s memory area, not the source’s.

Functions like copyOptions() perform this pointer adjustment through careful memory arithmetic, traversing a linked list of all memory locations that need updating.

There are three possible sources for options.

Not all options are relevant in all cases.

All options sources uses exactly the same format so that the same code for decoding them can be used.

When the client first operates on a file, the server determines which project it belongs to. On the auto-detect path the server walks upward from the file’s path looking for a project-local .c-xrefrc, and the directory containing it becomes the project root (detectedProjectRoot). If none is found, the file is not in any project. The -getproject operation drives discovery and reports the result back to the editor.

The configuration is re-checked on each request: the .c-xrefrc modification time is compared against the loaded version, and if it has changed, reloadProjectConfig() re-reads the file and the project scan re-runs with the updated includeDirs. This keeps the in-memory project description aligned with on-disk config without requiring a server restart.

The "watchers" are make targets that continuously watch, using a file watcher program, for changes in appropriate ("source") files and trigger rebuilds. The most used ones are watch-for-unittests which builds and runs all unittests, and watch-for-systemtests, which builds the c-xref program and then runs all systemtests except the very slow ones. watch-for-systemtests doesn’t actually watch sources but a special flag file, .unittests.done, which is touched by watch-for-unittests when successful. Each also writes its output to a log, src/unittests.log and src/systemtests.log.

The result is that if you have both running, you will get very quick feedback if a unittest fails, and slightly slower feedback on the systemtests. But always automatically on all saves. (HINT: use save-some-buffers in Emacs to save all your edits at the same time.)

C-xref started (probably) as a cross-referencer for the languages supported (C, Java, C++), orginally had the name "xref" which became "xrefactory" when refactoring support was added. And when Mariàn released a "C only" version in 2009 most of all the "xref" references and names was changed to "c-xref". So, as most software, there is a history and a naming legacy to remember.

The source code for c-xrefactory was using a very old C style with a separate proto.h where all prototypes for all externally visible functions were placed. Definitions are all over the place and it was hard to see where data is actually declared. This must change into module-oriented include-strategy.

Of course this will have to change into the modern x.h/x.c externally visible interface model so that we get clean modules that can be unittested.

The function prototypes have been now moved out to header files for each "module". Some of the types have also done that, but this is still a work in progress.

With -info the server logs why it analyses a file, e.g. 'cxref.c' is out of date: 'cxref.h' changed. The other reasons are that there is no knowledge of it, that it changed, and that its include closure is too large to check.

To get this from the server in your own Emacs, set c-xref-server-startup-options to ("-info" "-log=/tmp/c-xref.log") and restart the server. Then count the reasons with

The preprocessor macro expansion code uses arena allocators (ppmMemory, macroBodyMemory) with stack-like discipline. Arena allocators are fast (pointer bumping) but require careful lifetime management.

The Problem Pattern

Arena allocators can only resize the most recent allocation ("top-of-stack"). A common violation occurs when trying to resize a buffer after other allocations:

The correct pattern frees temporaries before growing the buffer:

Lifetime Violation Guards

The arena allocator includes guards that catch lifetime violations with detailed diagnostics:

Example: test_collation_long_expansion

This test case triggered the violation guard when macro expansion created a very large output (19 FLAG_STRING invocations). The collate() function was calling ppmFreeUntil() after copyRemainingLexems(), which needed to grow the caller’s buffer.

The fix: move ppmFreeUntil() to before copyRemainingLexems(). By this point, temporary allocations from macro expansion were already used and could be freed, allowing the buffer to become top-of-stack again.

Debugging With Guards

When a guard triggers:

The guards turn subtle crashes into clear diagnostics that point to the fix.

There are not very many unittests at this point, only covering a quarter of the code. The "units" in this project are unclear and entangled so creating unittests is hard since it was not build to be tested, test driven or even clearly modularized.

All unittests use Cgreen as the unittest framework. If you are unfamiliar with it the most important point is that it can mock functions, so you will find mock implementations of all external functions for a module in a corresponding <module>.mock file.

Many modules are at least under test, meaning there is a <module>_tests.c in the unittest directory. Often only containing an empty test.

In the tests directory you will find tests that exercise the external behaviour of c-xref, "acceptance tests" or "system tests". Some tests actually do only that, they wouldn’t really count as tests as there are no verification except for the code being executed.

There are two basic strategies for the tests:

Some tests do not even test its output in any meaningful way and only provide coverage.

Some tests do a very bad job at verifying, either because my understanding at that time was very low, or because it is hard to verify the output. E.g. a "test" for generating references might only grepping the CXrefs files for some strings, not verifying that they actually point to the correct place.

Hopefully this will change as the code gets into a better state and the understanding grows.

Tests live in the tests directory and are auto-discovered by name: any directory starting with test_ will be recognized as a test case.

Each test typically includes:

Most tests should use tests/Makefile.server, but tests/Makefile.cli is also available, which provides common macros:

The key macros are:

When $(VERIFY) passes, the output file is removed. This means you can easily identify failing tests by looking for test directories that still contain an output file. The utils/failing script lists these.

To suspend a test (skip it during test runs), create a .suspended file in the test directory.

Since all(?) c-xref operation rely on an options file which must contain absolute file paths (because the server runs as a separate process) it must be generated whenever the tests are to be run in a different location (new clone, test was renamed, …​).

This is performed by using a common template in tests and a target in tests/Maefile.boilerplate.

Each test should have a clean target that removes any temporary and generated files, including the .c-xrefrc file and generated references. This way it is easy to ensure that all tests have updated .c-xrefrc files.

Since many operations are performed from the editor, and the editor starts an "edit server" process, many tests need to emulate this behaviour.

The edit server session is used for navigation and refactorings alike.

In utils there is a server_driver.py script, which will take as input a file containing a sequence of commands. You can use this to start an edit, refactory or reference server session and then feed it with commands in the same fashion as an editor would do. The script also handles the communication through the buffer file (see [Editor Interface](./Design:-Editor-Interface)).

You can relatively easy re-create a sequence of interactions by using Emacs with all the latest source using utils/emacs-devel.

Once you have figure out which part of the *Messages* buffer are interesting you can copy that out to a file and run utils/messages2commands.py on it to get a file formatted for input to server_driver.py.

The system tests run in parallel. The watchers use -j$(JOBS). Until 2026-09-25 make ci, and so Travis, used an unbounded -j. Since 137421da it uses -j$(JOBS) too, as an experiment. Parallel runs work, but now and then a test fails with "Server did not exit cleanly within 5 seconds, killing…​" from the driver (--timeout in utils/server_driver.py). The tests vary, and each passes on its own in well under a second.

Sightings on Travis:

Before 2026-09-18 there were none in 194 builds, back to December 2025, although the unbounded -j had been there since 2022. From 2026-09-18 to 2026-09-25 there were 4 in 85 builds, about one in twenty. So something changed around 2026-09-18, not only the load. Candidates are more tests (223 to 243 in those days) and requests that parse more (594de6a3, 3a67dc91). Not verified.

How to read the experiment: if the rate were still one in twenty, 63 builds in a row without a sighting would happen by chance only one time in twenty. So count the builds since 137421da, and add every new sighting to the table.

Running the suite yourself while the watchers run it gives failures that look the same. Both runs write output.tmp and output in the same directories, so different tests fail each time and none of them fails on its own. The two earlier sightings, both local, may have been that: test_searching_cexercise with the same warning on 2026-09-15, and two tests whose server answered nothing after startup in a -j8 run on 2026-09-16.

If you see one of these, add it here with the date, the test and where it ran.

The Emacs client has ERT tests in editors/emacs/tests/*-tests.el, run by make -C editors/emacs test and from src’s `test target. make -C editors/emacs check is a separate thing: it byte-compiles everything as a lint and keeps no .elc.

Do not redefine a primitive in a test. Our own functions are byte-compiled Lisp, always called through the symbol’s function cell, so cl-letf on symbol-function is reliable for them. A C primitive is not: natively compiled Lisp calls it directly, and to keep it redefinable at all Emacs needs a trampoline - a small natively compiled shim in front of the primitive that does go through the function cell. Emacs ships trampolines for common subrs and generates the rest on demand, by writing a temporary .el, compiling it, and caching an .eln.

Both halves of that bite. Seen on the Mac (session 7c731c79, 2026-09-25, Emacs 31.1) while writing tests/c-xref-database-tests.el:

Reach for the state rather than the accessor (setq buffer-file-name inside a real buffer), or use the real thing (a temporary file, so file-exists-p is honestly true). Beyond the lost time there is a CI reason: a test that redefines a primitive passes or fails according to how Emacs was built and whether its .eln cache is writable, and Travis' Emacs is not the one on any of our machines.

utils/covers.py is a Python script that, in some enviroments, can list which test cases execute a particular line.

This is handy when you want to debug or step through a particular part of the code. Find a test that covers that particular line and run it using the debugger (usually make debug in the test directory).

Synopsis:

utils/emacs-devel starts an Emacs that uses the current elisp code and the c-xref from src. This allows you to run a test changes without having to polute your own setup.

Synopsis:

The cxfile module was designed when the .cx files on disk were the reference database. Operations like "show all references to symbol X" or "find unused symbols" were table-driven scans over the disk format — hash the symbol name, stream the right partition, apply a per-operation filter. This was efficient for the batch cross-referencer that c-xrefactory evolved from, where the workflow was c-xref -create → c-xref -update → query the on-disk database.

Under memory-as-truth (Chapter 10), all operations now read from the in-memory referenceableItemTable; the .cx files serve only as a startup snapshot.

The major cleanup is complete. The operation-specific disk scanners have been removed, replaced by in-memory walks:

The disk-load helper that fed several of them has also been removed:

A disk-read tripwire in scanCxFile asserts that no .cx read happens after the initial snapshot load. See "Memory As Truth" in Principles.

The current public interface in cxfile.h is:

One small interface tightening — not a major improvement, but worth listing as a known loose end:

The yylex.c file is 2353 lines and combines multiple responsibilities:

The macro expansion code is a substantial, cohesive subsystem that would benefit from extraction into its own module. Currently, it’s deeply embedded in yylex.c, making both lexing and macro expansion harder to understand and test in isolation.

The macro expansion system in yylex.c comprises:

Core Responsibilities (~800 lines)

Key State

Memory Lifetime Separation

The system uses two distinct memory arenas with different lifetimes:

This separation is fundamental and should be preserved in any refactoring.

Extract macro expansion into a new module: macroexpansion.c/h

Public Interface

The new module would expose a minimal, focused API:

Module Boundaries

What moves to macroexpansion.c:

What remains in yylex.c:

Dependencies:

The macro module would depend on:

Architectural

Maintainability

Future flexibility

Phase 1: Preparation (Already Complete)

✓ Create LexemBufferDescriptor type for buffer management
✓ Refactor buffer expansion functions to use descriptor
✓ Eliminate return values for size updates

Phase 2: Create Module Structure

Phase 3: Incremental Function Migration

Move functions in this order (lowest risk first):

Phase 4: Integration and Cleanup

Risk: Complex dependencies

Mitigation:

Risk: Performance overhead

Mitigation:

Assessment: Low risk - macro operations are complex enough that function call overhead is negligible

Risk: Breaking existing tests

Mitigation:

Refactoring operations that need structural information from parsing (function boundaries, move target validation, parameter positions, etc.) do not call the parser directly. Instead, they go through parseBufferUsingServer() — a function that constructs magic string arguments (like "-olcxmovetarget", "-olcxgetfunctionbounds") and runs a full initServer() + callServer() cycle. This is the internal equivalent of the editor sending a command over the protocol.

This makes the control flow circular and hard to follow: a refactoring operation (already running inside the server) constructs fake arguments, re-enters the server, which dispatches the operation, which calls the parser, which produces a side-effect in global state, which the original caller then reads.

A clean parser API in parsing.h/c already exists and handles all call sites in server.c:

Two convenience functions have clean APIs but still bridge internally:

These demonstrate the migration pattern: the public API is already clean, only the implementation still bridges.

9 bridge calls to eliminate (2 in parsing.c, 7 in refactory.c):

Each call follows the same pattern: set global flags → call parseBufferUsingServer → read results from global state. The migration for each is: set ParsingConfig fields → call parser directly → read results from the same global state.

The two parsing.c bridges show the pattern clearly. Today:

The ParsingConfig is already set up — the bridge call is redundant, it just re-enters the server which re-reads the same config. Replacing it with a direct callParser() (similar to parseToCreateReferences) is the straightforward next step.

The refactory.c calls are slightly more involved because they also need buffer preloading (the editor marker’s buffer must be set as input for parsing) and some set additional global flags. But the mechanism is the same.

This refactoring is independent of the memory-as-truth architecture. The bridge removal is about how refactoring operations invoke parsing internally — not about where references come from or how they’re stored. It can proceed at any time, one call site at a time.

The Options struct is one global blob serving four distinct lifetimes:

Session and Project are tangled in time as well as in the struct. Process start no longer does project work, but the first request that names a file sets up the project, whatever that request is, so project setup is not yet bound to one step.

These lifetimes are held together by a save/restore discipline scattered across five call sites:

The refactoringOptions parallel global is the clearest symptom. Refactoring sub-tasks (extract, rename, parameter manipulation) synthesize argv like "-olcxextract" and re-enter processOptions, which overwrites options. Reading the refactoring intent (refactoringOptions.theRefactoring, refactoringOptions.renameTo, etc.) from a shadow copy is how that survives.

Compounding the problem, processOptions is not a pure parser. It opens the output file, turns each -D into a macro definition, and triggers file-processing initialization. Reading the config is not pure either. It picks the pass sections by looking at the global currentPass, and it raises the global maxPasses when it sees a -passN marker. Those side effects are why the global-mutation pattern is load-bearing today: parsing and its consequences are entangled.

Three tiers with explicit owners and binding moments:

Option parsing becomes a pure function:

No globals mutated, no output files opened, no pass flags flipped. The dispatcher decides what side effects to run based on the returned Request. Each operation gets a typed struct — a tagged union or base struct with per-operation extensions — so the dispatch switch reads typed fields instead of guessing which Options fields are meaningful for which operation.

options.index is one such field, the =N of seven options with three meanings: where to go in a list (-browse-goto=, -completion-goto=, -search-goto=), which item to select (-completion-select=, -search-select=), and a parameter number (-internal-parse-to-goto-param-name=, -internal-parse-to-get-param-coordinates=). With typed requests each of them gets its own field, and the parameter number is probably the same thing as parameterNumber (-parameter=).

The refactoring coordinator holds its own long-lived RefactoringRequest (the current refactoringOptions content, but owned by the coordinator, not a shadow global). Sub-operations construct their own fresh Request structs via normal parsing; the coordinator’s state is safe because it lives elsewhere. refactoringOptions disappears. The save/restore sites disappear — each tier has one owner, and nothing trampled means nothing to restore.

The client already forces server restart on project switch ("Do you want to switch project? Server will be restarted."). Project-tier state therefore genuinely locks in after the first project-defining operation — there is no mid-session project change to invalidate, no need for a ProjectChanged event kind, no cache-invalidation logic. The tier model matches the runtime commitment: one server, one project, immutable after bind.

Each step is independently shippable. No grand rewrite.

1. Continue growing ProjectConfig. Move project-scoped fields off Options one at a time (include paths, project name, compiler discovery results, detected project root). Field-by-field migration; each field is a self-contained change.

2. Extract side effects from option parsing. Parsing becomes pure; side effects remain where they belong but with a clear trigger point. This is the load-bearing step — without it, typed requests still have to round-trip through global state.

The comment in loadProjectSettings that ends in "!!! berk" describes the knot. Request options must be read before the config, the cached options after it, and the cache must not hold request options. With no side effects to replay, the order stops mattering.

A likely order, cheapest first:

The order comes from reading the code and has not been tried. The first attempt at the compiler test also changed three move function tests, for a reason not yet understood.

3. Introduce typed Request per operation. Pilot with one small, self-contained operation (candidates: OP_SEARCH, OP_ABOUT). Define the per-operation request struct, have the parser produce it, have the dispatcher consume it. Prove the shape end-to-end on one operation before spreading to the rest.

4. Migrate refactoring to own its Request. The coordinator (currently the refactoring call graph in refactory.c) holds a typed RefactoringRequest as first-class state. Sub-operations parse their own fresh requests. refactoringOptions, deepCopyOptionsFromTo(&options, &refactoringOptions), and beInteractive’s `localSavedOptions all die together.

The inputFileName global (already flagged as a cleanup target) becomes a Request field naturally in step 3 — same disease, same cure.

The codebase has two hash table implementations generated by C macro templates:

Only two tables still use hashtab:

All other tables (editorBufferTable, symbolTable, referenceableItemTable) already use hashlist.

The lightweight file structure scanning work (ADR 22) exposed hashtab’s limitations:

Migrate fileTable and macroArgumentTable from hashtab to hashlist, then remove hashtab entirely.

fileTable migration:

macroArgumentTable migration:

Remove hashtab:

The file table serves a dual purpose: hash lookup by name AND array indexing by file number. File numbers are used throughout the codebase as compact identifiers (stored in references, positions, sessions, etc.).

Hashtab provides both naturally — the hash slot index is the file number. Hashlist doesn’t — elements live in chains, not at stable array positions.

Possible approaches:

The parallel array approach is likely cleanest — it separates the two concerns (name lookup vs number lookup) explicitly.

The editor.c module (630+ lines) bundles at least four unrelated responsibilities under the misleading name "editor". The name suggests it’s about the external editor (Emacs), but most of the code has nothing to do with editor integration — it’s internal text manipulation infrastructure used by refactoring.

1. Text memory allocator (editorMemory[], allocateNewEditorBufferTextSpace, freeTextSpace)

A power-of-2 block allocator for buffer text content. Custom malloc pool with free lists indexed by size class. No relationship to "editing" — this is a memory management concern.

2. In-buffer text editing (replaceStringInEditorBuffer, moveBlockInEditorBuffer, removeBlanksAtEditorMarker)

The refactoring engine’s hands — insert, delete, replace, and move text within buffers, with undo tracking and marker adjustment. This is the actual "editing" code, but it operates on the server’s internal buffers, not the external editor.

3. Reference/marker coordinate conversion (convertReferencesToEditorMarkers, convertEditorMarkersToReferences)

Bridges two coordinate systems: the reference world (file number, line, column) and the buffer world (buffer pointer, byte offset). Walks buffer text to convert line/col to offset and vice versa. Used by refactoring to go from "where are the references" to "where do I edit".

4. Content buffer lifecycle (loadFileIntoEditorBuffer, loadAllOpenedEditorBuffers, closeAllEditorBuffers, quasiSaveModifiedEditorBuffers, editorFileModificationTime)

Buffer loading, closing, quasi-save (mtime bookkeeping for refactoring), and file-stat wrappers that check buffers before falling back to filesystem. This is the content buffer management that belongs with the ContentBuffer type.

The "Editor" prefix pervades the codebase beyond just this module:

None of these are about the external editor. They’re the server’s internal text manipulation primitives. TextMarker, TextRegion, TextUndo would be more accurate names, but renaming has cascading effects throughout the codebase.

editorInit() would move to wherever initialization belongs (it currently just calls initEditorBufferTable()).

editorMapOnNonExistantFiles is a completion helper that iterates all buffers to find files existing in buffers but not on disk. It belongs with completion or content buffer management.

ADR-0032 keeps knowledge up to date the way make keeps targets up to date. The code has the parts, but the names do not show which part of make each one is. freshness.c answers whether the knowledge of a CU is out of date, and nothing else. Running the rebuild is in server.c: the entry refresh with its passes, the held files and the sibling parsing. That is most of server.c today, and callServer() mixes it with setting up the request. (CodeScene’s code health for server.c went from 8.41 to 4.11 as this moved in.)

callServer() would then read as setting up, rebuilding, and running the operation.

Building c-xref runs $(MAKE) -C ../editors/emacs from the c-xref recipe, and that Makefile’s all target does this:

So every build deletes any .elc files, byte-compiles all the elisp, then deletes the result again. compile is a phony target with no prerequisites, so this happens whether or not any .el file changed. It costs roughly two seconds per build here, which is most of the time an otherwise up-to-date make c-xref takes, and it makes make test (which now depends on c-xref) pay the same on every run.

The comment says why the output is thrown away: utils/emacs-devel loads the sources, not the compiled files. So the compilation is not for using the result — it is a check that the elisp still compiles, and byte-compiler warnings are the real product.

Decide what the step is for and make it say so:

The protocol generation (c-xrefprotocol.el from protocol.th) is already a proper rule with prerequisites and should stay as it is.

Implemented. TBD.

Implemented. TBD.