This document specifies the completion engine architecture, provider and source interfaces, and integration with the input object.
The completion system provides context-aware candidate suggestions for interactive input. It is built around three layers:
Engine (term.input.completion) — layered candidate management,
source registry, provider dispatch
Provider — domain-specific search logic (e.g. shell commands, Lua symbols); injected into the engine at construction
Sources — individual data backends (filesystem, binaries, environment, CALM model) queried by the provider
The engine maintains two completion layers:
Primary (layer 1) — traditional candidates from the provider's
search() (e.g. matching filenames, commands)
Secondary (layer 2) — auxiliary candidates from the provider's
search_secondary() (e.g. CALM / AI ghost text)
Rendering is handled by a pluggable renderer attached to the input object. The default renderer draws inline ghost text.
| Path | Role |
|---|---|
src/term/term/input/completion.lua | Core engine |
src/term/term/input/renderer.lua | Renderer interface and inline ghost renderer |
src/term/term/input.lua | Input object integration |
src/shell/shell/completion/shell.lua | Shell mode provider |
src/shell/shell/completion/lua.lua | Lua mode provider |
src/shell/shell/completion/wiki.lua | Wiki mode provider |
src/shell/shell/completion/utils.lua | Shared helpers (match_prefix, match_prefix_keys) |
src/shell/shell/completion/source/ | Completion sources |
src/shell/shell/completion/source/learned.lua | Learned subcommand source (MNEME-backed) |
local completion = require("term.input.completion")
local theme_mod = require("theme")
local comp, err = completion.new({
path = "shell.completion.shell",
tss = theme_mod.subscribe("shell", "completion"),
sources = {
"shell.completion.source.bin",
"shell.completion.source.builtins",
"shell.completion.source.fs",
"shell.completion.source.calm",
},
-- additional config fields are passed through as comp.cfg
})
The optional tss field is a theme-subscribe thunk (a closure returned
by theme.subscribe()). When provided, the engine's default get()
uses it for metadata-driven styling — providers no longer need to
implement their own get() method.
The path module is loaded and its exported methods are injected
directly onto the completion object. Each source module path is
loaded and instantiated via its new() constructor.
These operate on layer 1 (traditional candidates).
| Method | Returns | Description |
|---|---|---|
search(input, history, cursor_pos?) | bool | Provider search; populates candidates via provide() + set_meta() |
available() | bool | Whether primary candidates exist; auto-selects first |
count() | num | Number of primary candidates |
get(promoted?) | str | Current candidate; raw when promoted=true, styled otherwise |
chosen_index() | num | Index of selected candidate (1-based, 0 = none) |
set_chosen_index(idx) | Set selected index (clamped to valid range) | |
meta_at(idx) | tbl? | Metadata for candidate at index |
common_prefix() | str? | Longest common prefix across all candidates |
provide(candidates) | Replace candidate list | |
set_meta(metadata) | Replace metadata table | |
flush() | Clear both layers |
These operate on layer 2 (AI / auxiliary candidates).
| Method | Returns | Description |
|---|---|---|
search_secondary(input, history) | bool | Provider search for secondary candidates |
secondary_set(results) | Set candidates; results is {{text=, score=}, ...} or nil | |
secondary_set_ghost(text) | Convenience: set single ghost text | |
secondary_text() | str? | Current secondary candidate text |
secondary_count() | num | Number of secondary candidates |
secondary_scroll(direction) | bool | Cycle "up" or "down" through candidates |
secondary_flush() | Clear secondary layer only | |
refresh_secondary(input, history) | Re-trigger secondary search if provider supports it |
| Method | Description |
|---|---|
source(name) | Get registered source by name |
update() | Call update() on all sources |
register_source(name, src) | Add a source at runtime |
unregister_source(name) | Remove a source; calls close() if available |
close() | Call close() on all sources |
| Method | Returns | Description |
|---|---|---|
promote(candidate, metadata, line) | {line=, action=} | Provider-defined; apply candidate to line |
should_promote_full() | bool | Provider-defined; skip common-prefix narrowing? |
should_auto_promote() | bool | Provider-defined; auto-promote on ENTER? |
When no promote is provided by the provider, the input object
falls back to simple append: line = line .. candidate.
A provider is a Lua module that returns a table of methods. These
are injected onto the completion object at construction time, so
inside all provider methods self refers to the completion instance.
Providers that need to keep state between calls should use the
self.__provider table rather than self.__state (which is
reserved for engine internals like layers and sources).
-- Populate primary candidates for the given input.
-- Call self:provide() and self:set_meta() inside.
-- cursor_pos is passed when the input has eol_only = false.
search = function(self, input, history, cursor_pos)
local candidates = self:source("fs"):search(input)
self:provide(candidates)
self:set_meta(metadata)
return self:available()
end
-- Populate secondary (AI) candidates.
-- Call self:secondary_set() inside.
search_secondary = function(self, input, history)
local results = self:source("calm"):search(input, opts)
self:secondary_set(results)
return true
end
-- Custom styling for the current candidate.
-- Return raw text when promoted=true, styled text otherwise.
-- Not needed when config.tss is set — the engine's default get()
-- uses the tss thunk with the candidate's source metadata.
get = function(self, promoted)
local l = self.__state.layers[1]
local variant = l.candidates[l.chosen]
if promoted then return variant end
return my_tss:apply(l.meta[l.chosen].source, variant).text
end
-- Apply a candidate to the current line.
-- Return {line = new_line, action = "execute" | nil}.
promote = function(self, candidate, metadata, line)
return { line = line .. candidate }
end
-- Whether the current candidate must skip common-prefix narrowing.
should_promote_full = function(self)
return false
end
-- Whether ENTER should auto-promote the current candidate.
should_auto_promote = function(self)
return false
end
Shell (shell.completion.shell) — command, path, env, history
completion with learned subcommands and CALM secondary search.
When the input has arguments, a dispatch table routes known
commands (z, x, zx, cd, setenv, unsetenv) to
dedicated handler functions. All other commands fall through to
the generic chain: dedicated builtin handlers (cmds source),
learned subcommands (learned source), then filesystem fallback.
When a builtin handler or learned source returns empty candidates,
the engine falls through to filesystem completion instead of
returning nothing.
Learned subcommands are recorded after each successful command
execution (exit code 0): the first positional argument (skipping
flags) is stored as a sorted set member in the completions
keyspace of shell.mneme, keyed by binary name, with scores
tracking frequency. Commands that primarily operate on file paths
(rm, ls, cp, mv, stat, rsync) are excluded from
recording entirely. For other commands, arguments that look like
file paths are filtered out — anything containing /, starting
with . or ~, having a file extension, or resolving to an
existing filesystem entry is skipped.
Implements promote with replace_prompt / exec_on_prom /
trim_promotion / reduce_spaces metadata.
Lua (shell.completion.lua) — keyword and symbol completion
for Lua mode. No secondary search.
Each source is a module that exports new():
local M = {}
M.new = function(config)
return {
search = function(self, query, ...)
-- Return array of candidate strings
return { "match_a", "match_b" }
end,
update = function(self)
-- Optional: refresh cached data
end,
close = function(self)
-- Optional: release resources
end,
}
end
return M
The shell.completion.utils module provides helpers for the
common prefix-match-and-sort pattern used by most sources:
local cu = require("shell.completion.utils")
-- Match array items starting with prefix, return suffixes
cu.match_prefix(items, prefix, opts?)
-- Match table keys starting with prefix, return suffixes
cu.match_prefix_keys(tbl, prefix, opts?)
opts is an optional table: suffix (string to append, default
"") and sort (sort by string length, default true).
| Source | Module suffix | Candidates |
|---|---|---|
bin | source.bin | Executables from $PATH |
builtins | source.builtins | Shell builtins and aliases |
fs | source.fs | Filesystem paths |
env | source.env | Environment variables |
cmds | source.cmds | Builtin-specific args (secrets, theme, calm, mneme, lilpack, ...) |
learned | source.learned | Learned subcommands from successful executions |
calm | source.calm | CALM language model suggestions |
lua_keywords | source.lua_keywords | Lua language keywords |
lua_symbols | source.lua_symbols | Lua symbols and members |
| Source | Module suffix | Candidates |
|---|---|---|
wiki_commands | source.wiki_commands | Wiki mode commands (wiki, search, browse, view, info) |
wiki_dbs | source.wiki_dbs | .mneme files from ~/.local/share/lilush/wiki/ |
wiki_entries | source.wiki_entries | Entry keys from the open wiki database |
wiki_indexes | source.wiki_indexes | Browsing index names from the wiki manifest |
wiki_results | source.wiki_results | Search result keys for the view command |
A renderer controls how completion candidates are displayed. It is
attached to the input object via the renderer config field.
local renderer = require("term.input.renderer")
-- Use the built-in inline ghost renderer (default)
local r = renderer.new_inline_ghost()
-- Render completion candidates at the cursor position.
-- Returns the number of characters drawn (for clearing).
draw = function(self, completion, cursor_col, available_width, tss)
-- completion: the completion object
-- cursor_col: cursor column within the visible area
-- available_width: max drawable characters
-- tss: TSS instance for styling
return draw_length
end
-- Erase previously drawn completion.
clear = function(self, draw_length, blank_char)
end
The default renderer draws a single candidate as dimmed text after
the cursor. When both primary and secondary candidates are present
and the secondary text starts with the primary text, it merges them:
the overlapping prefix is styled as "completion" and the remainder
as "secondary".
TSS rules used:
| Rule | Purpose | Default |
|---|---|---|
completion | Primary candidate text | Gray ({146, 153, 167}) |
secondary | Secondary / AI ghost text | Dark gray italic ({100, 110, 125}) |
Metadata is a table indexed by candidate position (1-based).
Providers can use plain tables or metatables with __index for
lazy generation:
local meta_with_index = function(fn)
local m = {}
setmetatable(m, { __index = fn })
return m
end
self:set_meta(meta_with_index(function(t, key)
if key <= builtins_count then
return { source = "builtin" }
end
return { source = "bin" }
end))
| Field | Type | Used by | Description |
|---|---|---|---|
source | str | All providers | TSS rule name for styling |
These are handled by the shell provider's promote() method, not
by the generic engine.
| Field | Type | Description |
|---|---|---|
replace_prompt | str | Replace entire line with this prefix + candidate |
exec_on_prom | bool | Execute immediately after promotion |
trim_promotion | bool | Strip leading whitespace from candidate |
reduce_spaces | bool | Collapse consecutive spaces |
The input object (term.input) drives the completion lifecycle.
local input = require("term.input")
local inp = input.new({
completion = comp, -- completion object
history = hist, -- history object (passed to search)
renderer = my_renderer, -- custom renderer (default: inline ghost)
eol_only = true, -- complete only at end-of-line (default)
})
| Key | Condition | Action |
|---|---|---|
| Any character | At EOL, no trailing space | search_completion() + draw |
| TAB | Candidates available | promote_completion() — common prefix or full |
| UP | At EOL, candidates available | promote_completion_full() |
| DOWN | At EOL, not in history | Scroll secondary or trigger search_secondary_completion() |
| ESC | Buffer not empty | scroll_completion("up") — cycle primary candidates |
| RIGHT | At EOL, secondary ghost | accept_secondary_ghost() — accept ghost text |
| ENTER | should_auto_promote() | promote_completion_full() — auto-execute |
User types a character at EOL
search_completion() calls completion:search(line, history)
Provider queries sources, calls provide() + set_meta()
draw_completion() delegates to renderer
User presses TAB:
Single candidate or should_promote_full(): full promotion
Multiple candidates: append common prefix, re-search
After promotion: refresh_secondary() triggers secondary search
When eol_only = false, completion triggers at any cursor position.
The cursor position is passed to the provider's search() as the
cursor_pos parameter. The provider is responsible for determining
what to complete based on cursor position. The promote() method
handles how the candidate is inserted (append, replace token, etc.).
Note: the default inline ghost renderer draws at the cursor, which may overwrite text after the cursor in mid-line mode. A popup renderer is recommended for mid-line completion.
Minimal provider for a hypothetical "tags" completion:
-- my_app/completion/tags.lua
local search = function(self, input)
self:flush()
local prefix = input:match("#(%w*)$")
if not prefix then return false end
local tag_source = self:source("tags")
local candidates = tag_source:search(prefix)
self:provide(candidates)
self:set_meta(setmetatable({}, {
__index = function() return { source = "tag" } end,
}))
return self:available()
end
return { search = search }
Usage:
local theme_mod = require("theme")
local comp = completion.new({
path = "my_app.completion.tags",
tss = theme_mod.subscribe("my_app", "completion"),
sources = { "my_app.completion.source.tags" },
})
local inp = input.new({ completion = comp })
The tss thunk handles styling via the source metadata field,
so a custom get() is only needed for non-standard rendering.