Completion

Overview

This document specifies the completion engine architecture, provider and source interfaces, and integration with the input object.

The completion system provides context-aware candidate suggestions for interactive input. It is built around three layers:

The engine maintains two completion layers:

Rendering is handled by a pluggable renderer attached to the input object. The default renderer draws inline ghost text.

Files

PathRole
src/term/term/input/completion.luaCore engine
src/term/term/input/renderer.luaRenderer interface and inline ghost renderer
src/term/term/input.luaInput object integration
src/shell/shell/completion/shell.luaShell mode provider
src/shell/shell/completion/lua.luaLua mode provider
src/shell/shell/completion/wiki.luaWiki mode provider
src/shell/shell/completion/utils.luaShared helpers (match_prefix, match_prefix_keys)
src/shell/shell/completion/source/Completion sources
src/shell/shell/completion/source/learned.luaLearned subcommand source (MNEME-backed)

Engine API

Construction

local completion = require("term.input.completion")
local theme_mod = require("theme")

local comp, err = completion.new({
    path = "shell.completion.shell",
    tss = theme_mod.subscribe("shell", "completion"),
    sources = {
        "shell.completion.source.bin",
        "shell.completion.source.builtins",
        "shell.completion.source.fs",
        "shell.completion.source.calm",
    },
    -- additional config fields are passed through as comp.cfg
})

The optional tss field is a theme-subscribe thunk (a closure returned by theme.subscribe()). When provided, the engine's default get() uses it for metadata-driven styling — providers no longer need to implement their own get() method.

The path module is loaded and its exported methods are injected directly onto the completion object. Each source module path is loaded and instantiated via its new() constructor.

Primary layer methods

These operate on layer 1 (traditional candidates).

MethodReturnsDescription
search(input, history, cursor_pos?)boolProvider search; populates candidates via provide() + set_meta()
available()boolWhether primary candidates exist; auto-selects first
count()numNumber of primary candidates
get(promoted?)strCurrent candidate; raw when promoted=true, styled otherwise
chosen_index()numIndex of selected candidate (1-based, 0 = none)
set_chosen_index(idx)Set selected index (clamped to valid range)
meta_at(idx)tbl?Metadata for candidate at index
common_prefix()str?Longest common prefix across all candidates
provide(candidates)Replace candidate list
set_meta(metadata)Replace metadata table
flush()Clear both layers

Secondary layer methods

These operate on layer 2 (AI / auxiliary candidates).

MethodReturnsDescription
search_secondary(input, history)boolProvider search for secondary candidates
secondary_set(results)Set candidates; results is {{text=, score=}, ...} or nil
secondary_set_ghost(text)Convenience: set single ghost text
secondary_text()str?Current secondary candidate text
secondary_count()numNumber of secondary candidates
secondary_scroll(direction)boolCycle "up" or "down" through candidates
secondary_flush()Clear secondary layer only
refresh_secondary(input, history)Re-trigger secondary search if provider supports it

Source management

MethodDescription
source(name)Get registered source by name
update()Call update() on all sources
register_source(name, src)Add a source at runtime
unregister_source(name)Remove a source; calls close() if available
close()Call close() on all sources

Promotion

MethodReturnsDescription
promote(candidate, metadata, line){line=, action=}Provider-defined; apply candidate to line
should_promote_full()boolProvider-defined; skip common-prefix narrowing?
should_auto_promote()boolProvider-defined; auto-promote on ENTER?

When no promote is provided by the provider, the input object falls back to simple append: line = line .. candidate.

Provider interface

A provider is a Lua module that returns a table of methods. These are injected onto the completion object at construction time, so inside all provider methods self refers to the completion instance.

Providers that need to keep state between calls should use the self.__provider table rather than self.__state (which is reserved for engine internals like layers and sources).

Required

-- Populate primary candidates for the given input.
-- Call self:provide() and self:set_meta() inside.
-- cursor_pos is passed when the input has eol_only = false.
search = function(self, input, history, cursor_pos)
    local candidates = self:source("fs"):search(input)
    self:provide(candidates)
    self:set_meta(metadata)
    return self:available()
end

Optional

-- Populate secondary (AI) candidates.
-- Call self:secondary_set() inside.
search_secondary = function(self, input, history)
    local results = self:source("calm"):search(input, opts)
    self:secondary_set(results)
    return true
end

-- Custom styling for the current candidate.
-- Return raw text when promoted=true, styled text otherwise.
-- Not needed when config.tss is set — the engine's default get()
-- uses the tss thunk with the candidate's source metadata.
get = function(self, promoted)
    local l = self.__state.layers[1]
    local variant = l.candidates[l.chosen]
    if promoted then return variant end
    return my_tss:apply(l.meta[l.chosen].source, variant).text
end

-- Apply a candidate to the current line.
-- Return {line = new_line, action = "execute" | nil}.
promote = function(self, candidate, metadata, line)
    return { line = line .. candidate }
end

-- Whether the current candidate must skip common-prefix narrowing.
should_promote_full = function(self)
    return false
end

-- Whether ENTER should auto-promote the current candidate.
should_auto_promote = function(self)
    return false
end

Existing providers

Shell (shell.completion.shell) — command, path, env, history completion with learned subcommands and CALM secondary search.

When the input has arguments, a dispatch table routes known commands (z, x, zx, cd, setenv, unsetenv) to dedicated handler functions. All other commands fall through to the generic chain: dedicated builtin handlers (cmds source), learned subcommands (learned source), then filesystem fallback. When a builtin handler or learned source returns empty candidates, the engine falls through to filesystem completion instead of returning nothing.

Learned subcommands are recorded after each successful command execution (exit code 0): the first positional argument (skipping flags) is stored as a sorted set member in the completions keyspace of shell.mneme, keyed by binary name, with scores tracking frequency. Commands that primarily operate on file paths (rm, ls, cp, mv, stat, rsync) are excluded from recording entirely. For other commands, arguments that look like file paths are filtered out — anything containing /, starting with . or ~, having a file extension, or resolving to an existing filesystem entry is skipped.

Implements promote with replace_prompt / exec_on_prom / trim_promotion / reduce_spaces metadata.

Lua (shell.completion.lua) — keyword and symbol completion for Lua mode. No secondary search.

Source interface

Each source is a module that exports new():

local M = {}

M.new = function(config)
    return {
        search = function(self, query, ...)
            -- Return array of candidate strings
            return { "match_a", "match_b" }
        end,
        update = function(self)
            -- Optional: refresh cached data
        end,
        close = function(self)
            -- Optional: release resources
        end,
    }
end

return M

Source utilities

The shell.completion.utils module provides helpers for the common prefix-match-and-sort pattern used by most sources:

local cu = require("shell.completion.utils")

-- Match array items starting with prefix, return suffixes
cu.match_prefix(items, prefix, opts?)
-- Match table keys starting with prefix, return suffixes
cu.match_prefix_keys(tbl, prefix, opts?)

opts is an optional table: suffix (string to append, default "") and sort (sort by string length, default true).

Built-in sources (shell)

SourceModule suffixCandidates
binsource.binExecutables from $PATH
builtinssource.builtinsShell builtins and aliases
fssource.fsFilesystem paths
envsource.envEnvironment variables
cmdssource.cmdsBuiltin-specific args (secrets, theme, calm, mneme, lilpack, ...)
learnedsource.learnedLearned subcommands from successful executions
calmsource.calmCALM language model suggestions
lua_keywordssource.lua_keywordsLua language keywords
lua_symbolssource.lua_symbolsLua symbols and members

Built-in sources (wiki)

SourceModule suffixCandidates
wiki_commandssource.wiki_commandsWiki mode commands (wiki, search, browse, view, info)
wiki_dbssource.wiki_dbs.mneme files from ~/.local/share/lilush/wiki/
wiki_entriessource.wiki_entriesEntry keys from the open wiki database
wiki_indexessource.wiki_indexesBrowsing index names from the wiki manifest
wiki_resultssource.wiki_resultsSearch result keys for the view command

Renderer interface

A renderer controls how completion candidates are displayed. It is attached to the input object via the renderer config field.

local renderer = require("term.input.renderer")

-- Use the built-in inline ghost renderer (default)
local r = renderer.new_inline_ghost()

Required methods

-- Render completion candidates at the cursor position.
-- Returns the number of characters drawn (for clearing).
draw = function(self, completion, cursor_col, available_width, tss)
    -- completion: the completion object
    -- cursor_col: cursor column within the visible area
    -- available_width: max drawable characters
    -- tss: TSS instance for styling
    return draw_length
end

-- Erase previously drawn completion.
clear = function(self, draw_length, blank_char)
end

Inline ghost renderer

The default renderer draws a single candidate as dimmed text after the cursor. When both primary and secondary candidates are present and the secondary text starts with the primary text, it merges them: the overlapping prefix is styled as "completion" and the remainder as "secondary".

TSS rules used:

RulePurposeDefault
completionPrimary candidate textGray ({146, 153, 167})
secondarySecondary / AI ghost textDark gray italic ({100, 110, 125})

Metadata

Metadata is a table indexed by candidate position (1-based). Providers can use plain tables or metatables with __index for lazy generation:

local meta_with_index = function(fn)
    local m = {}
    setmetatable(m, { __index = fn })
    return m
end

self:set_meta(meta_with_index(function(t, key)
    if key <= builtins_count then
        return { source = "builtin" }
    end
    return { source = "bin" }
end))

Common fields

FieldTypeUsed byDescription
sourcestrAll providersTSS rule name for styling

Shell-specific fields

These are handled by the shell provider's promote() method, not by the generic engine.

FieldTypeDescription
replace_promptstrReplace entire line with this prefix + candidate
exec_on_promboolExecute immediately after promotion
trim_promotionboolStrip leading whitespace from candidate
reduce_spacesboolCollapse consecutive spaces

Input integration

The input object (term.input) drives the completion lifecycle.

Configuration

local input = require("term.input")

local inp = input.new({
    completion = comp,        -- completion object
    history = hist,           -- history object (passed to search)
    renderer = my_renderer,   -- custom renderer (default: inline ghost)
    eol_only = true,          -- complete only at end-of-line (default)
})

Key bindings

KeyConditionAction
Any characterAt EOL, no trailing spacesearch_completion() + draw
TABCandidates availablepromote_completion() — common prefix or full
UPAt EOL, candidates availablepromote_completion_full()
DOWNAt EOL, not in historyScroll secondary or trigger search_secondary_completion()
ESCBuffer not emptyscroll_completion("up") — cycle primary candidates
RIGHTAt EOL, secondary ghostaccept_secondary_ghost() — accept ghost text
ENTERshould_auto_promote()promote_completion_full() — auto-execute

Completion lifecycle

  1. User types a character at EOL

  2. search_completion() calls completion:search(line, history)

  3. Provider queries sources, calls provide() + set_meta()

  4. draw_completion() delegates to renderer

  5. User presses TAB:

    • Single candidate or should_promote_full(): full promotion

    • Multiple candidates: append common prefix, re-search

  6. After promotion: refresh_secondary() triggers secondary search

eol_only flag

When eol_only = false, completion triggers at any cursor position. The cursor position is passed to the provider's search() as the cursor_pos parameter. The provider is responsible for determining what to complete based on cursor position. The promote() method handles how the candidate is inserted (append, replace token, etc.).

Note: the default inline ghost renderer draws at the cursor, which may overwrite text after the cursor in mid-line mode. A popup renderer is recommended for mid-line completion.

Writing a new provider

Minimal provider for a hypothetical "tags" completion:

-- my_app/completion/tags.lua
local search = function(self, input)
    self:flush()
    local prefix = input:match("#(%w*)$")
    if not prefix then return false end

    local tag_source = self:source("tags")
    local candidates = tag_source:search(prefix)
    self:provide(candidates)
    self:set_meta(setmetatable({}, {
        __index = function() return { source = "tag" } end,
    }))
    return self:available()
end

return { search = search }

Usage:

local theme_mod = require("theme")

local comp = completion.new({
    path = "my_app.completion.tags",
    tss = theme_mod.subscribe("my_app", "completion"),
    sources = { "my_app.completion.source.tags" },
})

local inp = input.new({ completion = comp })

The tss thunk handles styling via the source metadata field, so a custom get() is only needed for non-standard rendering.