SPEECH — lightweight English text-to-speech

Overview

sound.speech is a built-in, English-only TTS: a Klatt-style formant synthesizer (after Klatt 1980) with a small rule-based text front end.

It has no models and no data files, the whole pipeline is pure Lua tables and DSP on top of the sound.core float32 buffer. Its output is deterministic: the same text and options always produce byte-identical samples.

The voice quality is deliberately in the DECtalk/SAM class — clear, robotic, unmistakably synthetic. That is the intended aesthetic, achieved in ~2k lines of Lua.

local sound = require("sound")
local dev = assert(sound.open({ rate = 48000, channels = 2 }))
local mx = sound.mixer({ channels = dev:channels() })

local buf = assert(sound.speech.say("Docking request granted.", {
    voice = "computer",
    rate = dev:rate(),      -- resample to the device rate
}))

lev.run(function()
    sound.run(dev, mx)
    mx:play(buf, { gain = 0.9 })
    lev.sleep(3)
end)

Pipeline

 text ──► normalize ──► tokens ──► G2P + prosody ──► events ──► Klatt ──► buffer
          (speech.text)            (g2p, prosody)               (klatt)

          speech.phonemize(text, opts) ──────────────► events
          speech.synth(events, opts)   ──────────────────────────────► buffer
          speech.say(text, opts) = synth(phonemize(text, opts), opts)

Stage	Module	What it does
normalize	`sound.speech.text`	lowercase, numbers→words ("pad 3"→"pad three"), abbreviations, `word` emphasis, punctuation → phrase boundaries
G2P	`sound.speech.g2p`	exceptions dictionary (common irregular words, real stress) + NRL-formalism letter-to-sound rules (first-vowel stress heuristic)
prosody	`sound.speech.prosody`	durations (inherent × stress × speed, phrase-final lengthening, pauses) + F0 contour (declination, stress accents, terminal fall / question rise)
synthesis	`sound.speech.klatt`	5 ms parameter frames → cascade/parallel formant DSP: Rosenberg glottal source + flutter, seeded-LCG noise, resonators R1–R5 + nasal pole/zero, parallel frication branch

API

`speech.say(text, opts)` → buffer, info | nil, err

Text in, mono sample buffer out. info = { rate, dur (s), frames, events }.

opts:

key	default	meaning
`voice`	`"narrator"`	preset name (below) or a knob table
`rate`	16000	output sample rate — pass `dev:rate()`
any knob	preset value	direct knob override, e.g. `{ voice = "computer", f0 = 70 }`

`speech.phonemize(text, opts)` → events | nil, err

Runs the front half only and returns the editable event list: each event is { ph = "AA", stress = 0|1|2, dur = ms, f0 = Hz } (ARPAbet phonemes plus SIL pauses). This is the inflection/singing hook — edit dur/f0 freely (hand-set values are never rescaled), then render with synth.

`speech.synth(events, opts)` → buffer, info | nil, err

The phoneme-level entry. Accepts hand-authored lists too: missing dur/f0 are filled from the phoneme tables and the voice contour. Use it to fix a pronunciation the G2P mangles, or to compose chants/alarms directly:

local ev = { { ph = "EY", dur = 500, f0 = 220 }, { ph = "EY", dur = 500, f0 = 165 } }
local siren = assert(sound.speech.synth(ev, { voice = "robot" }))

`speech.voices`

The preset table (sound.speech.voices). Included presets: narrator, assistant, computer, robot, whisper, evil, imp.

Voice knobs

knob	default	meaning
`f0`	105	base pitch, Hz
`f0_range`	1.0	intonation depth; 0 = dead monotone
`speed`	1.0	speech rate
`formant`	1.0	formant scale = vocal-tract length (≈1.15 reads female, 1.3 small creature, 0.9 big machine)
`breath`	0.0	0..1, trades voicing for aspiration; 1 = full whisper
`flutter`	0.15	slow random pitch wobble; 0 = eerily steady
`growl`	0	0..1, alternate-pitch-period modulation (vocal fry / creak) — gravelly roughness; useful range 0.2–0.5
`quantize`	0	hold synthesis parameters in blocks of N 5 ms frames — the stepped "machine voice"
`gain`	1.0	output gain (peak-limited at 0.95)

ARPAbet phoneme set

Vowels IY IH EH AE AA AO UH UW AH ER, diphthongs EY AY OY AW OW, glides W Y, liquids R L, nasals M N NG, fricatives F V TH DH S Z SH ZH, aspirate HH, stops P B T D K G, affricates CH JH, pause SIL. Acoustic targets live in sound.speech.phonemes — they are tuning data, meant to be adjusted by ear.

Tuning & testing

Unit tests (G2P, prosody, determinism, pitch/formant placement via Goertzel) run hardware-free with the normal suite:

./run_tests.bash sound

The audition harness plays every preset — or dumps WAVs when there is no audio device (for off-box listening or spectrogram inspection):

lilush tests/sound/manual/speak.lua                      # play all presets
lilush tests/sound/manual/speak.lua computer "Hello."    # one preset, custom line
lilush tests/sound/manual/speak.lua --wav ./speech_demo  # dump WAVs

Overview

Pipeline

API

speech.say(text, opts) → buffer, info | nil, err

speech.phonemize(text, opts) → events | nil, err

speech.synth(events, opts) → buffer, info | nil, err

speech.voices