Expanding the Roots ecosystem into language intelligence. Roots-LLM is our ongoing research into a foundational language engine built for Africa’s deep linguistic diversity, cross-lingual context, and heavy code-switching.
A speech engine like Roots Magana is only as good as the language intelligence behind it. Roots-LLM is an ongoing research initiative exploring a foundational language engine that handles multi-dialect context, local compliance, and regional reasoning — designed to complete the loop between speech recognition and speech synthesis.
Generic tokenizers fragment African languages into 4–6 sub-tokens per word, inflating inference costs and destroying semantic understanding. Extending the vocabulary is our highest-ROI research focus.
We are exploring extending base tokenizer vocabularies with 10,000–20,000 custom tokens drawn from local language corpuses — reducing fragmentation and improving generation speed.
High-quality parallel data for translation and text generation across dozens of African languages — a key candidate corpus for tokenizer and CPT research.
An excellent resource for fine-tuning on regional nuances, sentiment, and dialectal expressions — informing our SFT and DPO research directions.
Rather than training a 7B model from scratch, our research direction is to execute continued pre-training (CPT) on open-weight backbones, blending extended vocabulary into existing embedding layers with minimal catastrophic forgetting.
Feeding 50–100 billion tokens of mixed African text — local news, literature, digitized government archives, local web crawls, and conversational datasets — using a low learning rate (~2×10⁻⁵) to blend new vocabulary without forgetting general reasoning.
Curating high-quality instruction-tuning pairs that mimic real conversational patterns — switching from English to Yoruba or French to Wolof midway through a sentence — to make code-switching a native strength.
Roots-LLM and Roots Magana are designed to work together — with a long-term research ambition of a native, end-to-end audio LLM rather than a cascaded framework.
The foundational language engine handling multi-dialect context, local compliance, and regional reasoning — the intelligent core that processes text to form responses in an interactive voice platform.
The specialized voice interface layer — STT and TTS units that feed directly into the language model. Roots Magana-ASR converts speech to text, and Roots Magana-TTS synthesizes responses back to audio.
An exploration of an end-to-end native audio LLM — connecting the acoustic encoder directly to the LLM using a projection layer — to preserve emotional nuance, sarcasm, and exact accent cadences that are lost in transcription.
Interested in collaborating on African language intelligence research, exploring the Roots ecosystem, or discussing data partnerships? Let’s talk.