Ukrainian developer Slava S., who uses the name slvDev on GitHub, has published ESP32-AI, an ESP32-S3 AI model project that runs a 28.9-million-parameter language model locally on a low-cost microcontroller. The interesting part is the memory trick: the board is not remotely built like an AI workstation, yet the model fits and runs on-device.
The target hardware is the ESP32-S3, a cheap microcontroller board commonly available for under $10 as a bare board and under $20 with a case in the United States. The variant described for the project has 512KB of SRAM, 8MB of PSRAM, and 16MB of flash storage. That is microscopic by normal local AI standards, where even small models usually expect their weights to live in fast memory during inference.
Slava said on X that an earlier mark for a chip in this class was about 260,000 parameters, achieved by Dave Bennett. ESP32-AI is much larger on paper, although its workload is also tightly constrained.
How did ESP32-AI fit a 28.9-million-parameter model on an ESP32-S3?
Slava used a technique from Google’s Gemma architecture called Per-Layer Embeddings. In plain terms, it changes which parts of the model must be kept in fast memory and which parts can sit in slower storage without wrecking inference.
The model was quantized to 4-bit precision, bringing the file size to 14.9MB. Slava then placed the 25-million-parameter embedding table in the ESP32-S3’s 16MB flash memory instead of trying to load it into the 512KB SRAM or 8MB PSRAM. According to the project description, this model architecture needs only a few rows from that table for each token, so flash latency does not dominate the whole process.
That leaves the small amount of fast SRAM available for the model’s “thinking core,” the reasoning weights used during generation. The mechanism is less glamorous than a GPU demo and more useful: avoid moving data you do not need, and stop pretending every parameter deserves the same memory tier.
What can this tiny model actually do?
The model was trained on the TinyStories dataset, so its output is limited to short, simple fictional stories. It is not a chat assistant. It does not answer factual questions, follow general instructions, write Python, or carry useful knowledge about the real world, according to the description of the project.
That limitation is central to reading the result correctly. ESP32-AI is a proof of concept for fitting a comparatively large neural model onto extremely constrained embedded hardware. It is closer to a narrow small language model, or even a micro language model, than a general-purpose local AI system.
Slava has suggested a practical direction for models in this size class: embedded devices that know one domain well and work offline. His example was a coffee machine with information about beans, grind, ratios, and water temperature, without depending on an app or network connection.
The project also connects to a broader point about AI hardware. Model size, memory placement, and task design decide whether local inference is practical. A small embedded language model can be useful for a narrow function, while a general assistant requires far more compute and memory.
A separate project by developer Emanuele Rodolà showed an even smaller version of that idea last year, fitting a bigram name generator into original NES games including Dragon Warrior and Final Fantasy. Rodolà’s project used a 729-byte model weight table and about 140 bytes of hand-written assembly inference code to generate character names on a MOS 6502-based system.
ESP32-AI does not make an ESP32-S3 a replacement for a desktop GPU. It does show that Per-Layer Embeddings can scale down into embedded hardware, and it gives another data point for the idea that tiered memory, including flash, can make inference more practical across a wider range of machines.
This story draws on original reporting from Tom's Hardware.
