·
We publish letscodeit.dev blog posts from a separate GitHub repository. The Next.js app fetches markdown at runtime with a one-hour cache,…
GEO (generative engine optimization) is how you structure and publish content so AI chatbots and answer engines quote your pages when…
When you send a message to an LLM, you are not paying for characters or words. You pay for tokens . A token is the unit the model splits…
A 28.9 million parameter language model is running on an $8 ESP32-S3 microcontroller.
The model operates entirely on the chip, generating text at about 9 tokens per second and displaying it on a small screen. This achievement is possible because most of the model's parameters are stored in flash memory, using Google's Per-Layer Embeddings technique.
Source: github.com
Impressive to see a 28.9M parameter model running on such low-cost hardware!
I've been working with ESP32-S3 for IoT projects, but this AI implementation is next level. The use of flash memory for model storage is clever.
Generating text at 9 tokens per second might be too slow for some applications. Could this be optimized further?
Comments