Set Marvin Free

Sebastião was looking to explore using a completely vocal interface for a toy or game, with no visual cues to rely on, and to make it completely offline with everything running locally with no need for external APIs: “This greatly simplifies the infrastructure required, but at the cost of trying to fit everything on-device,” he says.

Rasp-beary Pi

Landing on Raspberry Pi to power Marvin seemed like a natural conclusion.

“Raspberry Pi is the perfect fit for this kind of project: it’s powerful enough to run a wake-word detector, a lightweight speech-recognition model, and the full NLU pipeline entirely offline, while its Linux-based environment makes setup straightforward,” Sebastião explains. “I experimented with a Raspberry Pi Zero as well, but its underlying CPU architecture wasn’t compatible with some of the deep-learning packages needed. Oh, and [Raspberry Pi’s] compact format allows for it to be ‘easily’ concealed inside a stuffed animal!”

As Sebastião developed the project – adding the ability to process a wake word and tell a story – he decided to add more, feeling the idea at this stage was ‘a little bit empty’: “As a result, several more features were added, such as cracking jokes, giving random facts, riddles, tongue-twisters, etc. This culminated in the idea of adding a more ambitious feature through the means of a speech-based game that not only provided a different type of interaction, but also a fun venue for more storytelling.”

These speech-based games resulted in their own extra challenges – mostly memorising details necessary for the games as they progress. “It’s crazy how much work goes into making some of these seemingly simple games!” Sebastião notes.