narrated reality

new forms and interfaces for writing, enabled by machine intelligence


A camera that narrates images

A fully autonomous and self-contained image-to-text narrator: take a photograph, and receive expressive text related to the image in about 15 seconds.


A compass that narrates locations

Inspired by wardriving, a tool that lets you write with your car: built inside the center console from a police cruiser, and mounted with a navigator's compass from a B-17 Flying Fortress—with this compass, you'll always know where you're headed, as the device narrates your GPS locations.


<

A clock that narrates time

A punch clock that delivers narration when you punch in. Like the camera and compass, this clock works with any neural network cartridge in the narrated reality library.


A common platform

Each device uses an Nvidia Jetson TK1 system-on-chip computer, and a Datamax-O'Neil MicroFlash 4T thermal printer. Neural network cartridges from the narrated reality library may be inserted into the Jetson's SD card slot.


A library of options

Each device can use any SD card from a library of LSTM recurrent neural network models trained on different source texts.


Read more

Adventures in Narrated Reality is an essay about this project and the research that led to it.

By Ross Goodwin, 2016