January 30, 2017

How can A.I. help parents raise great kids?

Introducing Avo, the family robot


Our studio had just joined forces with the Microsoft Research group. The company was making great strides in voice synthesis and recognition, image recognition and labeling, conversational interfaces with Cortana, and a whole lot more. With this momentum, our leadership wanted to explore new ways to leverage these advances.

So, the whole studio broke into groups of five or so designers and researchers. Our group’s initial ideation phase was noticeably unique due to the extra-wide reach. Most of us had long laundry-lists of wild ideas. From augmenting education with kids to assisting elderly adults. I was chasing scenarios that involved creativity but could never quite bring a focus to a particular use case, specifically in a problem-solving sense. 

After aggregating everyone’s lists, an interesting pattern began to emerge. Lifecycle-specific use cases kept coming in. Whether it was helping Grandma remember to take her medicine or help Timmy with his stutter. Then there was the question of having an array of robots with very specific functions or a do-it-all, much like Rosie from the Jetsons. Would the robot belong to the parents, the kids, or the whole family? Would the robot enable lazy parents to not raise their kids properly? Would the kids hack it for nefarious reasons? The back-and-forth discussions were truly fascinating!

Possible areas of discovery

  1. Add sound/color to photos/videos. (Silent Film enhancer?)
  2. Auto-translate films into my language
  3. Summarize complex/long texts (Legal docs, prevent the news filter bubble, news, research, plagiarism detection)
  4. Auto-generate film trailers or YouTube clips based on existing patterns of “how to make a good trailer”
  5. Preserve dying languages (reading texts/audio interacting)
  6. Going deeper with existing image labeling tech Not “this is a cat” but “this is a cat on a couch in the evening in Seattle around 1987”
  7. Estimate value of car/401k/house/item.
  8. Creativity. What happens when you randomly throw two datasets together and allow ML to find patterns?
  9. Better notification platform (devices need to be aware of each other and user behavior patterns
  10. Better contextual awareness in mapping scenarios (“turn left after the fire station”) Also discussed: other layer awareness – “entering the Tolt River watershed”, “west through Wallingford”, etc.
  11. Creativity: throw tons of image/video/text, have AI compose a story/film/etc.
  12. AI Critic: “This film is good because of X, Y, & Z”

Early ideation


Core value proposition: Augmenting parents



Helping parents help their families

We decided not to be the family pet or babysitter, nor the one-trick pony stuck in the kitchen. Instead, we began to think of a household A.I. entity as a bridge between all the knowledge of the internet and busy, multi-tasking parents. The robot would also have an intimate context of everyone’s schedules and the age-related context of children’s developmental sensitivities. The robot would also need to have a ubiquitous presence, much like a home’s electrical system, except better—accessible through several of the family’s rooms and devices.

Crafting the story

When it came down to how we’d express this story, there were a few touchpoints we agreed to ignore in order to maintain focus. We chose not to show any mocked-up software or hardware in order to focus on voice as the primary interface, we shied away from the perceived gender of the agent, and finally, we wanted to avoid any of the children interacting with the robots—this was primarily a tool for parents.

Presenting results

After putting together a few scenes into a narrative, recording audio commentary, and compiling it into a gigantic, auto-playing PowerPoint presentation, we sent it off to our senior leadership. We were also able to present in-person for a selected audience of design, PM, and development partners.

I’ve attempted to capture that presentation below, in a more browser-friendly manner. Each slide has an audio commentary with the talking points. It’s a little janky, hope you can make due.



What if there was a thorough, unbiased, & inclusive source of knowledge that helped parents manage their kids’ crucial developmental milestones? A super-smart guardian angel that’s got your back.


Maya just turned seven. Super-energetic! Very social at school & in the neighborhood. Loved piano on Monday, Ponyo on Tuesday, on Wednesday she’ll fall in love with soccer. Smart girl—sometimes struggles with maintaining focus.

Avo (4:47)


Bing Intelligence and Knowledge Services as the fabric for unifying a multitude of life experiences


Contextual notifications

Knowledge graph

Complex task completion


Emotional intelligence

Internet of things

Human as hero

Thank you.

Our team: Brad Singley, Brian Morris, Brook Durant, EJ Lee, Eric Schuh, and Sunny Park.

Special thanks to our wonderful actors Flora, Steven Heller, and Jennifer Wallace, and our resident audio engineer, Gary Spradling.

* * *


Search IngressProduct design

HeaderProduct design

Query FormulationProduct design

Zo, Emoji That SongProduct design


AutopilotPoster design

Real EstateProduct design

OSD logoBrand identity

Breaking NewsProduct design

.... .- ...- . .- -. .. -.-. . -.. .- -.--