April 12, 2017 · DRAFT

From search box to success in no time flat.

Query formulation and auto-suggest

BEFORE AFTER 

Definition

Query formulation (Q.F.), in general, is the nomenclature for the process of retrieving information from a system. So, technically, when you go to the library and ask the librarian to show you where all the Eva Hesse books are, you have just successfully formulated a query—the old-school way. Or, consider a book’s index or table of contents. Both are tools to help people navigate a large corpus of information effectively, enabling readers to get from Point A to B. People designing search experiences are focussed on easing this tension as well by providing relevant suggestions with accessible feedback in order for people to reach the ideal destination.

It’s worth noting that in our office, QF and Auto-suggest are routinely used interchangeably, where in fact, auto-suggest is a tool to make it easier to formulate queries—like the librarian. 

  1. au·to–sug·ges·t
    noun \ ˌȯ-tō-sə(g)-ˈjest\
    : a feature found in many computer programs (such as search engines and text messaging apps) that analyzes a user’s typing and presents a list of words and phrases that can be selected and entered based on history, context, and the first few typed letters
    // Autocomplete, as the name suggests, is the feature that completes a partial search query by matching the character string that the user types. Auto-suggest is the clever little feature that makes it seem like Google is nesting in your brain, predicting what you want to search seemingly before you even know it yourself. —Louise Merry
  1. que·ry
    noun
     \ˈkwir-ē, ˈkwer-ē\
    1  :  question, inquiry
    // I have a query about my order.
    2  :  a question in the mind: doubt
    // The librarian responded to my query.

  2. que·ry
    transitive verb \ˈkwir-ē, ˈkwer-ē\
    1  : to ask questions of especially with a desire for authoritative information
    //“can I buy a ticket?” he queried
    2  :  to ask questions about especially with an indication of doubt
    // They conducted a survey in which several hundred people were queried about their dietary habits.
Query formulation user flow diagram

Getting started

Core Search team logo designed by Mr Brian Morris

Core Search

First things first, design a new logo for the team. The positive-reverse magnifying glass, or spy glass, not only cut a mold for a "C," but the circle in the center denotes "core."

After another organizational re-shuffling, I landed on the Core Search design team. This team was responsible for the algorithmic results, ads, global headers and footers, overall framework for the results page, and query formulation—basically everything except for the structured data and instant answers. The product team for QF was also in flux—three different product managers in a month and the development team was shifting from the London office to Bellevue.

As the new design lead, I inherited an existing backlog of items—with limited context. Initially, the two most important items were:

  1. Feature parity with Google’s Segment links on Zero input (the entry points for Weather, Sports, etc.)
  2. Allowing people to scope their search through a variety of verticals (Images, Videos, or News)
170417-m-01-Google-ZI

1a. Google: Zero input

170417-m-01-Bing-ZI

1b. Proposed: Zero input

170417-m-01-Bing-Scopes

2. Proposed: Inline scope controls

After checking off a few items on said backlog, I bought myself some time to investigate and get a better understanding of the behavior and processes working under the hood. My initial revelation was that many of us had mental models that QF was a single canvas with a few states. And, sure, maybe I’m splitting semantic hairs, but what became clear to me was that formulating queries was a collection of many, oftentimes fleeting, canvases. There are, indeed, a few canonical states worthy of singling out and giving proper names to:

  1. Zero input is when the search box is empty; the user has not yet specified a query but has given the system a signal of intent by clicking on the search box. There’s also a post-query zero input state which I might get to later.
  2. Active querying or Active QF is when you’re actively inputting into the system. Most of the time this means each of those keystrokes when typing into the search box. Active QF also includes alternate input modalities as well as multi-modal inputs: using microphones, cameras, or files to use voice or image data to formulate your query.
Bing Query formulation user flow

Bing Query formulation user flow

The other realization I had, which wasn’t really that actionable but fascinating nonetheless, was that all of these tiny pieces that fit into this experience were atomic representations of individual results pages. So, every suggestion, trending news story, misspelled or incomplete query—anything in that search box was going to be translated into a results page. So, it was critical we do everything possible to help people come up with high-fidelity queries.

QF: Atomic Results pages

Atomic elements of query formulation

Rhea Loader
Rhea

I’m extremely grateful that I had a manager who afforded me the support and operational space to dig deeper than what was originally asked of our team. Taking a bit more time upfront to build a product narrative paid off big time for me, the team, the product, and the company—both in exceeding company monetization goals as well as crafting useful and engaging experiences for people who used our search engine. She deserves a lot of credit for this kind of leadership and direction.

General design principles

Once I understood the space better, I dug deeper into the framework. Historically, this part of the product didn’t have a dedicated designer and got bounced around the studio or was updated by the developers. There was no existing source file, so I took a lot of screenshots and inspected a lot of code on different devices. By the way, I love constructing my own source files as it always seems to provide good insight into the structure.  In any case, there were a lot of general design principles that I knew would bring this thing back to life. I began to think of this effort as a Baseline UX, fortifying the foundation so the team would have a stronger source to build onto:

  1. Unify search boxes: size, position, and visual treatment
  2. Unify icons: style, stroke, color, size, and alignment
  3. Unify fonts: font-family, size, color, weight, hit-highlight treatment
  4. Unify vertical spacing for clarity and better targets
  5. Remove as many borders that tend to house individual elements, especially the horizontal lines between list elements

Readability

The readability of any text can be impacted by a number of factors. Consider the wayfinding sign below. Not only is there semantic tension regarding the proximity of the names of these places, but there’s also multiple places to look for the directional arrows, which further complicates the sign.

Each one of the following rows represents individual experiments we ran in order to validate with data that these design principles were indeed helping people read the list efficiently. The left column is the control, the right is the treatment (which ended up shipping in the product).

Bad readability

Good readability

Typography: Weight and color

  1. The prefix, in this case, “new y,” was styled as a lighter grey (#666) and a normal weight (400). The hit-highlighting (the other part of the suggestion) was pure black (#000) and Semibold (500), which only rendered correctly in PC environments due to Segoe UI font availability.
  2. With the treatment, we used a single color, #444. With hit-highlighting, we used a higher contrast font-weight which also rendered the same way across all platforms—bold/600.

Number of suggestions

  1. We all think we want more choices, but in this case, it takes longer to process and tends to project a little less intelligence with the repetition of the prefix.
  2. After analyzing the data, we found most people were selecting the top 3–4 suggestions most of the time, with the latter suggestions getting very little engagement. Having fewer suggestions made it easier and quicker to parse, as well as good at signaling a re-query in the case of no good suggestions.

Vertical spacing

  1. Tight vertical spacing (call it leading, list item height, line-height, etc.) not only makes it more difficult to scan or read, but it also makes it a difficult target to hit on touch-based interfaces.
  2. My hypothesis was to go with a 40-pixel height since the search box was the same height. But after testing several treatments we discovered that a 36-pixel height was the sweet spot.
QF Baseline UX composition

Early schematic of Baseline UX

Secondary goals

The Visual Systems team was also working on a brand refresh at the time—rounding corners, introducing shadows, and figuring out how much teal is too much. I ended up partnering closely with this team throughout the Baseline UX initiative. Without the hard work and support from these talented individuals, I couldn’t have accomplished most of this work, and this page would be a lot shorter.

I wanted to influence the typography (size, color, weight, etc.), minimal touch target sizes (40-pixels), and to begin aligning icons based on an underlying wrapper rather than visually aligning each one. As I began to evaluate all three platforms—desktop, tablet, and mobile—I also saw an opportunity to unify how we presented the search box across the board (which also led to taking on the header redesign). It keeps the box familiar across platforms, and it would also set the stage for designing a responsive search experience in the future.

Becky Johnson
Becky
Bobby Bernethy
Bobby
David Gardner
David
Janny Woo
Janny
170417-searchboxes-mobile-03

BEFORE: Audit of mobile search boxes

QF: Search box unification across desktop, tablet, and mobile platforms

AFTER: Unified, cross-platform search boxes

Let’s get this party started…

As I’m figuring these things out and putting it all together, I’m leaning on other groups of designers and researchers for input and validation. Then I find out this project has a new development team and they are in the same building. So, right before the holidays, I share it with the new developer lead. He invites his boss into the room. They’re both smiling at me and I’m getting super excited because I realize this work is actually going to get done!

For the next year, we tested, deployed, and shipped about 95% of the items on my original list. And when I say we, I’m mostly referring to the most bad-assed, talented, hard-working, and motivated dev team I’ve ever worked with at Microsoft!

Saulo Santos
Saulo
Rahul Lal
Rahul
Marcelo de Barros
Marcelo
Nicke Peine
Nick

32 frames of shipped experiments

QF: Baseline UX (composition of Desktop and mobile)

Baseline UX composite

Chasing rainbows.

An array of useful and engaging auto-suggest features

Time of day

QF feature: Time of day signal (7:00 AM)

7:00 AM

QF feature: Time of day signal (7:00 PM)

7:00 PM

Event signal

QF feature: Event signal (Before the event)

Before the event

QF feature: Event signal (During the event)

During the event

QF feature: Event signal (After the event)

After the event

Session signal

QF feature: Session signal (Default suggestions)

Default suggestions

QF feature: Session signal (After you searched for

Suggestions after you previously searched for prada handbags

Moments in time

QF feature: Moments in time (Trending now on Zero input) and Emmys results page with carousel

Special events in Trending 

We began promoting special events by listing the event query with a unique icon. The core principle was to steer people to results pages containing specialized structured data designed by our Answer teams. This featurette ended up helping showcase a lot of the outstanding work from the Instant Answer teams—everything from Sports to Entertainment and more. 

This example shows a sample of which events we covered. Highlighted is the click-through experience of the Emmys result page with a custom carousel.

Targeted queries

This was an easter egg designed for pure delight. This particular one celebrates various Pride parades around the world. We targeted a whitelist of queries to light up a custom QF experience. As you type for one of the queries, like pride parades, the auto-suggest menu changed to the colors of the Pride flag. Our team’s general directive was to never get in the way of anyone’s search. But this was an exception to the rule and our feature was well-received company-wide. Most importantly, a lot of our fans really seemed to love it as well.

We also did this for Dota2 The International.

QF feature: Moments in time (Targeted query, Pride 2018 special events experience)

Gestures

QF feature: Gestures (Deleting search history)

Deleting search history

QF feature: Gestures (Saving for later and/or Feedback)

Saving for later and/or Feedback

QF feature: Gestures (Inline actions)

Inline actions

Next-word

Instant answers

Display entities

Flagging inappropriate suggestions

Illustration of Eva Hesse's Untitled, 1966 from MoMA