Inside Search: Video! The search quality meeting, uncut (annotated)

Inside Search

The official Google Search blog

Video! The search quality meeting, uncut (annotated)

March 12, 2012

how search worksalgorithm change

Even relatively subtle changes get intense scrutiny by our search evaluation and ranking teams. The specific change discussed in this video improves spelling suggestions for searches with more than 10 words and it impacts only .1% of our traffic. Still, you can see the scrutiny and thoughtfulness that goes into approving this change.

Every change has a dedicated search quality analyst assigned to study the impact. This analyst is not part of the engineering team building the change, but instead offers a separate opinion on whether the change is good for users.

The search team relies heavily on the results of experimental data to make decisions. During the meeting, we rely on detailed analyst reports including the results of click evaluations and side-by-side experiments. These reports can sometimes be more than 25 pages long.

Launch reports include specific examples to illustrate broader trends in the data. Rather than manually change one example, our engineers look for algorithmic ways to improve millions of queries.

Search algorithm improvements often rely on and impact many different systems, so engineers with expertise in different areas all need to come together to make the best decision for the user, balancing all the tradeoffs involved (relevance, spam, latency, cost, language impact, etc.)