- •Releasing Tasks
- •Introduction to Page Quality Rating
- •Understanding Webpages and Websites
- •Important Definitions
- •What is the Purpose of a Webpage?
- •Your Money or Your Life (ymyl) Pages
- •Understanding Webpage Content
- •2.4.1 Identifying the Main Content (mc)
- •2.4.2 Identifying the Supplementary Content (sc)
- •2.4.3 Identifying Advertisements/Monetization (Ads)
- •2.4.4 Summary of the Parts of the Page
- •Understanding the Website
- •2.5.1 Finding the Homepage
- •2.5.2 Finding Who is Responsible for the Website and Who Created the Content on the Page
- •2.5.3 Finding About Us, Contact Information, and Customer Service Information
- •Website Maintenance
- •Website Reputation
- •2.7.1 Reputation Research
- •2.7.2 Sources of Reputation Information
- •2.7.3 Customer Reviews of Stores/Businesses
- •2.7.4 How to Search for Reputation Information
- •2.7.5 What to Do When You Find No Reputation Information
- •Overall Page Quality Rating Scale
- •High Quality Pages
- •Characteristics of High Quality Pages
- •A Satisfying Amount of High Quality Main Content
- •A High Level of Expertise/Authoritativeness/Trustworthiness (e-a-t)
- •Positive Reputation
- •Helpful Supplementary Content
- •Functional Page Design
- •A Satisfying Amount of Website Information
- •A Well Cared For and Maintained Website
- •Examples of High Quality Pages
- •Highest Quality Pages
- •Very High Quality mc
- •Very High Level of e-a-t
- •Very Positive Reputation
- •Examples of Highest Quality Pages
- •6.0 Low Quality Pages
- •Low Quality Main Content
- •Unsatisfying Amount of Main Content
- •Lacking Expertise, Authoritativeness, or Trustworthiness (e-a-t)
- •Negative Reputation
- •Characteristics Which May Be Evidence of Low Quality
- •6.5.1 Unhelpful or Distracting Supplementary Content
- •6.5.2 Lacking Supplementary Content
- •6.5.3 Poor Page Design
- •6.5.4 Lacking Care and Maintenance
- •Unsatisfying Amount of Information about the Website
- •Examples of Low Quality Pages
- •7.0 Lowest Quality Pages
- •Harmful or Malicious Pages
- •Lack of Purpose Pages
- •Deceptive Pages
- •7.3.1 Deceptive Page Purpose
- •7.3.2 Deceptive Page Design
- •7.3.3 Sneaky Redirects
- •Lowest Quality Main Content
- •7.4.1 No Main Content
- •7.4.2 “Keyword Stuffed” Main Content
- •7.4.3 Gibberish or Meaningless Main Content
- •7.4.4 Automatically-Generated Main Content
- •7.4.5 Copied Main Content
- •7.4.6 More About Copied Content
- •7.4.7 How to Determine if Content is Copied
- •No Website Information
- •Highly Untrustworthy, Unreliable, Unauthoritative, Inaccurate, or Misleading
- •Abandoned Websites or Spammed Pages on a Website
- •Extremely Negative or Malicious Reputation
- •Examples of Lowest Quality Pages
- •8.0 Medium Quality Pages
- •Examples of Medium Quality Pages
- •9.0 Page Quality Rating: Important Considerations
- •Instructions for Rating Page Quality Tasks
- •The Top Three pq Considerations
- •Page Quality Considerations for Specific Types of Pages
- •Ratings for Encyclopedia Pages
- •Ratings for Pages with Error Messages or No mc
- •Ratings for Forums and q&a pages
- •Page Quality Rating faQs
- •Understanding Mobile Users, Mobile Queries, and Mobile Results
- •Important Rating Definitions and Ideas
- •Understanding the Query
- •Task Location (Locale) and User Location
- •Queries with an Explicit Location
- •Queries with Multiple Meanings
- •Query Meanings Can Change Over Time
- •Understanding User Intent
- •Know and Know Simple Queries
- •Do and Device Action Queries
- •Website Queries
- •Visit-in-Person Queries and User Location
- •Queries with Multiple User Intents
- •Understanding Result Blocks
- •Web Search Result Block Examples
- •Special Content Result Block Examples
- •Device Action Result Block Examples
- •How Device Action Results are Displayed in Rating Tasks
- •Rating on Your Phone Issues
- •Rating Using the Needs Met Scale
- •Rating Result Blocks: Block Content and Landing Pages
- •Fully Meets (FullyM)
- •Examples of Fully Meets (FullyM) Result Blocks
- •Examples of Result Blocks that Cannot be Fully Meets
- •Highly Meets (hm)
- •Examples of Highly Meets (hm) Result Blocks
- •Moderately Meets (mm)
- •Examples of Moderately Meets (mm) Result Blocks
- •Slightly Meets (sm)
- •Examples of Slightly Meets (sm) Result Blocks
- •Fails to Meet (FailsM)
- •Examples of Fails to Meet (FailsM) Result Blocks
- •14.0 Rating Porn, Foreign Language, Didn’t Load, and Hard to Use Results
- •Porn Flag
- •Needs Met Rating for Porn Results
- •Needs Met Rating for Clear Non-Porn Intent Queries
- •Needs Met Rating for Possible Porn Intent Queries
- •Needs Met Rating for Clear Porn Intent Queries
- •Reporting Illegal Images
- •Foreign Language Flag
- •14.4.1 Using the Foreign Language Flag
- •14.4.2 Needs Met Ratings for Foreign Language Results
- •14.4.3 English Language Results
- •Didn’t Load Flag
- •14.5.1 Using the Didn’t Load Flag
- •14.5.2 Needs Met Rating for Didn’t Load Results
- •Hard to Use Flag
- •14.6.1 Using the Hard to Use Flag
- •The Relationship between e-a-t and Needs Met
- •Rating Queries with Multiple Interpretations and Intents
- •Rating Queries with Both Website and Visit-in-Person Intent
- •Specificity of Queries and Landing Pages
- •Needs Met Rating and Freshness
- •Misspelled and Mistyped Queries and Results
- •Misspelled and Mistyped Queries
- •19.2 Name Queries
- •19.3 Spelling Suggestion Result Blocks
- •21.0 Product Queries: Action (Do) vs. Information (Know) Intent
- •22.0 Rating Visit-in-Person Intent Queries
- •26.0 Needs Met Task Page Screenshot
- •27.0 Notes about Using the Needs Met Rating Interface
- •28.0 Using the “Report a Problem / Release this Task” Button
- •29.0 Reporting Results with Duplicate Landing Pages
- •Rater-Identified Duplicates
Highly Meets (hm)
A rating of Highly Meets is assigned to results that meet the need of many or most users. Highly Meets results are highly satisfying and a good “fit” for the query. In addition, they often have some or all of the following characteristics: high quality, authoritative, entertaining, and/or recent (e.g., breaking news on a topic).
A query can have many Highly Meets results.
Have high standards for using the Highly Meets rating. This is especially important for queries with many, many on-topic results.
Proprietary and Confidential – Copyright 2015 99
Examples of Highly Meets (hm) Result Blocks
|
Query, User Location, |
|
|
Result |
|
|
Rating |
|
|
Explanation |
|
|
|||||||||
|
User Intent |
|
|
|
|
|
|
|
|
||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Query: [trader joes] |
|
|
|
|
|
|
|
|
|
|
|
This result shows a complete |
|
|||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||||||
|
|
|
|
|
|
|
|
|
|
|
|
list of all three locations in the |
|
||||||||
|
User Location: |
|
|
|
|
|
|
|
|
|
|
|
Charlotte area, with |
|
|||||||
|
|
|
|
|
|
|
|
|
|
|
|
information that is especially |
|
||||||||
|
Charlotte, North |
|
|
|
|
|
|
|
|
|
|
|
helpful for users who want to |
|
|||||||
|
Carolina |
|
|
|
|
|
|
|
|
|
|
|
visit |
the store. |
|
||||||
|
User Intent: There are |
|
|
|
|
|
|
|
|
|
|
|
Note: This result block is not |
|
|||||||
|
two possible user |
|
|
|
|
|
|
|
|
|
|
|
Fully Meets because users |
|
|||||||
|
intents: most users |
|
|
|
|
|
|
|
|
|
|
|
who want to go to the website |
|
|||||||
|
probably want to |
visit |
a |
|
|
|
|
|
|
|
|
|
|
|
to see coupons, promotions, |
|
|||||
|
nearby location or go |
to |
|
|
|
|
|
|
|
|
|
|
|
etc. would have to see |
|
||||||
|
the website. |
|
|
|
|
|
|
|
|
|
|
|
additional results. |
|
|||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Query: [trader joes] |
|
|
|
|
|
|
|
|
|
|
|
This is the official website for |
|
|||||||
|
|
|
|
|
|
|
|
|
|
|
|
Trader Joe’s, which has |
|
||||||||
|
User Location: |
|
|
|
|
|
|
|
|
|
|
|
highly authoritative |
|
|||||||
|
|
|
|
|
|
|
|
|
|
|
|
information on the |
|
||||||||
|
Charlotte, North |
|
|
|
|
|
|
|
|
|
|
|
businesses’ stores, |
|
|||||||
|
Carolina |
|
|
|
|
|
|
|
|
|
|
|
promotions, recipes, news, |
|
|||||||
|
User Intent: There are |
|
|
|
|
|
|
|
|
|
|
|
etc. |
|
|||||||
|
|
|
|
|
|
|
|
|
|
|
|
Note: This result block is not |
|
||||||||
|
two possible user |
|
|
|
|
|
|
|
|
|
|
|
|
||||||||
|
intents: most users |
|
|
|
|
|
|
|
|
|
|
|
Fully Meets because users |
|
|||||||
|
probably want to |
visit |
a |
|
|
|
|
|
|
|
|
|
|
|
who want information to |
visit |
|
|
|||
|
nearby location or go |
to |
|
|
|
|
|
|
|
|
|
|
|
the store have to do a bit of |
|
||||||
|
the website. |
|
|
|
|
|
|
|
|
|
|
|
work to find that information— |
|
|||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
it is not immediately available. |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||||
|
Query: [museum of |
|
|
|
|
|
|
|
|
|
|
|
The result block shows |
|
|||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||||||
|
modern art] |
|
|
|
|
|
|
|
|
|
|
|
information about the |
|
|||||||
|
User Location: |
|
|
|
|
|
|
|
|
|
|
|
museum, and is especially |
|
|||||||
|
|
|
|
|
|
|
|
|
|
|
|
helpful for users who want to |
|
||||||||
|
Manhattan, New York |
|
|
|
|
|
|
|
|
|
|
|
visit |
the museum. |
|
||||||
|
User Intent: There are |
|
|
|
|
|
|
|
|
|
|
|
Note: This result block is not |
|
|||||||
|
two possible user |
|
|
|
|
|
|
|
|
|
|
|
Fully Meets— even though |
|
|||||||
|
intents: most users |
|
|
|
|
|
|
|
|
|
|
|
there is a website link, it is not |
|
|||||||
|
probably want to |
visit |
|
|
|
|
|
|
|
|
|
|
|
|
very satisfying for users who |
|
|||||
|
the museum (note |
that |
|
|
|
|
|
|
|
|
|
|
|
want to go to the website |
|
||||||
|
the user is located in |
|
|
|
|
|
|
|
|
|
|
|
because it is not as |
|
|||||||
|
NYC where the museum |
|
|
|
|
|
|
|
|
|
|
|
informative as a web search |
|
|||||||
|
is located) |
|
|
|
|
|
|
|
|
|
|
|
result block with a title, |
|
|||||||
|
or go to the website. |
|
|
|
|
|
|
|
|
|
|
|
snippet, and URL. |
|
|||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Proprietary and Confidential – Copyright 2015 100
|
Query, User Location, |
|
|
Result |
|
|
Rating |
|
|
Explanation |
|
|
|||||||
|
User Intent |
|
|
|
|
|
|
|
|
||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Query: [museum of |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||
|
modern art] |
|
|
|
|
|
|
|
|
|
This is the museum’s official |
|
|||||||
|
User Location: |
|
|
|
|
|
|
|
|
|
website and has very helpful |
|
|||||||
|
|
|
|
|
|
|
|
|
|
information about exhibits, |
|
||||||||
|
Manhattan, New York |
|
|
|
|
|
|
|
|
|
membership, the store, etc. |
|
|||||||
|
User Intent: There are |
|
|
|
|
|
|
|
|
|
Note: This result block is not |
|
|||||||
|
two possible user |
|
|
|
|
|
|
|
|
|
Fully Meets because users |
|
|||||||
|
intents: most users |
|
|
|
|
|
|
|
|
|
who want information to |
visit |
|
|
|||||
|
probably want to |
visit |
|
|
|
|
|
|
|
|
|
|
the museum have to do |
a bit |
|
|
|||
|
the museum (note |
that |
|
|
|
|
|
|
|
|
|
of work to find that |
|
||||||
|
the user is located in |
|
|
|
|
|
|
|
|
|
information—it is not |
|
|||||||
|
NYC where the museum |
|
|
|
|
|
|
|
|
|
immediately available. |
|
|||||||
|
is located) or go to the |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||
|
website. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The query is the name of a |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
car dealership in New Jersey, |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
which is the only dealership |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
with that name in the user |
|
||||
|
Query: [prestige bmw] |
|
|
|
|
|
|
|
|
|
location (Ramsey, New |
|
|||||||
|
|
|
|
|
|
|
|
|
|
Jersey). The result block |
|
||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
User Location: |
|
|
|
|
|
|
|
|
|
shows information about the |
|
|||||||
|
|
|
|
|
|
|
|
|
|
dealership, and is especially |
|
||||||||
|
Ramsey, New Jersey |
|
|
|
|
|
|
|
|
|
|
||||||||
|
|
|
|
|
|
|
|
|
|
helpful for users who want to |
|
||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
User Intent: There are |
|
|
|
|
|
|
|
|
|
visit |
the business. |
|
||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||
|
two possible user |
|
|
|
|
|
|
|
|
|
Note: This result block is not |
|
|||||||
|
intents: most users |
|
|
|
|
|
|
|
|
|
|
||||||||
|
|
|
|
|
|
|
|
|
|
Fully Meets— even though |
|
||||||||
|
probably want to |
visit |
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
there is a website link, it is not |
|
|||||||
|
the car dealership or go |
|
|
|
|
|
|
|
|
|
|
||||||||
|
|
|
|
|
|
|
|
|
|
very satisfying for users who |
|
||||||||
|
to the website. |
|
|
|
|
|
|
|
|
|
|
||||||||
|
|
|
|
|
|
|
|
|
|
want to go to the website |
|
||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
because it is not as |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
informative as a web search |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
result block with a title, |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
snippet, and URL. |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||
|
Query: [prestige bmw] |
|
|
|
|
|
|
|
|
|
This is the car dealership’s |
|
|||||||
|
|
|
|
|
|
|
|
|
|
official website and has very |
|
||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
User Location: |
|
|
|
|
|
|
|
|
|
helpful information about |
|
|||||||
|
|
|
|
|
|
|
|
|
|
inventory, financing, etc. |
|
||||||||
|
Ramsey, New Jersey |
|
|
|
|
|
|
|
|
|
|
||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||
|
User Intent: There are |
|
|
|
|
|
|
|
|
|
Note: This result block is not |
|
|||||||
|
|
|
|
|
|
|
|
|
|
Fully Meets because users |
|
||||||||
|
two possible user |
|
|
|
|
|
|
|
|
|
|
||||||||
|
|
|
|
|
|
|
|
|
|
who want information to |
visit |
|
|
|
|||||
|
intents: most users |
|
|
|
|
|
|
|
|
|
|
|
|||||||
|
|
|
|
|
|
|
|
|
|
the dealership have to do a |
|
||||||||
|
probably want to |
visit |
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
bit of work to find that |
|
|||||||
|
the car dealership or go |
|
|
|
|
|
|
|
|
|
|
||||||||
|
|
|
|
|
|
|
|
|
|
information—it is not |
|
||||||||
|
to the website. |
|
|
|
|
|
|
|
|
|
|
||||||||
|
|
|
|
|
|
|
|
|
|
immediately available. |
|
||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Proprietary and Confidential – Copyright 2015 101
|
Query, User Location, |
|
Result |
|
|
Rating |
|
|
Explanation |
|
|
||||||
|
User Intent |
|
|
|
|
|
|
|
|
||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
There is only one library in |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
the user location (Belmont, |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
Massachusetts). The result |
|
||||
|
Query: [belmont |
library] |
|
|
|
|
|
|
|
|
|
block shows information |
|
||||
|
|
|
|
|
|
|
|
|
|
about the library, and is |
|
||||||
|
User Location: |
|
|
|
|
|
|
|
|
|
|
especially helpful for users |
|
||||
|
|
|
|
|
|
|
|
|
|
|
who want to |
visit |
the location. |
|
|||
|
Belmont, Massachusetts |
|
|
|
|
|
|
|
|
|
Note: This result block is not |
|
|||||
|
User Intent: Find |
|
|
|
|
|
|
|
|
|
|
||||||
|
|
|
|
|
|
|
|
|
|
Fully Meets— even though |
|
||||||
|
information about this |
|
|
|
|
|
|
|
|
|
there is a website link, it is not |
|
|||||
|
library or go to the |
|
|
|
|
|
|
|
|
|
very satisfying for users who |
|
|||||
|
website. |
|
|
|
|
|
|
|
|
|
|
want to go to the website |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
because it is not as |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
informative as a web search |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
result block with a title, |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
snippet, and URL. |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
There is only one library in |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
the user location (Belmont, |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
California). The result block |
|
||||
|
Query: [belmont |
library] |
|
|
|
|
|
|
|
|
|
shows information about the |
|
||||
|
|
|
|
|
|
|
|
|
|
library, and is especially |
|
||||||
|
User Location: |
|
|
|
|
|
|
|
|
|
|
helpful for users who want to |
|
||||
|
|
|
|
|
|
|
|
|
|
|
visit |
the location. |
|
||||
|
Belmont, California |
|
|
|
|
|
|
|
|
|
Note: This result block is not |
|
|||||
|
User Intent: Find |
|
|
|
|
|
|
|
|
|
|
||||||
|
|
|
|
|
|
|
|
|
|
Fully Meets— even though |
|
||||||
|
information about this |
|
|
|
|
|
|
|
|
|
there is a website link, it is not |
|
|||||
|
library or go to the |
|
|
|
|
|
|
|
|
|
very satisfying for users who |
|
|||||
|
website. |
|
|
|
|
|
|
|
|
|
|
want to go to the website |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
because it is not as |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
informative as a web search |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
result block with a title, |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
snippet, and URL. |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The user intent is too broad to |
|
||||
|
Query: [decemberists] |
|
|
|
|
|
|
|
|
|
have a Fully Meets result. |
|
|||||
|
|
|
|
|
|
|
|
|
|
However, this is The |
|
||||||
|
User Location: Long |
|
|
|
|
|
|
|
|
|
Decemberists’ official website |
|
|||||
|
|
|
|
|
|
|
|
|
|
and has a lot of content |
|
||||||
|
Beach, California |
|
|
|
|
|
|
|
|
|
|
including tour dates, music |
|
||||
|
User Intent: Learn |
|
|
|
|
|
|
|
|
|
videos, the latest album, etc. |
|
|||||
|
|
|
|
|
|
|
|
|
|
The result is uniquely |
|
||||||
|
more about the band, |
|
|
|
|
|
|
|
|
|
authoritative and most users |
|
|||||
|
The Decemberists. |
|
|
|
|
|
|
|
|
|
would be very satisfied by this |
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
website. |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
The user intent is too broad to |
|
||||
|
Query: [celine dion] |
|
|
|
|
|
|
|
|
|
have a Fully Meets result. |
|
|||||
|
|
|
|
|
|
|
|
|
|
However, this is Celine Dion's |
|
||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
User Location: |
|
|
|
|
|
|
|
|
|
|
official website and has a lot |
|
||||
|
|
|
|
|
|
|
|
|
|
|
of content including recent |
|
|||||
|
Bellevue, Washington |
|
|
|
|
|
|
|
|
|
|
||||||
|
|
|
|
|
|
|
|
|
|
news, show information, |
|
||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
User Intent: Learn |
|
|
|
|
|
|
|
|
|
music, photos, videos, etc. |
|
|||||
|
|
|
|
|
|
|
|
|
|
The result is uniquely |
|
||||||
|
more about the singer |
|
|
|
|
|
|
|
|
|
|
||||||
|
|
|
|
|
|
|
|
|
|
authoritative and most users |
|
||||||
|
Celine Dion. |
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
would be very satisfied by this |
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
website. |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Proprietary and Confidential – Copyright 2015 102
|
Query, User Location, |
|
|
|
Result |
|
Rating |
|
|
|
Explanation |
|
|
|
User Intent |
|
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Query: [shaun the |
|
|
|
sheep movie] |
|
The user intent is too broad to |
|
User Location: |
|
have a Fully Meets result. |
|
|
However, this interactive |
|
|
Mountain View, |
|
block has a large amount of |
|
California |
|
helpful content, showing a |
|
User Intent: |
|
summary of the movie, |
|
|
nearby showtimes, ratings, |
|
|
Find a movie summary, |
|
cast information, etc.—most |
|
showtimes, cast |
|
users would be very satisfied |
|
information, etc. |
|
by this result. |
|
|
|
|
|
Query: [manresa |
The LP for this web result has |
|
reviews] |
|
|
122 reviews for the |
|
|
|
|
|
User Location: San |
restaurant, and is easy to use |
|
on a mobile phone. The first |
|
|
Jose, California |
|
|
three reviews show |
|
|
|
|
|
User Intent: Find |
automatically, and then you |
|
can click to see more. This |
|
|
reviews for this |
|
|
result would be very helpful |
|
|
restaurant located in |
|
|
for many or most users. |
|
|
Los Gatos, California. |
|
|
|
|
Proprietary and Confidential – Copyright 2015 103
|
Query, User Location, |
|
|
Result |
|
|
Rating |
|
|
Explanation |
|
|
|
User Intent |
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Query: [broadway |
|
|
|
|
|
|
|
The LP for this web result |
|
||
|
tickets] |
|
|
|
|
|
|
|
offers many Broadway tickets |
|
||
|
User Location: New |
|
|
|
|
|
|
|
for sale, from a highly |
|
||
|
|
|
|
|
|
|
|
trustworthy source. While |
|
|||
|
York, New York |
|
|
|
|
|
|
|
you have to zoom in and |
|
||
|
User Intent: Research |
|
|
|
|
|
|
|
scroll to navigate the site, this |
|
||
|
|
|
|
|
|
|
|
result would be very helpful |
|
|||
|
prices and/or purchase |
|
|
|
|
|
|
|
for many or most users |
|
||
|
tickets to a Broadway |
|
|
|
|
|
|
|
because it is so highly |
|
||
|
show. |
|
|
|
|
|
|
|
trustworthy. |
|
||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The LP for this mobile-friendly |
|
|
|
|
|
|
|
|
|
|
|
|
web result offers many kids |
|
|
|
Query: [kids backpacks] |
|
|
|
|
|
|
|
backpacks for sale. The |
|
||
|
|
|
|
|
|
|
|
company is well known for |
|
|||
|
User Location: New |
|
|
|
|
|
|
|
manufacturing and selling |
|
||
|
|
|
|
|
|
|
|
high quality backpacks for |
|
|||
|
York, New York |
|
|
|
|
|
|
|
kids of different ages. There |
|
||
|
User Intent: Research |
|
|
|
|
|
|
|
is a lot of product information |
|
||
|
|
|
|
|
|
|
|
and many user reviews for |
|
|||
|
prices and/or purchase |
|
|
|
|
|
|
|
each backpack, in addition to |
|
||
|
backpacks for kids. |
|
|
|
|
|
|
|
filter options that are easy to |
|
||
|
|
|
|
|
|
|
|
|
|
use on a phone. This result |
|
|
|
|
|
|
|
|
|
|
|
|
would be very helpful for |
|
|
|
|
|
|
|
|
|
|
|
|
many or most users. |
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
Query: [poison ivy] |
|
|
|
|
|
|
|
The LP for this mobile-friendly |
|
||
|
User Location: Atlanta, |
|
|
|
|
|
|
|
web result is part of an |
|
||
|
|
|
|
|
|
|
|
authoritative government |
|
|||
|
Georgia |
|
|
|
|
|
|
|
website and provides a lot of |
|
||
|
User Intent: Find |
|
|
|
|
|
|
|
helpful information about |
|
||
|
|
|
|
|
|
|
|
poison ivy. This result would |
|
|||
|
pictures of poison ivy |
|
|
|
|
|
|
|
be very helpful for many or |
|
||
|
plants, information |
|
|
|
|
|
|
|
most users. A few or some |
|
||
|
about how to treat |
|
|
|
|
|
|
|
users may wish to see |
|
||
|
poison ivy, etc. |
|
|
|
|
|
|
|
additional results. |
|
||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Query: [michael jordan] |
|
|
|
|
|
|
|
|
|
|
|
|
User Location: Boston, |
|
|
|
|
|
|
|
The LP of this web result is a |
|
||
|
Massachusetts |
|
|
|
|
|
|
|
mobile-friendly Wikipedia |
|
||
|
User Intent: Find |
|
|
|
|
|
|
|
article about Michael Jordan. |
|
||
|
|
|
|
|
|
|
|
This result would be helpful |
|
|||
|
information, news, |
|
|
|
|
|
|
|
for many or most users. |
|
||
|
images, etc. about |
|
|
|
|
|
|
|
|
|
|
|
|
Michael Jordan. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Query: [who is the |
|
|
|
|
|
|
|
This is a specific Know |
|
||
|
chancellor of germany] |
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
Simple query for the name of |
|
|||
|
|
|
|
|
|
|
|
|
|
|
||
|
User Location: New |
|
|
|
|
|
|
|
the current Chancellor of |
|
||
|
|
|
|
|
|
|
|
Germany. While the answer |
|
|||
|
York, New York |
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
is in the description of the |
|
|||
|
|
|
|
|
|
|
|
|
|
|
||
|
User Intent: The user |
|
|
|
|
|
|
|
web result, it is not displayed |
|
||
|
|
|
|
|
|
|
|
prominently in a way that is |
|
|||
|
wants to find the name |
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
very easy for mobile users to |
|
|||
|
of the current |
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
read. |
|
|||
|
Chancellor of Germany. |
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
|
|
|
|
Proprietary and Confidential – Copyright 2015 104
|
Query, User Location, |
|
|
Result |
|
Rating |
|
|
Explanation |
|
|
|||||
|
User Intent |
|
|
|
|
|
|
|
||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Users are looking for good or |
|
|
|
Query: [dance video] |
|
|
|
|
|
|
|
|
|
|
|
entertaining dance videos |
|
||
|
|
|
|
|
|
|
|
|
|
|
|
and there are many options |
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
||||
|
User Location: New |
|
|
|
|
|
|
|
|
|
|
|
online. |
|
||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
York, New York |
|
|
|
|
|
|
|
|
|
|
|
There are many on-topic |
|
||
|
User Intent: Find a |
|
|
|
|
|
|
|
|
|
|
|
results for this query and this |
|
||
|
|
|
|
|
|
|
|
|
|
|
|
is one very popular video of a |
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
||||
|
dance video to watch. |
|
|
|
|
|
|
|
|
|
|
|
comedian demonstrating |
|
||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
dance styles from previous |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
decades. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Query: [purple |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
coneflowers] |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
User Location: New |
|
|
|
|
|
|
|
|
|
|
|
This block contains many |
|
||
|
|
|
|
|
|
|
|
|
|
|
|
helpful images, and the |
|
|||
|
York, New York |
|
|
|
|
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
landing page shows even |
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
User Intent: Find |
|
|
|
|
|
|
|
|
|
|
|
more images—it Highly |
|
||
|
|
|
|
|
|
|
|
|
|
|
|
Meets the user intent. |
|
|||
|
images of or information |
|
|
|
|
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
about purple |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
coneflowers. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This is a broad query for a |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
city. Different users may be |
|
|
|
Query: [seattle, |
|
|
|
|
|
|
|
|
|
|
|
looking for different things. |
|
||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
washington] |
|
|
|
|
|
|
|
|
|
|
|
The result contains a wide |
|
||
|
User Location: |
|
|
|
|
|
|
|
|
|
|
|
variety of helpful information |
|
||
|
|
|
|
|
|
|
|
|
|
|
|
such as a map, weather |
|
|||
|
Arlington, Texas |
|
|
|
|
|
|
|
|
|
|
|
information, several points of |
|
||
|
User Intent: Find |
|
|
|
|
|
|
|
|
|
|
|
interest, and has an option to |
|
||
|
|
|
|
|
|
|
|
|
|
|
|
see more information |
|
|||
|
information, news, |
|
|
|
|
|
|
|
|
|
|
|
containing nearby events and |
|
||
|
maps, etc. related to |
|
|
|
|
|
|
|
|
|
|
|
facts about the city. |
|
||
|
Seattle, Washington. |
|
|
|
|
|
|
|
|
|
|
|
Moderately Meets+ to |
|
||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Highly Meets is an |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
appropriate rating for this |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
result. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Query: [invasive |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
species] |
|
|
|
|
|
|
|
|
|
|
|
The result block has a good |
|
||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
User Location: St. |
|
|
|
|
|
|
|
|
|
|
|
summary of what invasive |
|
||
|
|
|
|
|
|
|
|
|
|
|
|
species are, but some users |
|
|||
|
Louis, Missouri |
|
|
|
|
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
would probably want to see |
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
User Intent: Find more |
|
|
|
|
|
|
|
|
|
|
|
more information like |
|
||
|
|
|
|
|
|
|
|
|
|
|
|
examples, impact, etc. |
|
|||
|
information about |
|
|
|
|
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
invasive species. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Proprietary and Confidential – Copyright 2015 105
|
Query, User Location, |
|
|
Result |
|
|
Rating |
|
|
Explanation |
|
|
|||||||
|
User Intent |
|
|
|
|
|
|
|
|
||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
For users in Sunnyvale, |
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
California, the query |
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
[turmeric] could have two |
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
different interpretations: a |
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
popular restaurant named |
|
|||
|
Query: [turmeric] |
|
|
|
|
|
|
|
|
|
|
|
|
Turmeric or the spice |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
turmeric. Because the user |
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||
|
User Location: |
|
|
|
|
|
|
|
|
|
|
|
|
location for this query is in |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
Sunnyvale, many mobile |
|
|||||
|
Sunnyvale, California |
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
|
users are probably looking for |
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||
|
User Intent: Find |
|
|
|
|
|
|
|
|
|
|
|
|
the restaurant. However, it is |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
not completely clear because |
|
|||||
|
information about the |
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
|
some users may want to find |
|
|||||
|
spice or the restaurant |
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
|
information about the spice. |
|
|||||
|
in Sunnyvale. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
|
This result about the |
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
restaurant has satisfying |
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
information for users who |
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
want to |
visit |
—it Highly |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Meets the |
most likely user |
|
||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
need. |
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||
|
Query: [kristen wiig] |
|
|
|
|
|
|
|
|
|
|
|
|
This is a fan site dedicated to |
|
||||
|
User Location: New |
|
|
|
|
|
|
|
|
|
|
|
|
Kristen Wiig. The website |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
has comprehensive info |
|
|||||
|
York, New York |
|
|
|
|
|
|
|
|
|
|
|
|
including over 50,000 pictures |
|
||||
|
User Intent: Find more |
|
|
|
|
|
|
|
|
|
|
|
|
and 300 video clips, |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
interviews, articles, and more. |
|
|||||
|
information about the |
|
|
|
|
|
|
|
|
|
|
|
|
This website Highly Meets |
|
||||
|
actress, comedian, |
|
|
|
|
|
|
|
|
|
|
|
|
the need of many or most |
|
||||
|
writer, and producer. |
|
|
|
|
|
|
|
|
|
|
|
|
users. |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Query: [greek alphabet] |
|
|
|
|
|
|
|
|
|
|
|
|
This is a broad query. |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||
|
User Location: Albany, |
|
|
|
|
|
|
|
|
|
|
|
|
Different users may be |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
looking for different things. |
|
|||||
|
New York |
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
User Intent: Find |
|
|
|
|
|
|
|
|
|
|
|
|
This result shows multiple |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
tables of images with the |
|
|||||
|
historical information |
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
|
letters in the Greek alphabet. |
|
|||||
|
about the Greek |
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
|
This result would be helpful |
|
|||||
|
alphabet, the names of |
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
|
for many users. Some users |
|
|||||
|
the letters in the Greek |
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
|
may wish to see additional |
|
|||||
|
alphabet, images of the |
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
results. |
|
|||||
|
letters, etc. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This is a broad query for an |
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
actress. Different users may |
|
|||
|
Query: [jennifer aniston] |
|
|
|
|
|
|
|
|
|
|
|
|
be looking for different things. |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
This result shows recent |
|
|||||
|
User Location: Dover, |
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
|
news about Jennifer |
|
|||||
|
Delaware |
|
|
|
|
|
|
|
|
|
|
|
|
Anniston. The articles are |
|
||||
|
User Intent: Find |
|
|
|
|
|
|
|
|
|
|
|
|
timely (at the time this was |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
written) and relatively |
|
|||||
|
images, recent news, |
|
|
|
|
|
|
|
|
|
|
|
|
interesting, and not just |
|
||||
|
gossip, information, etc. |
|
|
|
|
|
|
|
|
|
|
|
|
common everyday news |
|
||||
|
about Jennifer Aniston. |
|
|
|
|
|
|
|
|
|
|
|
|
about the actress. Many |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
users would be satisfied with |
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
the result and some users |
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
may wish to see more results. |
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Proprietary and Confidential – Copyright 2015 106
