- •Releasing Tasks
- •Introduction to Page Quality Rating
- •Understanding Webpages and Websites
- •Important Definitions
- •What is the Purpose of a Webpage?
- •Your Money or Your Life (ymyl) Pages
- •Understanding Webpage Content
- •2.4.1 Identifying the Main Content (mc)
- •2.4.2 Identifying the Supplementary Content (sc)
- •2.4.3 Identifying Advertisements/Monetization (Ads)
- •2.4.4 Summary of the Parts of the Page
- •Understanding the Website
- •2.5.1 Finding the Homepage
- •2.5.2 Finding Who is Responsible for the Website and Who Created the Content on the Page
- •2.5.3 Finding About Us, Contact Information, and Customer Service Information
- •Website Maintenance
- •Website Reputation
- •2.7.1 Reputation Research
- •2.7.2 Sources of Reputation Information
- •2.7.3 Customer Reviews of Stores/Businesses
- •2.7.4 How to Search for Reputation Information
- •2.7.5 What to Do When You Find No Reputation Information
- •Overall Page Quality Rating Scale
- •High Quality Pages
- •Characteristics of High Quality Pages
- •A Satisfying Amount of High Quality Main Content
- •A High Level of Expertise/Authoritativeness/Trustworthiness (e-a-t)
- •Positive Reputation
- •Helpful Supplementary Content
- •Functional Page Design
- •A Satisfying Amount of Website Information
- •A Well Cared For and Maintained Website
- •Examples of High Quality Pages
- •Highest Quality Pages
- •Very High Quality mc
- •Very High Level of e-a-t
- •Very Positive Reputation
- •Examples of Highest Quality Pages
- •6.0 Low Quality Pages
- •Low Quality Main Content
- •Unsatisfying Amount of Main Content
- •Lacking Expertise, Authoritativeness, or Trustworthiness (e-a-t)
- •Negative Reputation
- •Characteristics Which May Be Evidence of Low Quality
- •6.5.1 Unhelpful or Distracting Supplementary Content
- •6.5.2 Lacking Supplementary Content
- •6.5.3 Poor Page Design
- •6.5.4 Lacking Care and Maintenance
- •Unsatisfying Amount of Information about the Website
- •Examples of Low Quality Pages
- •7.0 Lowest Quality Pages
- •Harmful or Malicious Pages
- •Lack of Purpose Pages
- •Deceptive Pages
- •7.3.1 Deceptive Page Purpose
- •7.3.2 Deceptive Page Design
- •7.3.3 Sneaky Redirects
- •Lowest Quality Main Content
- •7.4.1 No Main Content
- •7.4.2 “Keyword Stuffed” Main Content
- •7.4.3 Gibberish or Meaningless Main Content
- •7.4.4 Automatically-Generated Main Content
- •7.4.5 Copied Main Content
- •7.4.6 More About Copied Content
- •7.4.7 How to Determine if Content is Copied
- •No Website Information
- •Highly Untrustworthy, Unreliable, Unauthoritative, Inaccurate, or Misleading
- •Abandoned Websites or Spammed Pages on a Website
- •Extremely Negative or Malicious Reputation
- •Examples of Lowest Quality Pages
- •8.0 Medium Quality Pages
- •Examples of Medium Quality Pages
- •9.0 Page Quality Rating: Important Considerations
- •Instructions for Rating Page Quality Tasks
- •The Top Three pq Considerations
- •Page Quality Considerations for Specific Types of Pages
- •Ratings for Encyclopedia Pages
- •Ratings for Pages with Error Messages or No mc
- •Ratings for Forums and q&a pages
- •Page Quality Rating faQs
- •Understanding Mobile Users, Mobile Queries, and Mobile Results
- •Important Rating Definitions and Ideas
- •Understanding the Query
- •Task Location (Locale) and User Location
- •Queries with an Explicit Location
- •Queries with Multiple Meanings
- •Query Meanings Can Change Over Time
- •Understanding User Intent
- •Know and Know Simple Queries
- •Do and Device Action Queries
- •Website Queries
- •Visit-in-Person Queries and User Location
- •Queries with Multiple User Intents
- •Understanding Result Blocks
- •Web Search Result Block Examples
- •Special Content Result Block Examples
- •Device Action Result Block Examples
- •How Device Action Results are Displayed in Rating Tasks
- •Rating on Your Phone Issues
- •Rating Using the Needs Met Scale
- •Rating Result Blocks: Block Content and Landing Pages
- •Fully Meets (FullyM)
- •Examples of Fully Meets (FullyM) Result Blocks
- •Examples of Result Blocks that Cannot be Fully Meets
- •Highly Meets (hm)
- •Examples of Highly Meets (hm) Result Blocks
- •Moderately Meets (mm)
- •Examples of Moderately Meets (mm) Result Blocks
- •Slightly Meets (sm)
- •Examples of Slightly Meets (sm) Result Blocks
- •Fails to Meet (FailsM)
- •Examples of Fails to Meet (FailsM) Result Blocks
- •14.0 Rating Porn, Foreign Language, Didn’t Load, and Hard to Use Results
- •Porn Flag
- •Needs Met Rating for Porn Results
- •Needs Met Rating for Clear Non-Porn Intent Queries
- •Needs Met Rating for Possible Porn Intent Queries
- •Needs Met Rating for Clear Porn Intent Queries
- •Reporting Illegal Images
- •Foreign Language Flag
- •14.4.1 Using the Foreign Language Flag
- •14.4.2 Needs Met Ratings for Foreign Language Results
- •14.4.3 English Language Results
- •Didn’t Load Flag
- •14.5.1 Using the Didn’t Load Flag
- •14.5.2 Needs Met Rating for Didn’t Load Results
- •Hard to Use Flag
- •14.6.1 Using the Hard to Use Flag
- •The Relationship between e-a-t and Needs Met
- •Rating Queries with Multiple Interpretations and Intents
- •Rating Queries with Both Website and Visit-in-Person Intent
- •Specificity of Queries and Landing Pages
- •Needs Met Rating and Freshness
- •Misspelled and Mistyped Queries and Results
- •Misspelled and Mistyped Queries
- •19.2 Name Queries
- •19.3 Spelling Suggestion Result Blocks
- •21.0 Product Queries: Action (Do) vs. Information (Know) Intent
- •22.0 Rating Visit-in-Person Intent Queries
- •26.0 Needs Met Task Page Screenshot
- •27.0 Notes about Using the Needs Met Rating Interface
- •28.0 Using the “Report a Problem / Release this Task” Button
- •29.0 Reporting Results with Duplicate Landing Pages
- •Rater-Identified Duplicates
19.3 Spelling Suggestion Result Blocks
When a user misspells or mistypes a query, search engines may display spelling suggestions.
Users rely on “Did you mean” suggestions to tell them when they might be spelling something incorrectly and as a way to get better search result pages. “Did you mean” type result blocks are Special Content Result Blocks, but they do have a prominent link to a page of search results for the suggested spelling.
Your Needs Met rating should reflect both the helpfulness of the suggestion itself and the helpfulness of the LP of the suggestion. For every spelling suggestion, please be sure to look at the suggestion and click through to the landing page of the suggestion before rating. Sometimes, you will find it helpful to compare the landing page of the suggestion to the landing page of the original query.
Proprietary and Confidential – Copyright 2015 144
Here is some specific guidance on assigning Needs Met ratings to spelling suggestion result blocks:
|
Rating |
|
Use this rating when: |
|
|
|
Fully Meets |
|
Do not use the FullyM rating for spelling suggestion result blocks. It doesn’t make sense for this |
|
|
|
|
type of result block. |
|
||
|
|
|
|
||
|
|
|
|
|
|
|
|
|
The result is very helpful for many or most users because: |
|
|
|
Highly Meets |
|
• The query is clearly and severely misspelled. |
|
|
|
|
• |
The spelling suggestion is very likely or exactly what users are looking for. |
|
|
|
|
|
|
||
|
|
|
• The landing page of the suggestion is very helpful. |
|
|
|
|
|
|
|
|
|
|
|
The result is very helpful for some users or helpful for many users because: |
|
|
|
Moderately Meets |
|
• The query is likely or very likely misspelled. |
|
|
|
|
• |
The spelling suggestion is the correct spelling for a likely or somewhat likely intent of the |
|
|
|
|
|
|
original query. |
|
|
|
|
• The landing page of the suggestion is helpful. |
|
|
|
|
|
|
|
|
|
|
|
The result is helpful for few users. One or more of the following apply: |
|
|
|
|
|
• It is likely that the query is not misspelled. |
|
|
|
Slightly Meets |
|
• The spelling suggestion represents an unlikely interpretation. |
|
|
|
|
• |
The spelling suggestion is trivial, such as adding or deleting a space which makes little |
|
|
|
|
|
|
||
|
|
|
|
difference to the query and/or the results on the landing page of the suggestion. |
|
|
|
|
• The landing page of the suggestion is less helpful. |
|
|
|
|
|
|
|
|
|
|
|
The result is unhelpful or useless. One or more of the following apply: |
|
|
|
|
|
• The query is clearly not misspelled. |
|
|
|
|
|
• The spelling suggestion itself is misspelled. |
|
|
|
Fails to Meet |
|
• The spelling suggestion changes the meaning of the query. |
|
|
|
|
• |
The spelling suggestion is clearly not what the user is looking for. |
|
|
|
|
|
|
||
|
|
|
• The spelling suggestion is offensive (e.g., uninvited porn, racial slurs). |
|
|
|
|
|
• The landing page of the suggestion is unhelpful (e.g., the results are useless for the |
|
|
|
|
|
|
original query). |
|
|
|
|
|
|
|
Here are some examples.
|
Query, User Location, Possible |
|
|
SERP for the Query, SERP for the Spelling |
|
|
Explanation |
|
|
|
User Intents |
|
|
Suggestion, Needs Met Rating |
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
|
|
Query: [irratated] |
|
SERP for the original query: LP |
|
|
|
|
||
|
User Location: Los Angeles, |
|
SERP for the spelling suggestion: LP |
|
The query is clearly |
|
|||
|
|
|
|
|
|
||||
|
|
|
|
|
misspelled. The suggestion |
|
|||
|
California |
|
|
|
|
|
|||
|
|
|
|
|
is the correct spelling of the |
|
|||
|
|
|
|
|
|
|
|
||
|
User Intent: This query has one |
|
|
|
|
query, and the LP of the |
|
||
|
|
|
|
|
|
||||
|
|
|
|
|
suggestion is very helpful. |
|
|||
|
reasonable interpretation: the word |
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
||
|
“irritated.” |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
Query: [“jack angle” scrap metal] |
|
SERP for the original query: LP |
|
|
|
|
||
|
User Location: Los Angeles, |
|
SERP for the spelling suggestion: LP |
|
The query is clearly |
|
|||
|
|
|
|
|
|
||||
|
California |
|
|
|
|
misspelled. The suggestion |
|
||
|
User Intent: This query has one |
|
|
|
|
is the correct spelling of the |
|
||
|
|
|
|
|
query, and the LP of the |
|
|||
|
|
|
|
|
|
||||
|
interpretation: Jack Engle & Co., a |
|
|
|
|
suggestion is very helpful. |
|
||
|
scrap metal company in Los |
|
|
|
|
|
|
|
|
|
Angeles. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Proprietary and Confidential – Copyright 2015 145
|
Query, User Location, Possible |
|
|
SERP for the Query, SERP for the Spelling |
|
|
Explanation |
|
|
|
User Intents |
|
|
Suggestion, Needs Met Rating |
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
|
|
Query: [utube] |
|
SERP for the original query: LP |
|
|
|
|
||
|
|
|
|
SERP for the spelling suggestion: LP |
|
The query is clearly |
|
||
|
User Location: Los Angeles, |
|
|
|
|
|
|||
|
|
|
|
|
misspelled. The suggestion |
|
|||
|
California |
|
|
|
|
|
|||
|
|
|
|
|
is the correct spelling of the |
|
|||
|
|
|
|
|
|
|
|
||
|
User Intent: This query has one |
|
|
|
|
query, and the LP of the |
|
||
|
|
|
|
|
|
||||
|
|
|
|
|
suggestion is very helpful. |
|
|||
|
reasonable interpretation: the |
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
||
|
youtube.com website. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
Query: [twilight new moon sound |
|
SERP for the original query: LP |
|
“Soundtrack” may be the |
|
|||
|
track] |
|
|
more common and preferred |
|
||||
|
|
SERP for the spelling suggestion: LP |
|
|
|||||
|
|
|
|
|
spelling, but “sound track” is |
|
|||
|
User Location: Los Angeles, |
|
|
|
|
|
|||
|
|
|
|
|
not technically incorrect. |
|
|||
|
California |
|
|
|
|
The LP of the suggestion is |
|
||
|
User Intent: The query intent is |
|
|
|
|
|
|||
|
|
|
|
|
perhaps a bit more helpful |
|
|||
|
clear. |
|
|
|
|
than the LP for the original |
|
||
|
There is a possible mild misspelling: |
|
|
|
|
query. |
|
||
|
|
|
|
|
|
|
|
||
|
“soundtrack” is the more common |
|
|
|
|
This suggestion is helpful for |
|
||
|
and possibly preferred spelling. |
|
|
|
|
some or few users. |
|
||
|
|
|
|
|
|
|
|
||
|
Query: [cynderella], English (US) |
|
SERP for the original query: LP |
|
It is likely that the query is not |
|
|||
|
|
|
|
|
misspelled. The results on |
|
|||
|
User Location: Los Angeles, |
|
SERP for the spelling suggestion: LP |
|
|
||||
|
|
|
the landing page of the |
|
|||||
|
|
|
|
|
|
||||
|
California |
|
|
|
|
suggestion probably do not |
|
||
|
User Intent: There are several |
|
|
|
|
match the intent of the |
|
||
|
|
|
|
|
original query. |
|
|||
|
interpretations for the query as |
|
|
|
|
|
|
|
|
|
spelled, including a bridal shop, an |
|
|
|
|
However, this suggestion |
|
||
|
entertainment agency, a cake |
|
|
|
|
could be helpful for a few |
|
||
|
company, etc. |
|
|
|
|
users. |
|
||
|
|
|
|
|
|
|
|
||
|
Query: [he likes me a lot but doesn't |
|
SERP for the original query: LP |
|
|
|
|
||
|
want to get hurt], English (US) |
|
SERP for the spelling suggestion: LP |
|
|
|
|
||
|
User Location: Los Angeles, |
|
|
|
|
The suggestion is misspelled |
|
||
|
|
|
|
|
even though the original |
|
|||
|
California |
|
|
|
|
|
|||
|
|
|
|
|
query is not. |
|
|||
|
User Intent: This query is not |
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
||
|
misspelled. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
Query: [how shades are cut], |
|
SERP for the original query: LP |
|
|
|
|
||
|
English (US) |
|
SERP for the spelling suggestion: LP |
|
The original query is clearly |
|
|||
|
User Location: Los Angeles, |
|
|
|
|
|
|||
|
|
|
|
|
spelled correctly. The |
|
|||
|
California |
|
|
|
|
suggestion changes the |
|
||
|
User Intent: This query is not |
|
|
|
|
meaning of the query. |
|
||
|
|
|
|
|
|
|
|
||
|
misspelled. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
20.0 Non-Fully Meets Results for URL Queries
Raters sometimes ask the question, “For a well-formed working URL query, are the only acceptable Needs Met ratings for a result either Fully Meets or Fails to Meet?” The answer is no. There can be other helpful results for URL queries.
Some users issue URL queries to find information about a website, such as reviews or recent news. We recommended this to you as one method of reputation research in the PQ guidelines. Real users do this too. Results which give reviews and reputation information can be very helpful for a URL query.
-
Proprietary and Confidential – Copyright 2015
146
However, websites that offer usage statistics about a website are not usually helpful results for URL queries. Most users aren’t interested in this kind of information.
|
Query, User Location, |
|
Result Block and LP, E-A-T Rating, Needs Met Rating |
|
|
|
Explanation |
|
|
|
User Intent |
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
This LP is the target of the query.
Query: |
|
|
|
|
|
|
|
|
|
[coldwatercreek.com] |
|
|
Online coupon codes are very |
|
User Location: Palo Alto, |
|
|
popular in the U.S. and this |
|
|
|
LP (from a well-known |
|
|
California |
|
|
coupon site) displays codes |
|
User Intent: Coldwater |
|
|
for online purchases on the |
|
|
|
Coldwater Creek website. |
|
|
Creek is a well-known |
|
|
Users may be interested in |
|
U.S. company that sells |
|
|
coupon codes and this would |
|
women’s clothing and |
|
|
be very helpful for those |
|
accessories online and in |
|
|
shopping online at |
|
|
|
|
||
retail stores. Users may |
|
|
coldwatercreek.com, |
|
want to go to the website, |
|
|
especially frequent shoppers. |
|
do research, or find more |
|
|
|
|
information on the |
|
|
|
|
company. |
|
|
This LP (from a well-known |
|
|
|
|
|
|
|
|
|
review site) has reviews of |
|
|
|
|
the coldwatercreek.com |
|
|
|
|
website. Users may be |
|
|
|
|
interested in reading reviews |
|
|
|
|
when considering whether to |
|
|
|
|
make purchases at |
|
|
|
|
coldwatercreek.com, |
|
|
|
|
especially new shoppers. |
|
|
|
|
|
|
-
Proprietary and Confidential – Copyright 2015
147
|
Query, User Location, |
|
|
Result Block and LP, E-A-T Rating, Needs Met Rating |
|
|
Explanation |
|
|
|
|
User Intent |
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This LP is the target of the |
|
|
|
|
|
|
|
|
|
|
query. |
|
|
|
Query: |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
||
|
[ratemyprofessors.com] |
|
|
|
|
|
This LP is a Huffington Post |
|
||
|
User Location: Los |
|
|
|
|
|
article dated August 31, 2012 |
|
||
|
|
|
|
|
|
about ratemyprofessors.com. |
|
|||
|
Angeles, California |
|
|
|
|
|
The article was newly |
|
||
|
User Intent: Go to |
|
|
|
|
|
published when it was added |
|
||
|
|
|
|
|
|
to the guidelines. Some |
|
|||
|
www.ratemyprofessors.co |
|
|
|
|
|
users might be interested in |
|
||
|
|
|
|
|
|
|
||||
|
m, a website where |
|
|
|
|
|
this article. |
|
||
|
students can rate their |
|
|
|
|
|
|
|
|
|
|
college professors. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This LP is an article dated |
|
|
|
|
|
|
|
|
|
|
April 17, 2006 about the |
|
|
|
|
|
|
|
|
|
|
ratemyprofessors.com |
|
|
|
|
|
|
|
|
|
|
website. Few or no users |
|
|
|
|
|
|
|
|
|
|
would be interested in this |
|
|
|
|
|
|
|
|
|
|
outdated information. |
|
|
|
|
|
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
|
|
