- •Releasing Tasks
- •Introduction to Page Quality Rating
- •Understanding Webpages and Websites
- •Important Definitions
- •What is the Purpose of a Webpage?
- •Your Money or Your Life (ymyl) Pages
- •Understanding Webpage Content
- •2.4.1 Identifying the Main Content (mc)
- •2.4.2 Identifying the Supplementary Content (sc)
- •2.4.3 Identifying Advertisements/Monetization (Ads)
- •2.4.4 Summary of the Parts of the Page
- •Understanding the Website
- •2.5.1 Finding the Homepage
- •2.5.2 Finding Who is Responsible for the Website and Who Created the Content on the Page
- •2.5.3 Finding About Us, Contact Information, and Customer Service Information
- •Website Maintenance
- •Website Reputation
- •2.7.1 Reputation Research
- •2.7.2 Sources of Reputation Information
- •2.7.3 Customer Reviews of Stores/Businesses
- •2.7.4 How to Search for Reputation Information
- •2.7.5 What to Do When You Find No Reputation Information
- •Overall Page Quality Rating Scale
- •High Quality Pages
- •Characteristics of High Quality Pages
- •A Satisfying Amount of High Quality Main Content
- •A High Level of Expertise/Authoritativeness/Trustworthiness (e-a-t)
- •Positive Reputation
- •Helpful Supplementary Content
- •Functional Page Design
- •A Satisfying Amount of Website Information
- •A Well Cared For and Maintained Website
- •Examples of High Quality Pages
- •Highest Quality Pages
- •Very High Quality mc
- •Very High Level of e-a-t
- •Very Positive Reputation
- •Examples of Highest Quality Pages
- •6.0 Low Quality Pages
- •Low Quality Main Content
- •Unsatisfying Amount of Main Content
- •Lacking Expertise, Authoritativeness, or Trustworthiness (e-a-t)
- •Negative Reputation
- •Characteristics Which May Be Evidence of Low Quality
- •6.5.1 Unhelpful or Distracting Supplementary Content
- •6.5.2 Lacking Supplementary Content
- •6.5.3 Poor Page Design
- •6.5.4 Lacking Care and Maintenance
- •Unsatisfying Amount of Information about the Website
- •Examples of Low Quality Pages
- •7.0 Lowest Quality Pages
- •Harmful or Malicious Pages
- •Lack of Purpose Pages
- •Deceptive Pages
- •7.3.1 Deceptive Page Purpose
- •7.3.2 Deceptive Page Design
- •7.3.3 Sneaky Redirects
- •Lowest Quality Main Content
- •7.4.1 No Main Content
- •7.4.2 “Keyword Stuffed” Main Content
- •7.4.3 Gibberish or Meaningless Main Content
- •7.4.4 Automatically-Generated Main Content
- •7.4.5 Copied Main Content
- •7.4.6 More About Copied Content
- •7.4.7 How to Determine if Content is Copied
- •No Website Information
- •Highly Untrustworthy, Unreliable, Unauthoritative, Inaccurate, or Misleading
- •Abandoned Websites or Spammed Pages on a Website
- •Extremely Negative or Malicious Reputation
- •Examples of Lowest Quality Pages
- •8.0 Medium Quality Pages
- •Examples of Medium Quality Pages
- •9.0 Page Quality Rating: Important Considerations
- •Instructions for Rating Page Quality Tasks
- •The Top Three pq Considerations
- •Page Quality Considerations for Specific Types of Pages
- •Ratings for Encyclopedia Pages
- •Ratings for Pages with Error Messages or No mc
- •Ratings for Forums and q&a pages
- •Page Quality Rating faQs
- •Understanding Mobile Users, Mobile Queries, and Mobile Results
- •Important Rating Definitions and Ideas
- •Understanding the Query
- •Task Location (Locale) and User Location
- •Queries with an Explicit Location
- •Queries with Multiple Meanings
- •Query Meanings Can Change Over Time
- •Understanding User Intent
- •Know and Know Simple Queries
- •Do and Device Action Queries
- •Website Queries
- •Visit-in-Person Queries and User Location
- •Queries with Multiple User Intents
- •Understanding Result Blocks
- •Web Search Result Block Examples
- •Special Content Result Block Examples
- •Device Action Result Block Examples
- •How Device Action Results are Displayed in Rating Tasks
- •Rating on Your Phone Issues
- •Rating Using the Needs Met Scale
- •Rating Result Blocks: Block Content and Landing Pages
- •Fully Meets (FullyM)
- •Examples of Fully Meets (FullyM) Result Blocks
- •Examples of Result Blocks that Cannot be Fully Meets
- •Highly Meets (hm)
- •Examples of Highly Meets (hm) Result Blocks
- •Moderately Meets (mm)
- •Examples of Moderately Meets (mm) Result Blocks
- •Slightly Meets (sm)
- •Examples of Slightly Meets (sm) Result Blocks
- •Fails to Meet (FailsM)
- •Examples of Fails to Meet (FailsM) Result Blocks
- •14.0 Rating Porn, Foreign Language, Didn’t Load, and Hard to Use Results
- •Porn Flag
- •Needs Met Rating for Porn Results
- •Needs Met Rating for Clear Non-Porn Intent Queries
- •Needs Met Rating for Possible Porn Intent Queries
- •Needs Met Rating for Clear Porn Intent Queries
- •Reporting Illegal Images
- •Foreign Language Flag
- •14.4.1 Using the Foreign Language Flag
- •14.4.2 Needs Met Ratings for Foreign Language Results
- •14.4.3 English Language Results
- •Didn’t Load Flag
- •14.5.1 Using the Didn’t Load Flag
- •14.5.2 Needs Met Rating for Didn’t Load Results
- •Hard to Use Flag
- •14.6.1 Using the Hard to Use Flag
- •The Relationship between e-a-t and Needs Met
- •Rating Queries with Multiple Interpretations and Intents
- •Rating Queries with Both Website and Visit-in-Person Intent
- •Specificity of Queries and Landing Pages
- •Needs Met Rating and Freshness
- •Misspelled and Mistyped Queries and Results
- •Misspelled and Mistyped Queries
- •19.2 Name Queries
- •19.3 Spelling Suggestion Result Blocks
- •21.0 Product Queries: Action (Do) vs. Information (Know) Intent
- •22.0 Rating Visit-in-Person Intent Queries
- •26.0 Needs Met Task Page Screenshot
- •27.0 Notes about Using the Needs Met Rating Interface
- •28.0 Using the “Report a Problem / Release this Task” Button
- •29.0 Reporting Results with Duplicate Landing Pages
- •Rater-Identified Duplicates
Examples of High Quality Pages
|
Webpage/Type of Content |
|
High Quality Characteristics/PQ Rating |
|
PQ Rating and Explanation |
|
|||||
|
High: Government agency |
|
• |
Functional page design |
|
|
|
||||
|
|
|
|
|
|
|
|
|
|
||
|
US Naval Observatory |
|
• |
Highly trustworthy for the purpose of the |
The purpose of this page is to display the official |
|
|||||
|
|
|
page |
|
|||||||
|
Master Clock page |
|
|
US Naval Observatory Master Clock time in 7 |
|
||||||
|
|
|
|
|
|||||||
|
|
|
|
|
|
|
|
|
different time zones. The page displays the clock |
|
|
|
|
|
|
|
|
|
|
|
information in a clear, easy-to-read format. The |
|
|
|
|
|
|
|
|
|
|
|
Naval Observatory is highly trustworthy and |
|
|
|
|
|
|
|
|
|
|
|
authoritative for this type of information. |
|
|
|
|
|
|
|
|
|
|
|
|
||
|
High: Humor |
|
• |
High quality humorous MC |
|
|
|
||||
|
|
|
|
|
|
|
|
|
|
||
|
Article on a humor website |
|
• |
Positive reputation as a humor website |
|
This website is well known for its humorous, |
|
||||
|
|
|
|
|
|
||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
satirical articles. This is a cute example of a |
|
|
|
|
|
|
|
|
|
|
|
satisfying and funny article. |
|
|
|
|
|
|
|
|
|
|
|
||
|
High: Small business 1 |
|
|
|
|
This is an “about us” page on a restaurant |
|
||||
|
|
|
|
|
|
|
|
|
|
|
|
|
Local fish & chips |
|
• |
Authoritative for the topic of the page |
|
website. This page provides information on when |
|
||||
|
|
|
|
|
the restaurant opened and what visitors can |
|
|||||
|
restaurant |
|
|
|
|
|
|||||
|
|
|
|
|
expect. Other pages on the website provide |
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
information about the restaurant including the |
|
|
|
|
|
|
|
|
|
|
|
address, menu, other contact information, etc. |
|
|
|
|
|
|
|
|
|
|
|
This website is highly authoritative because it is |
|
|
|
|
|
|
|
|
|
|
|
about itself. |
|
|
High: Small business 2 |
|
• |
Authoritative for the topic of the page |
|
This is the News and Updates section of a local |
|
||||
|
|
|
|
|
|
|
|
|
|||
|
Local preservation center |
|
• |
A satisfying amount of high quality MC |
|
preservation center selling poultry, vegetables, |
|
||||
|
|
|
|
|
and more. While the 2007 copyright date is |
|
|||||
|
|
|
|
|
|
|
|
|
|
outdated, most of the pages, including this one, |
|
|
|
|
|
|
|
|
|
|
|
have recent updates from 2014. For this small |
|
|
|
|
|
|
|
|
|
|
|
business, the page design is functional for the |
|
|
|
|
|
|
|
|
|
|
|
purpose of the page. |
|
|
|
|
|
|
|
|
|
||||
|
High: News 1 |
|
• |
A satisfying amount of high quality MC |
|
|
|
||||
|
|
|
|
|
|
|
|
|
|
||
|
Homepage of a newspaper |
|
• |
Positive reputation |
|
|
|
||||
|
|
|
|
This newspaper has won seven Pulitzer Prize |
|
||||||
|
|
|
|
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
awards. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Proprietary and Confidential – Copyright 2015 23
|
Webpage/Type of Content |
|
High Quality Characteristics/PQ Rating |
|
PQ Rating and Explanation |
|
||
|
High: News 2 |
|
• |
A satisfying amount of high quality MC |
|
|
|
|
|
|
|
|
|
This is an article on a newspaper website which |
|
||
|
Article on a newspaper |
|
• |
Positive reputation for the website |
|
|
||
|
|
|
|
|
has won nine Pulitzer Prize awards. There is a |
|
||
|
website |
|
|
|
|
|
||
|
|
|
|
|
satisfying amount of high quality MC. While there |
|
||
|
|
|
|
|
|
|
are Ads on the page, the MC is easy to find and |
|
|
|
|
|
|
|
|
the Ads are clearly labeled as Ads. |
|
|
|
|
|
|
|
|
|
|
|
High: Blog post |
|
• |
A satisfying amount of high quality MC |
|
|
|
|
|
Parenting article about |
|
• |
Very positive reputation for the website |
|
This is a blog post on a newspaper which has won |
|
|
|
|
• |
Positive reputation for this specific blog and |
|
|
|||
|
|
|
over 100 Pulitzer Prize awards. The author of this |
|
||||
|
strollers |
|
|
author |
|
blog post has become known as an expert on |
|
|
|
|
|
|
|
|
|
parenting issues. She is a regular contributor to |
|
|
|
|
|
|
|
|
this and other media websites. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The purpose of this page is to allow users to buy a |
|
|
High: Shopping 1 |
|
|
|
|
school backpack. The page provides a lot of |
|
|
|
|
|
|
• |
A satisfying amount of high quality MC |
|
different backpack options, and some of them |
|
|
Backpack shopping page |
|
• |
Positive reputation |
|
have user reviews. |
|
|
|
on a popular store website |
|
• |
Very helpful SC |
|
This is a well-known, reputable merchant, with |
|
|
|
(YMYL) |
|
|
|
|
|
||
|
|
|
|
|
|
|
detailed Customer Service information on the site. |
|
|
|
|
|
|
|
|
The SC features are particularly helpful. For |
|
|
|
|
|
|
|
|
example, the filters allow users to show results by |
|
|
|
|
|
|
|
|
categories such as color, style, and price. |
|
|
|
|
|
|
|
|
|
|
|
High: Shopping 2 |
|
• |
Satisfying amount of high quality MC |
|
This company sells its own line of high end, |
|
|
|
|
|
|
• |
Expertise in the type of merchandise they |
|
|
|
|
Bathroom décor page on a |
|
|
are selling |
|
fashionable baby and children’s furniture and |
|
|
|
store website (YMYL) |
|
• |
Positive reputation |
|
accessories. It has a positive reputation as well |
|
|
|
|
|
|
|
|
|
as expertise in these specific types of goods. |
|
|
|
|
|
|
|
|
Many products sold on the site are unique to this |
|
|
|
|
|
|
|
|
company. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
• |
A satisfying or comprehensive amount of |
|
|
|
|
High: Shopping 3 |
|
|
very high quality MC |
|
|
|
|
|
TomTom GPS page on an |
|
• |
Expertise in the type of merchandise they |
|
There is a very large quantity of MC on this page. |
|
|
|
|
|
are selling |
|
|
|||
|
electronics website (YMYL) |
|
• |
Positive reputation |
|
Note that the tabs on the page lead to even more |
|
|
|
|
|
|
|
|
|
information, including many customer reviews. |
|
|
|
|
|
|
|
|
The tabs should be considered part of the MC. |
|
|
|
|
|
|
|
|
|
|
Proprietary and Confidential – Copyright 2015 24
|
Webpage/Type of Content |
|
High Quality Characteristics/PQ Rating |
|
PQ Rating and Explanation |
|
|||
|
High: Video 1 |
|
• |
A satisfying or comprehensive amount of |
|
|
|
||
|
|
|
|
|
|
The MC of this video page is an episode of |
|
||
|
Saturday Night Live video |
|
|
very high quality MC |
|
|
|||
|
|
• |
Very helpful SC |
|
Saturday Night Live, a very popular television |
|
|||
|
on a video website |
|
|
|
|||||
|
|
|
|
|
show. Below the main video, there are many |
|
|||
|
|
|
|
|
|
|
|
other videos that users may also be interested in. |
|
|
|
|
|
|
|
|
|
This SC is very helpful. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This is a humorous, high quality, well produced |
|
|
|
|
|
|
• |
A satisfying amount of high quality MC |
|
video with more than 6.5 million views (and |
|
|
High: Video 2 |
|
• |
Everyday expertise in the topic of cat |
|
counting), illustrating the proper care and practical |
|
||
|
|
|
|
|
|
ownership |
|
benefits of cats. The two engineers in the video |
|
|
“An Engineer’s Guide to |
|
• |
Very helpful SC |
|
have everyday expertise on cat ownership. For a |
|
||
|
Cats” video |
|
|
|
|
humorous video like this one, expertise does not |
|
||
|
|
|
|
|
|
|
|
have to be in the form of a trained veterinarian. |
|
|
|
|
|
|
|
|
|
Also, this video is part of a humorous series of cat |
|
|
|
|
|
|
|
|
|
care videos and the SC includes many other cute |
|
|
|
|
|
|
|
|
|
and funny cat videos. |
|
|
|
|
|
|
|
|
|
|
|
|
High: Financial |
|
• |
High quality MC |
|
|
|
||
|
Financial Markets page on |
|
|
|
|
||||
|
|
• |
Helpful SC |
|
This newspaper has won two Pulitzer Prize |
|
|||
|
a newspaper website |
|
• |
Authoritative and trustworthy |
|
|
|||
|
(YMYL) |
|
|
|
|
awards and has a positive reputation, though it is |
|
||
|
|
|
|
|
|
|
|
not an acknowledged expert in financial news. |
|
|
|
|
|
|
|
|
|
||
|
High: Game |
|
• |
A satisfying or comprehensive amount of |
|
|
|
||
|
|
|
|
|
|
|
|
||
|
Online game |
|
|
very high quality MC |
|
Highly engaging game with multiple levels that |
|
||
|
|
• |
Positive reputation |
|
|
||||
|
|
|
|
|
|
could entertain a child for hours. This website has |
|
||
|
|
|
|
|
|
|
a positive reputation for educating children about |
|
|
|
|
|
|
|
|
|
animals. |
|
|
|
|
|
|
|
|
|
|
||
|
High: Q&A 1 |
|
• |
Everyday expertise |
|
The question on the page asks how long people |
|
||
|
|
|
|
|
|
|
|||
|
How long do cancer |
|
|
live with cancer. There are many results |
|
||||
|
|
• |
A satisfying amount of high quality MC |
|
|
||||
|
|
|
describing how long a loved one lived after |
|
|||||
|
patients live |
|
|
|
|
|
|||
|
|
|
|
|
diagnosis. There is very little medical advice and |
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
the focus of the page is sharing personal |
|
|
|
|
|
|
|
|
|
experience. Many results are heartfelt and well |
|
|
|
|
|
|
|
|
|
written. |
|
|
|
|
|
|
|
|
|
|
|
Proprietary and Confidential – Copyright 2015 25
|
Webpage/Type of Content |
|
High Quality Characteristics/PQ Rating |
|
PQ Rating and Explanation |
|
||
|
|
|
|
|
|
|
Since the question asks for ordinary people’s |
|
|
|
|
|
|
|
|
opinions, everyday expertise is desired. Many of |
|
|
|
|
|
|
|
|
the answers are well written and thoughtful. It is |
|
|
High: Q&A 2 |
|
• |
A satisfying or comprehensive amount of |
|
clear that many people spent time answering this |
|
|
|
|
|
very high quality MC |
|
question, doing their best to help Hillary Clinton |
|
||
|
Q&A page on which Hillary |
|
• |
Very high level of expertise, highly |
|
understand their perspectives. Web research can |
|
|
|
|
|
help confirm that it was really Hillary Clinton |
|
||||
|
|
|
authoritative/highly trustworthy for the |
|
|
|||
|
Clinton asks users for their |
|
|
|
asking the questions. |
|
||
|
|
|
purpose of the page |
|
|
|||
|
personal opinions |
|
|
|
|
|
||
|
|
|
|
|
|
|
||
|
|
|
|
|
|
|
Keep in mind that for any type of page, including |
|
|
|
|
|
|
|
|
Q&A pages, there may be a range of highest |
|
|
|
|
|
|
|
|
quality to lowest quality pages. Therefore, it’s |
|
|
|
|
|
|
|
|
important to evaluate the page using the same |
|
|
|
|
|
|
|
|
criteria as all other pages, regardless of what type |
|
|
|
|
|
|
|
|
of page it is. |
|
|
|
|
|
|
|
|
|
|
|
High: Encyclopedia |
|
• |
A satisfying or comprehensive amount of |
|
|
|
|
|
|
|
|
|
This is a comprehensive article about the |
|
||
|
Encyclopedia article about |
|
|
high quality MC |
|
|
||
|
|
• |
Positive reputation for accurate information |
|
American Civil War. The website has a good |
|
||
|
the American Civil War |
|
|
|
||||
|
|
|
|
|
reputation for accuracy. In addition to the article |
|
||
|
|
|
|
|
|
|
itself, there are many helpful references and |
|
|
|
|
|
|
|
|
citations to support the content. |
|
|
|
|
|
|
|
|
|
|
|
High: Forum 1 |
|
• |
High level of expertise for this topic; |
|
This forum is well known for discussions on luxury |
|
|
|
|
|
authoritative and trustworthy for the purpose |
|
|
|||
|
|
|
|
|
|
designer purses. On this particular forum page, |
|
|
|
|
|
|
|
of the page |
|
|
|
|
Authenticating a luxury |
|
|
|
members are consulting forum experts who have |
|
||
|
|
• |
A satisfying amount of high quality MC |
|
|
|||
|
designer purse |
|
|
expertise authenticating bags from this brand. |
|
|||
|
|
|
|
|
|
|
These experts can tell if a particular bag is |
|
|
|
|
|
|
|
|
authentic or fake. While there is an ad at the top |
|
|
|
|
|
|
|
|
and a few ads within the forum message, it does |
|
|
|
|
|
|
|
|
not distract from the MC, which is easy to find. |
|
|
|
|
|
|
|
|
|
|
|
High: Forum 2 |
|
• |
High level of expertise for this topic; |
|
This discussion focuses on the landscaping for a |
|
|
|
|
|
authoritative and trustworthy for the purpose |
|
|
|||
|
|
|
|
|
|
particular paludarium (an aquarium with terrestrial |
|
|
|
|
|
|
|
of the page |
|
|
|
|
Landscaping an aquarium |
|
|
|
and aquatic elements). There is a lot of |
|
||
|
|
• |
A satisfying amount of high quality MC |
|
|
|||
|
|
|
|
discussion and interaction between forum |
|
|||
|
|
|
|
|
|
members about the types of materials and |
|
|
|
|
|
|
|
|
species used in the aquarium. The posts show |
|
|
|
|
|
|
|
|
expertise in a niche topic aquarium landscaping. |
|
|
|
|
|
|
|
|
|
|
|
Proprietary and Confidential – Copyright 2015 26
