Although internet search engines like Google play an increasingly prominent role shaping voter opinions and perception of issues and candidates, their ranking algorithms aren’t designed to provide a fairly balanced or completely honest representation of controversial issues.
In order to assess how fairly search engine results portray political candidates and controversial issues, we collected over 1,200 URLs ranking highly in Google.com for politically-charged keywords such as “gun control”, “abortion”, “TPP”, and “Black Lives Matter”. Each URL was then assessed for political slant by politically active individuals from both the left and right. Finally, we used CanIRank’s SEO software to analyze how each URL compared in dozens of different ranking factors to determine whether Google’s algorithm treated websites similarly regardless of their political slant.
Among our key findings were that top search results were almost 40% more likely to contain pages with a “Left” or “Far Left” slant than they were pages from the right. Moreover, 16% of political keywords contained no right-leaning pages at all within the first page of results.
Our analysis of the algorithmic metrics underpinning those rankings suggests that factors within the Google algorithm itself may make it easier for sites with a left-leaning or centrist viewpoint to rank higher in Google search results compared to sites with a politically conservative viewpoint.
Why Web Search Matters to Democracy
This fall, Gallup reported that Americans’ trust in mass media had hit an all time low, with fewer than a third of the population saying they have a great deal or a fair amount of trust in mass media — such as newspapers, TV, and radio.
Perhaps a direct result of this distrust in traditional mass media, Americans are increasingly turning to online sources for their information.
According to recent Google findings, online search is the resource that 87% of the population turns to first when a question arises.
Online search plays a particularly prominent role in the democratic process during election season. During the 2012 election cycle, a survey of persuadable voters revealed that 49% get their news about campaigns and the election online, largely through search engines like Google, and that these voters generally trust the information they find online.
Top search results are broadly perceived as being the most accurate and authoritative by members of the public with the first five search results accounting for an estimated 67% of all clicks and the first three results alone accounting for over 55% of all clicks.
In their 2015 study, Robert Epstein and Ronald Robertson concluded that the order of search results can have a big impact on voter behavior — and in the event of a close election, this effect could even be profound enough to determine the outcome of the election.
The extent to which a single private company now controls the flow of information is unprecedented in a country historically characterized by pluralistic and ideologically diverse media. This trend is particularly troubling when one considers that the employees of this private company do not reflect the ideological diversity of the country at large, and have consistently been amongst the largest donors to Democratic party candidates.
Should Search Engines Be Required to Give Equal Time?
Fairly presenting both sides of an issue has been considered essential to a functioning democracy since the very early years, when America’s founding fathers used subsidies to support the development and survival of a diverse media corps. Since 1927, radio and later television broadcast stations have been required to give equal time to opposing political candidates. For many years, the Fairness Doctrine required media to present both sides of controversial issues in an honest and equitable manner, a standard that many journalists still aspire to.
Now that web search has largely superseded radio and television as the primary source of information for most Americans, should web search results also be expected to present both sides of an issue?
In our sample set of over 2,000 search results, we found that searchers are 39% more likely to be presented information with a left-leaning bias than they are information from the right.
But for some keywords, the search results are even more egregious. Does it make sense, for example, that someone researching “Republican platform” should be presented only the official text of the platform and seven left-leaning results highly critical of that platform, with zero results supporting it?
For other controversial keywords like “minimum wage”, “abortion,” “NAFTA”, “Iraq war”, “campaign finance reform”, “global warming”, “marijuana legalization”, and “tpp”, no right-leaning websites are to be found among the top results.
Search results for keywords like “Hillary Clinton seizures” and “Hillary Clinton sick”, on the other hand, were dominated by right-leaning websites and YouTube footage.
The proportion of results with a left-leaning bias increased for top ranking results, which typically receive the majority of clicks. For example, we found that search results denoted as demonstrating a left or far left slant received 40% more exposure in the top 3 ranking spots than search results considered to have a right or far right political slant.
Content with a left or far left viewpoint receives even more exposure in the top five search results. Searchers are 65% more likely to encounter liberal search results than they are conservative search results if they look at the first five returns to their query. Assuming typical click distributions, that means only 1 out of 10 searchers would end up clicking on a result with a conservative viewpoint.
Why do sites from the left feature more prominently in Google’s search results?
Google considers about 200 major ranking factors in their algorithm to determine which results to serve a user in response to a search query. In order to calculate accurate keyword difficulty scores, our software collects and analyzes many of those same factors, allowing us to gain some insight into why certain pages rank where they do. Of these many factors, Google has identified links (i.e., the number and quality of links pointing to a page) and content (i.e., relevancy and comprehensiveness) to be the two most important ranking factors.
Average number of external links
Based on previously documented correlations between specific ranking factor metrics analyzed by CanIRank and actual search rankings, we would expect top ranked search results to have more external links compared to lower ranked search results.
Instead, pages demonstrating a left or far left political slant made it into the top results with significantly fewer external links compared to pages rated balanced. Pages with a right-leaning slant needed significantly more links to make it into the top results.
Median content length
Content length is another metric that tends to correlate with higher rankings, as it indicates a greater depth of information on a subject. For example, small businesses and startups are often able to rank well despite having fewer links, if they provide more comprehensive content on a specific topic.
Among our sample list of search results, right-leaning pages had a higher median number of words across all 40 results compared to both neutral and left-leaning pages. Despite this, neutral and left-leaning pages represented a greater proportion of top search results.
Conclusion
In addition to external links and content length, we analyzed a host of other ranking factors including:
- keyword usage in title, body, headings, img alt tags, and other HTML tags
- related term usage in each of the above areas
- number of indexed pages of the ranking website
- relevancy of indexed pages to keyword topic
- relevancy of home page
- relevancy of the ranking website’s domain name
- number of external and internal links to page and website
- MozRank, MozTrust
- Page Authority and Website Authority
- Facebook shares/ likes/ comments, tweets, +1s
No discernible pattern emerged when looking at many of these factors. The evidence, we conclude, is not strong enough to prove an inherent bias within Google’s algorithm.
Though Google would like to portray itself as a fair and balanced arbiter of facts — a role it has recently tried to strengthen with the launch of a fact checking mechanism — searchers should be aware that ranking algorithms don’t currently incorporate an assessment of political bias or even factual accuracy. No attempt is made to present multiple viewpoints on controversial political issues, and the algorithm in its current form does not return results equally distributed across the entire political spectrum.
Methodology
To ascertain whether Google search results showed a political bias, we created a list of 50 politically-relevant keywords, which together receive over 20 million monthly searches.
After harvesting the URLs from the first 4 pages of Google.com’s search results for each keyword (with personalization features turned off), and then discarding any URLs where we were unable to collect data (mostly due to the limitations of Moz’s Linkscape index, our source for inbound link metrics), we were left with 1,207 URLs.
Next, we invited scorers from both sides of the political spectrum to assign a political slant score on a 1-5 scale to each individual URL, where:
- 1 represents content that demonstrates a heavy bias toward views aligned with the Democrat party or a progressive ideology
- 2 represents content that demonstrates some bias toward views aligned with the Democrat party or a progressive ideology
- 3 represent content that is neutral or balanced
- 4 represents content that demonstrates some bias toward views aligned with the Republican party or a conservative ideology
- 5 represents content that demonstrates a heavy bias toward views aligned with the Republican party or a conservative ideology
Despite the large divide in our current political climate, our scorers were in agreement as to the degree and direction of bias for about half of the results, and were within 1 point on approximately 90%. In order to determine a consensus score for our analysis, we used an average of the political slant scores. The results were largely consistent with the political bias scores determined using a different methodology in a previous Facebook Research study.
In order to better understand why each URL ranks where it does, we used CanIRank’s SEO Competitive Analysis software to collect data on over 200 ranking factors for each URL ranking in the Google search results for these keywords.
The data collected by CanIRank’s software is intended to assess the relative strengths of a given URL for each of the major ranking factors that determine the ordering of results by internet search engines. Previous research has shown that CanIRank’s assessment of a URL’s competitiveness has an approximately 60% rank order correlation with the results ranking on Google.com. Most metrics analyzed by CanIRank fall into one of 7 major Ranking Factor areas:
- Page Relevancy: relevancy of the specific page to the keyword
- Website Relevancy: relevancy of the entire website to the keyword
- Page External Relevancy: relevancy and strength of links to the specific URL
- Website External Relevancy: relevancy and strength of links to the entire website
- Page Strength: trust and authority of the specific URL
- Website Strength: trust and authority of the entire website
- Social: popularity of the page on social networks such as Facebook, Twitter, Google+, LinkedIn, etc.
Examples of some of the data points we collected and analyzed are:
- keyword usage in title, body, headings, img alt tags, and other HTML tags
- related term usage in each of the above areas
- content length
- # indexed pages from each domain
- relevancy of indexed pages to keyword topic
- relevancy of home page
- relevancy of domain name
- # external and internal links to the ranking page and website overall
- Moz’s MozRank, Page Authority and Domain Authority
- Facebook shares/ likes/ comments, tweets, Google +1s, LinkedIn shares, Pinterest Pins
Finally, we assessed the strength of individual and aggregated ranking metrics relative to the URL’s actual ranking for each of our 5 political slant scores (Far Left, Left, Center, Right, Far Right). Although Google’s search ranking algorithm is non-linear and third-party metrics are at best a rough approximation of the data they use internally, in general and in aggregate, higher ranking results score higher in most ranking factor metrics. In an algorithm with no political bias, we would expect this pattern to hold up regardless of the political slant of the specific result.
The study is interesting, however I would have ranked the selected politically relevant keywords by some independent observer, to see whether they are neutral or not. A specific expression could be more used on one side than on the other.
Agreed. We collected keyword suggestions from political bloggers on the left and right, and also ran them by our panelists.
You search for left leaning search terms, and you are surprised that the results where pointing too left leaning web pages? How about searching for right leaning search terms and see where the search results point too? Like “Conservative”? “Tea Party? etc.
“searchers should be aware that ranking algorithms don’t currently incorporate […] factual accuracy” . They don’t?
“Based on previously documented correlations between specific ranking factor metrics analyzed by CanIRank and actual search rankings”, you mean that page where it’s just an advertisement for you company? You talk about how you shouldn’t use words that are searched for on web pages to get a higher ranking.
Then you bring up the Fairness doctrine, but don’t mention that Reagan suspended it, and it has been suspended since then. But want it reinstated on something else that it wasn’t intended for, but still leave it suspended for what it was for.
Thanks for your comment Steve! The tendency of specific keywords to lean one way or another is a phenomenon we discussed in the post and accounted for in our research by utilizing keywords from both right-leaning and left-leaning political bloggers. You can see the full list of keywords embedded in the post and you’ll note there are many representing topics more likely to be searched by conservatives (e.g., “Hillary Clinton seizures”, “document dump”, “Clinton emails”, “Benghazi”, etc. in addition to many keywords that are relevant to both parties (“abortion”, “budget deficit”, “NAFTA”, etc…).
The bias of google is really obvious and isn’t hard to understand.
It’s run by a bunch of immature Lefties who just happened to strike it lucky (like winning the lottery) Suddenly they are Kings of the World. And they want to use their power to tell us how to live our lives.Hence they use their search engine to favor one side of politics. The Left/Marxist/Socialist model.
Winning the lottery is striking it lucky. Building the most sophisticated technology company the planet has ever seen does not happen without intelligence, dedication, maturity, and a whole lot more in addition to luck.
Interesting study, just a little confused about something:
Average number of external links & Median content length both seem to indicate a left leaning bias, but you conclude with “The evidence, we conclude, is not strong enough to prove an inherent bias within Google’s algorithm.”
What am I missing there?
That’s a fair point, I can certainly see how one could come to that conclusion. It’s just a little too reductive for my tastes; there are many more factors that determine Google rankings; even our software considers over 200 ranking factors and this is just two. My personal opinion is that there are factors inherent in the nature of biased content that makes it more likely to rank higher, such as a propensity for people to share and link to content that aligns with their personal views.
Here’s an alternative based on no fact and just a personal observation: 1.) People who are more tech-driven are typically younger than their none-techy counterparts. These “youth” 18-35 are likely to write more articles, create websites and backlinks to websites they agree with. 2.) The younger you are the more likely you are liberally leaning. The older you are, typically speaking, the more likely you are to be a right-winger. Put two and two together here. Google’s Algorithm isn’t biased but instead the stories/blogs/websites/social signals that Google use to create rank lean about 20% more left due to the younger, more techy left.
More signals to Google that come from a left-leaning source mean, slightly more left-leaning results. No conspiracy.
No conspiracy, Jeff! I agree completely, and am sorry if that didn’t come across in the post. The key point we were trying to make is that there are factors inherent in the Google algorithm (or any engagement-based algorithm) that will cause stories with a bias in one direction or the other to bubble to the surface rather than stories that are well-balanced. That’s a problem if people look to Google as an arbiter of fact.
Does anyone actually believe that Buzzfeed, LA times, “independent”, NYT, Huff post, BBC, CNN, Politico are anywhere NEAR the centre?
If so, I have this bridge for sale. Cheap, well maintained and high traffic – looks a bit like a coat-hanger
I noticed that too. Seems there’s a clear bias in reporting this ‘lack of conclusive evidence for bias’. Funny how that works.
To note: Our study was not evaluating the bias of major news publications, rather it was about Google, specifically, and the hard data of our results.
And categorizing that data incorrectly or with bias certainly effects that “hard data”.
Why does well more than 60% of my Google result come up far right wing? Maybe it’s what I’m searching for. Or maybe who ever did your poll was so far to the right that everything looks left to them
Google attempts to leverage predictive algorithmic science in it’s results when people are logged into Chrome, so it will skew towards your base proclivities to some extent.
I just did my own search for “abortion” while not logged in using no less than 4 ad/cookie filters. 5 of the 10 results on the first page were for planned parenthood. The remaining 5 were NYT, Atlantic and other liberal sites condemning conservatives for not supporting abortions. The entire second page was also various sites saying that abortion was OK and describing the procedures in medical terms. Not a single link to anywhere saying that a very very large percentage of the country believes that abortion is immoral and murder.
Google, your days of regulation are coming… and not soon enough…
Just a thought, if Google is so bad, use a different search engine, if none of them are up to your standards, well, get a group of like minded conservatives to build a better one. Instead of screaming for regulation, how about that whole free market thing you conservatives seem to like so much.
And I just searched for pro life and guess what all I got were a bunch of anti abortion articles how DARE google return results on the topic I searched for…
Of COURSE if you search for “abortion” they are going to direct you to places that perform abortions, and information ABOUT abortions. Why would google provide anti abortion information in a search for abortion? If you want to research the various sides of the debate, search for the various sides.. google is simply going to provide you with information on the topic you searched for. In this case information on abortions and where you can get on. Your complaint is like ordering a pizza and then complaining that they brought you a pizza instead of spaghetti…. if you want pro life articles search for pro life
You might have a point if you were comparing a search for “pro-life” to a search for “pro-choice”. You didn’t. “Abortion” should provide both.
Headline “Study: Google bias in search results; 40% lean left or liberal” doesn’t reflect body copy of “that top search results were almost 40% MORE LIKELY to contain pages with a “Left” or “Far Left” slant than they were pages from the right. ” [my caps]
Need to fix headline! The results are 40% more likely to be from the left than the right. It’s not that the results are 40% from the left. Depending upon how many are judged to be “neutral” that could reflect balance!
Matt Bentley, first, I thought the article was very informative, even-handed, and well written.
My question, and please forgive my suspicious nature, is this: Would it be possible for Google to install certain keywords in their search algorithm that could be selectively shared with webmasters to make certain that their “contributions” made it to the first page of a search result?
Technically possible but extremely unlikely. Their systems are so complex and powerful, I think that even if some rogue developer wanted to do something like that, it would be extremely difficult and would be immediately discovered.
This study leaves a lot to be desired if the goal is to understand actual causation of sort rank and bias. For one, it neglects all other search engines, other than Google. With very little effort, I type “Republican Platform” in a DuckDuckGo.com, a non-tracking search engine with a high value on privacy, and I see very similar results to Google’s.
With this being an SEO site, I am surprised to see nothing suggesting methods on how to improve ranking. Furthermore, this study is not scientific. To conclude any kind of bias against a political agenda, you’d need to know the popularity and amount of all left and right wing websites. Popularity, more than anything, is what drives SEO rankings. This applies to all industries, not just politics. Does Google exist to promote the little guy or to find the search results people are looking for? And is there anything stopping anyone from typing “Conservative News Sites” into Google to find such results? Absolutely not.
I find it all very misleading. People assume because someone in authority suggests it to be true that it must be true. But nothing is indicating Google is treating political news any different than searches on lawn mowers.
I think you may be misinterpreting the post and/or study.
America is overrun with leftist media promulgating its point of view. There doesn’t seem to be any end to it and it is getting worse by the day. The internet is not full of free ideas. It is full of leftist propaganda.
How about we talk about the “Project Veritas” videos dealing with google, and revisit this conclusion?
Google’s Top Stories box provides users with articles from left-leaning news organizations such as CNN 62.4 percent of the time — with only 11.3 percent coming from outlets that are considered conservative, according to a study by Northwestern University researchers.
Hi @Matt or anyone. Is there any tool to search these political keywords regionally? E.g. Montreal vs. New York. Or even further, if I put together a list of say, Irish equivalent political keywords can I compare Dublin to London’s equivalent of some term? Such a tool would be interesting to study how google treats regional elections and foreign policy issues.
I love the combo of Adwords + CanIRank!
Thanks Anurag!
I just tested 10 topics , 100% of the first page contained liberal bias…just saying
Excellent post, I’ve been a long-standing fan of your content! Thank you!
I have been noticing this for a long time now. I have switched my internet settings to DuckDuckGo and find a good mix of results again. I was getting so frustrated with having to dig past garbage news pages like msnbc and cnn that overwhelm the first several pages of a google search. Thanks, great article.
They do it because of the belief of once they will get there full power. Full power will come when Joe Biden is elected. The only thing that will stop them is if the republicans can gain the congress or hold senate. If trump loses, GOOGLE wins! I do my own studies all the time. Just type in Trump or a prominent republican name and then look at the bias results! Type in Obama, Biden or Hilary and it comes up golden.
I don’t know if I’d call that a study.