Search engine penalties: The actual rules are secret (& constantly changing)
Raymond Snoddy on what is becoming an increasingly controversial issue but something which most people barely know exists - search engine penalties. Is it time to extract from the mighty Google greater search engine transparency and fairness?
Exactly two years ago something very strange happened to One News Page, the news navigator website, or aggregator.
When founder and chief executive Dr Marc Pinter-Krainer came into work he found that the Google traffic to his website had dropped overnight by 96%.
Pinter-Krainer had been hit by something he barely knew existed - a search engine penalty.
These penalties are not financial - at least not directly. Instead the website involved suffers the penalty for a real or apparent breach of Google's rules of being moved down the ranking league tables of most sought key words towards online oblivion.
It could have been the end for what at the time was a fledgling business because Google controls something like 85% of search engine traffic in Europe.
Pinter-Krainer kept his business alive despite the massive hit by using social media websites like Twitter.
Then nine months later in February 2010 something equally strange happened - the Google traffic came back as unexpectedly as it had disappeared.
The average person would have merely breathed a sigh of relief, grateful that the torment was over and got on with building his business.
But during those nine months the One News Page executive found he was not alone, that other online companies not just in media or marketing but even retail giants such as JC Penney in the US had been hit by search engine penalties.
Gradually Pinter-Krainer uncovered a Kafkaesque world in which he was unable to find out what Google guideline - if any - he had breached or any effective method of appeal.
There had been no notification that he had been penalised and therefore no explanation for why it had happened, no-one to ring or write to, only an online "re-inclusion" address which issued the same standard acknowledgement time after time whatever detailed case was put forward.
In a courageous, some might even say foolhardy move, the One News Page has launched a campaign designed to extract from the mighty Google what it sees as greater search engine transparency and fairness.
The company has tried to identify the scale of the problem through its campaign website - www.haveibeenpenalised.com.
One News Page ran a survey of more than 1,000 UK and US companies with an online presence and found that 24% had suffered large, unexplained falls in their site traffic. One in 10 of the respondents had no idea that unexplained falls in site traffic could come from penalties imposed by the leading search engines.
Last week One New Page took its campaign one step further with a City seminar designed to highlight the problem - complete with case studies.
Tariq Admed, a search specialist for Adrac, the online marketing company told of how NetMovers, the property portal, received a 50-point penalty and had its homepage removed without explanation.
NetMovers moved to a new domain and recovered.
Admed told how Foundem, the price comparison site, is pursuing an EU anti-trust case against Google after being taken off the Google index - a penalty that lasted for three years before restoration.
The Adrac specialist also told how overstock.com had been given a lower search ranking with a 5% drop in sales after it had created links with educational institutes in apparent breach of the rules.
The campaigners concede that many of the Google criteria are fair and necessary to provide an even playing field for the ever-increasing number of companies doing business online. In particular it is entirely reasonable to prevent anyone obtaining unfair advantage online through unacceptable search optimisation techniques.
Online behaviour which is - perfectly correctly - deemed unacceptable includes:
- cloaking - submitting one thing to a search engine and then displaying something else to the end user
- winter-storming - adding invisible text designed to trick search engines on the relevance of an item
- keyword stuffing - trying to artificially inflate the prominence of your keywords
Apart from the lack of notification about a penalty or the reasons for it and lack of an effective appeal process, Pinter-Krainer argues that the guidelines are imprecise and can change without warning on an almost daily basis.
Bob Sakayame, a US specialist on search engine penalties and on trying to get the index rankings of those who have suffered penalties restored, told the seminar he had even sought deliberately to invoke penalties to try to understand the process - without much luck.
"It turns out that the published guidelines are only that, guidelines. The actual rules are secret. And constantly changing," said Sakayama.
What about pushing for an external ombudsmen to handle appeals?
Sakayama has suggested that among other potential remedies but Google appears reluctant to open its technical working to external gaze on grounds of commercial confidentiality, he claims.
Tim Cowen, an expert on EU and UK competition from law firm Sidley Austin, supported the appeal at the seminar for greater transparency on search engine penalties and expressed surprise that more companies had not sought legal redress in the past.
As for Marc Pinter-Krainer, he intends to pursue his campaign for greater transparency on grounds of natural justice.
All online companies facing search engine penalties, he believes, should know exactly what offence they have committed, know precisely what penalty has been imposed and for what period, how they can remedy any infringement if there has been one and have an effective right of appeal if they do not believe they have done anything wrong.
It would only be fair and the fate of some small online companies could depend on it.
The One News Page founder would be prepared to set up another seminar if a Google representative would take part to discuss what is becoming an increasingly controversial issue in an attempt to find a reasonable way forward.