Penalization has become a regular part of the search engine optimization experience. Hell, it has changed the entire business model of webmedsolutions to building tools and services around penalty recovery and not just optimization. While penalties used to be a crude badge of honor worn by those leaning towards the black-hat side of the SEO arts, it is now a regular occurrence that seems to impact those with the best intentions. At webmedsolutions, we have learned a lot about penalties over the last few years—discerning between manual and algorithmic, Panda and Penguin, recovery methodologies and risk mitigation—but not much study has been done on the general response of websites to penalizations. We have focused more on what developers ought to do without studying what developers actually do in response to various penalties.

How SEO experts respond matters

As much as we often feel a communion among other SEOs in our resistance to Google, the reality is that we are engaged in a competitive industry where we fight for customers in a very direct manner. This duality of competition—with Google and with each other—plays out in a very unique way when Google penalizes a competitor. We learn a great deal in the following months about the competition, such as the sophistication of their team (how quickly they respond, how many links they remove, how quickly they recover), their financial strength (do they increase ad spend, how much and on what terms), and whether they eventually recover.
It is also important from a wider perspective of understanding Google's justifications for particular types of penalties that seem sweeping and inconsistent. Conspiracy theories abound regarding Penguin updates; I can't count how many times I have heard someone say that penalties are placed to encourage developers to switch to AdWords.
So, I decided to investigate the behavior of developers post-Penguin from a macro perspective to determine what kinds of responses we are likely to see, and perhaps even answer some questions about Google's motivations in the process.

The strategy

1. Collect examples: I collected a list of 100 domains that were penalized by Penguin 2.0 last year and confirmed their penalization through SEMRush.
2. Establish controls: For each penalized site, I identified one website that ranked in the top 10 for their primary keyword that was not penalized.
3. Get rankings and AdWords data: For each site (both penalized and control), we grabbed their historical rankings and AdWords spend from SEMRush for the months leading up to and following Penguin 2.0
4. Get historical link data: For each site (both penalized and control), we grabbed their historical link data from Majsetic SEO for the months leading up to and following Penguin 2.0.
5. Analyze results: Using simple regression models, we identified patterns among penalized sites that differed significantly from the control sites.

Remove bad links?

After a Penguin 2.0 update, it is imperative to identify and remove bad links or, at minimum, disavow them. While we can't measure disavow data, we can measure link acquisition data quite easily.

So, do SEO experts in general follow the expectations of link removal following a penalty?

Aggressive link removal: It appears that aggressive link removal is a common response to Penguin, as expected. However, we have to be careful with the statistics to make sure we correctly examine the degree and frequency with which link removal is employed. The control group on average increased their root linking domains by 41 following Penguin 2.0, but that could best be explained by a few larger sites increasing their links. When looking at an average of link proportions, only about 22% of the control sites actually saw an increase in links in the three months post-Penguin. The sites that were penalized saw a drop of 578 root linking domains. However, once again, this statistic is impacted by the link graph size of the individual penalized sites. 15% of those penalized still saw an increase in links in the three months following Penguin.
So, approximately 22% of domains not impacted by Penguin 2.0 had more root linking domains three months after the penalty, while only 15% of those penalized had more root linking domains post-Penguin. Notice how small the discrepancy is here. Developers responded differently only by 7% depending on whether or not they were penalized. While certainly those penalized removed more links, the practice of link building in general was very similarly affected. In the three months following Penguin, 78% of the control websites either dropped links or at least stopped link building and lost them through attribution. This is remarkable. There appears to be a deadening effect related to Penguin that impacts all sites—not just those that are penalized. While many of us expected Penguin to have a profound impact on link growth as Developers respond to fears of future penalties, it is still amazing to see it borne out in the numbers.

What I find more interesting is the variation in webmaster responses to Penguin 2.0. Some penalized Developers actually doubled down on link building, likely attributing their rankings loss to having too few links, rather than being penalized. We can tease this type of behavior out of the numbers by looking at the variances in percentage link change over time.
The variance among link fluctuations for sites that were not penalized was .08, but the variance among sites that were penalized was .38. This means that the behavior of websites after being penalized was far more erratic than those that were not. Some penalized sites made the poor decisions to greatly increase their links, although more sites made the decision to greatly decrease their links. If all developers responded uniformly to penalties, one would not expect to see such an increase in variance.
As SEOs, we clearly have our work cut out for ourselves in teaching developers that the appropriate response to a penalty is very much NOT adding more and more links to your profile, because this behavior is actually more common than link removal post-penalty. It is worth pointing out that it is possible that the developers disavowed links rather than removing them. We do not have access to that data, so we cannot be certain regarding that procedure. It is possible that some developers chose to disavow while others removed, and that the net impact on link value was identical, thus making the variance calculation false.
Do developers increase their ad spend?

We admit, We had our fingers crossed on this one. Honestly, who doesn't want to show that Google is just penalizing developers because it helps their bottom line? Wouldn't it be great to catch the search quality team not being honest with us about their fiduciary independence?

Well, unfortunately it just doesn't bear out. The evidence is fairly clear that there is no reason to believe that developers increase ad-spend following a Penguin 2.0 penalty. Let's look at the numbers.

First, across our data set, no one who was an advertiser prior to Penguin 2.0 stopped advertising in AdWords in the three months after. Of the sites that were not advertisers prior to Penguin 2.0, 10% of those not penalized ended up becoming advertisers in AdWords, while only 4% of those penalized became advertisers. Sites that weren't penalized were far more likely to join the AdWords program than those that were.
It wasn't only true that those unaffected by Penguin 2.0 were more likely to sign up for AdWords; they increased their average Ad-spend, too. There was a 78% greater increase in ad-spend by those unaffected by Penguin 2.0 than those who were. Moreover, bidding shifts for those not impacted by Penguin remained similar in two month intervals across multiple randomly selected three-month differences, meaning that there appeared to be no related impact whatsoever.
We can safely conclude from this that there does not appear to be a direct, causal relationship between Penguin penalties and increased AdWords spending. Now, one could of course make the argument that better search results might increase ad revenue in the future as Google attracts more users to a better search engine, but accusations of a fiduciary motivation for releasing updates like Penguin 2.0 cannot be substantiated with this data.
Do they recover?

By the 5th month, approximately 24% of sites that were penalized were at or above their pre-Penguin 2.0 traffic. This is an exciting outcome because it does show recovery from Penguin is possible. Perhaps most important, sites that were penalized and removed links on average recovered 28% more traffic in the five months after Penguin than those that did not remove links. We have good evidence to suggest at least a correlation between post-penalty link removal and traffic recovery. Of course, we do have to take this with a grain of salt for a number of reasons:

• Sites that removed links may have been more likely to use the disavow tool as well.
• Sites that removed links may have been more SEO-savvy in general and fixed on-site issues.
• Sites that did not remove links may have had more intractable penalties, thus their lack of removal was a conscious decision related to the futility of a removal campaign.

These types of alternate explanations should always be entertained when using correlative statistics. What we do have good evidence of is that traffic recovery is possible for sites hit by Penguin, although it is by no means guaranteed or universal. Penguin 2.0 needn't be a death sentence.

Takeaways

So, in a few weeks, we are likely to see another Penguin update, assuming Google follows its late-spring release date. When Penguin hits, be ready—even if you aren't going to be penalized. Here are some things you should be doing...
1. Know your bad links already. There is no reason to wait to be prepared for removal or disavowal. While I personally think that preemptive disavowal is likely the best practice, there is no excuse to just wait.
2. Don't worry about AdWords. There is no statistical evidence that your competition will surge post-Penguin in any meaningful fashion. The competitors who might come to depend move on AdWords also have less organic revenue to invest in the first place. At best, these even out.
3. Don't double down. While we can't be certain that link removal gets you out of penalties (it is merely correlated), we can be certain that even a correlation doesn't exist for increasing links and earning recovery post-Penguin penalties.
4. Never assume. The behavior of your competitors and of Google itself is far more complex than off-the-cuff assumptions like "Google just penalizes sites to force people into AdWords" or that your business will know intuitively to remove or disavow links post-Penguin.

Hopefully, this time around we will all be more prepared for the appropriate response to Google's next big update—whether we are hit or not.
WebMedSolutions