Are Bad Bots on your Website Disrupting your SEO Strategy?
Search engine optimization is one of the most important aspects of any business’s online marketing strategy. A well-maintained SEO plan provides a low-cost, long-term stream of relevant traffic into a website.
Conversely, bad SEO can be very damaging to a business. Poor visibility on search engines like Google hands revenue over to competitors, forces higher spend on PPC advertising, and can damage trust with potential customers searching for you online.
This is even worse when your marketing team is actively working to improve your SEO, but malicious actors are obstructing their efforts. Bots are notorious for their negative effect on SEO.
This is ironic because search engine crawlers like Googlebot are – obviously – bots themselves. But there are also a multitude of bad bots that can damage your site’s SEO, either purposefully or as an indirect consequence of their main objectives.
In this blog post we’ll look at the bots that are affecting SEO on websites, intentionally or not.
Web scraping and SEO
One of the most damaging bots is one that operates similarly to those all-important search engine crawlers, but for a very different purpose. Web scrapers crawl websites looking for specific information, such as content or prices.
Content scraping can be particularly damaging to SEO. Scrapers are used to steal content, for example product listings or blog posts, to reuse elsewhere on the web.
Google hates duplicated content and will penalize sites that share the exact same text. So, if your pages are getting scraped and copied across to a lower ranking site, your organic search rankings could be dragged down too.
In a recent Netacea webinar, “What are bad bots costing your business?” Spike CEO and marketing expert Duncan Colman emphasized the negative impact scrapers have on SEO, noting that “there are sometimes issues with the search engines from a technical perspective where [the duplicated content] ends up outranking your own website.”
Price scraping is a bot attack with less obvious implications on SEO. Usually used by competitors to automatically undercut your own prices to lure away price-sensitive consumers, price scraping can cause your competitor’s product pages to outrank your own pages on Google Shopping when consumers search for specific items.
Spam bots are still harming SEO
One of the longest established types of bot attack is form and comment spam. Bots trawl web pages looking for text input forms, before injecting spammy content in huge quantities.
While this activity is often part of phishing attempts or identity fraud, or to direct victims towards malware, there is also an indirect SEO impact when bots spam forms with content. Google quickly recognizes spammy content, so if bot-generated comments make it onto your blog, it can harm your rankings.
Conversely, click spam bots are designed explicitly to harm SEO. Clicking a result on Google and then bouncing, immediately signals to Google that the page is low quality or not relevant to the search term. Competitors or bad actors can exploit this with a volumetric bot attack, spamming clicks on your organic links before immediately bouncing from the landing page and driving the page’s authority down.
Are bots slowing down your website?
Better SEO leads to better results for your business, but maximising the efficacy of your SEO means getting pages to load as quickly as possible – something which is near impossible if your site is inundated with bots.
Bots work by making thousands of page visits within a very short time, which can place huge strain on servers and prevent genuine users from accessing your website. In fact, a vast number of bad bots entering a website at once is often a direct attempt to overrun the servers and take the site. This is known as a Distributed Denial of Service (DDoS) attack.
Many bot threats are high volume in nature. For example, scrapers typically hit sites repeatedly to keep their information as up to date as possible, and websites could be scraped by multiple attackers simultaneously. Sniper bots and scalper bots also hit sites hundreds or even thousands of times each minute so they can detect the exact moment an item goes on sale, or the page’s information changes.
If traffic levels get too much for a website’s infrastructure to handle, page load times will rise dramatically. If Googlebot crawls the site during this time it will report that the site is slow. Given the effects of site speed on the overall user experience – a huge contributing factor to SEO rankings – it’s no surprise that sites with long loading times experience a drop in traffic and, in turn, site performance.
Bot traffic wreaks havoc on SEO strategy
Analytics are crucial to SEO executives. Without good information on web traffic, there is no way of knowing how effective their strategy is.
Unmitigated bot traffic of any type skews this information, especially if it comes in high volumes. Efforts can be focused on the wrong areas based on this information, wasting time and budget, and ultimately losing out on rankings.
Erroneously blocking search engine crawler bots kills your SEO
After learning of the damage bad bots can cause across the business, it’s tempting to block all bots – after all, we just want real customers (humans) to visit our website, right?
Not so fast! Blocking all bots can do more harm than good. If you block Googlebot, it won’t be able to crawl your website freely and your organic search rankings will suffer dramatically. All your other SEO efforts will be for nothing if search engine bots can’t crawl your site!
Sometimes bot management solutions mistakenly block search engine bots thinking they are another type of bot, or will blanket block all bots, even those providing a business benefit. It is crucial that your bot management solution can tell the difference between good and bad bots. This is not always easy to do when many bots disguise themselves using Googlebot’s user agent.
Intent Analytics™ distinguishes visitors by their behavior, not just their origin, resulting in an industry-low false positive rate of 0.001%.
How to stop bots from interfering in SEO
Check your backlinks and duplicated content
It’s good SEO practice to keep a close eye on new backlinks. If backlinks look spammy or duplicate your content, there’s a good chance that bots were at play. You can also use SEO tools to check where your content has been duplicated elsewhere, most likely by a bot.
Advanced bot management protects seo
Web scraping and content spamming are notoriously harmful to SEO, but they work in a very similar way to the scraper bots and scalper bots commonly detected and mitigated by Netacea’s bot management solution.
Whilst many bots attempt to disguise themselves as human users, or masquerade as Googlebot by spoofing its user agent, Netacea’s Intent Analytics™ engine is not fooled by such signals. Instead, advanced machine learning algorithms analyze the behaviors of every request made to the website, categorizing them as human, good bot or bad bot – and reacting accordingly to protect both legitimate customers and SEO.
