Bad bots represent 15.6% of e-commerce site traffic, skewing analytics

Dive Brief:

Bad bots accounted for 15.6% of web traffic on e-commerce sites last year, highlighting a trend that is skewing site analytics, including site traffic, and threatening site security, according to Distil Networks’s Bad Bot Report 2017.
In a blog post reviewing the study results, Distil said 94% of e-commerce sites are hit with bad bots that skew analytics. The study noted that 55.7% of all bad bots can load JavaScript. On an e-commerce site that sees 100,000 visitors per day, if 15.6% of those are bad bots and at least half of those (7.8%) are bad bots loading JavaScript, 7,800 of site visitors are bad bots. This translates to 234,000 bad data points per month.
The report also showed that 97% of websites with pricing and proprietary data get scraped and 96% of logins are hit with bad bots looking to takeover accounts or test stolen login credentials from other sites. Also, 31% of web forms in customer review sections are hit with spammer bots that post negative reviews or ads for competitors.

Dive Insight:

Security hacks and concerns about theft of customer data and credentials represent a rapidly growing problem for e-commerce sites, but Distil's bad bot study also highlights a different sort of threat. Bad bots can help create an inaccurate portrayal of site analytics and data that e-commerce retailers use to make important strategic decisions. They can also be used for what amounts to corporate espionage by competitors who can scrape data about things like future pricing plans and use that to undermine the site from which it was taken.

"However you think e-commerce sites could be abused, these bots are doing it," Edward Roberts, Distil's director of product marketing told Retail Dive. "They can take over so many IP addresses that if you try to handle the problem in-house, it can become a game of 'Whack-a-Mole' that you can't possibly keep up with."

Bad bots are getting very sophisticated in their ability to act like humans, Roberts said, using tools and getting into apps that are designed to emulate actual browser sessions each time they visit an e-commerce site. (Distil recently uncovered the phenomenon of GiftGhostBots, which were capable of quickly testing a rolling list of gift card account numbers with the intent to steal their balances).

Once behind the login page, bad bots also can access shopping carts to scrape more exclusive product pricing — resulting in skewed cart abandonment metrics. Bad bots are also used to arbitrage deals on competitor sites. Bots will reserve inventory while the items are advertised on other sites. They will only complete the transaction on the first site when an item sells on the other site, and whatever doesn’t sell is abandoned.

Roberts said e-commerce sites possess many of the attributes that are like catnip for bad bots — pricing details and proprietary data, logins, payment processors, and web forms. Sites that even have one of those attributes are likely to lure bad bots, so e-commerce sites are hot targets whether they know it or not.

The good news is that more e-commerce retailers are becoming aware of the problem. The 15.6% bad bot rate for last year was slightly lower than the 17% reported for 2015. Some of the difference likely can be accounted for by increasing usage of bot mitigation and cleaning services like those offered by Distil, but as Roberts said, bad bots are getting smarter, more elusive and are capable of large, wide-scale attacks. The bad bot war has only just begun.