March 2021 Newsletter

Bringing you curated Web Scraping, SEO and Digital Marketing news from around the web

March 2021 Newsletter

March 2021 Newsletter 150 150 admin
Cherry Blossom

Spring is in the air and it’s time to look ahead, so this month I’ve chosen to share with you a crystal ball, a final warning for poor mobile sites to shake themselves up, and a bit of ‘handbags at dawn’. Enjoy!

Web Scraping: Looking Ahead To 2026

Fortune Teller

When divining future movements and positions, short-term confidence is fine, but at some point in the distance, you’re not going to succeed without a crystal ball.

Fortunately, Big Market Research has helped out and released their ‘Web Scraping Software Market’ study, which peers into 2026.

Officially titled “Web Scraping Software market: Global Opportunity Analysis and Industry Forecast, 2026”, they’ve involved some key players, with the likes of Scrapinghub,, Mozenda and Octopus Data, to name a few.

As you might expect, the forecast is a collection of insights such as profitability, market size and market share, along with regional growth and sector by sector dynamics.

There’s a strong focus on challenges, growth opportunities and trends, and new application opportunities for web scraping. It doesn’t exclude the negatives either and includes any limitations and restraints that could dent growth and impact opportunities. As you’d expect, the shadow of coronavirus looms large too, and this isn’t overlooked.

The downside for this knowledge? It’s not cheap, coming in at over $3k, however you can request a discount here.

Final Call For Broken Mobile Sites

Times Up

It’s been almost five years since Google launched mobile-first indexing and one year since Google announced that they would move every site over to mobile-first.

Back then, Google had already moved around 70% of sites across, and the expectation was that the switch would complete around September 2020. However, as with pretty much everything else planned over the last year, that date was pushed back.

According to Google doyen, John Mueller, Google is prepping for the last batch of sites to move across to mobile-first indexing. That will likely take a couple of months, and once completed, mobile user-agents will become Google’s standard user agent when indexing sites.

But what does that mean for the average website, including small business sites that don’t have a mobile site? Fear not. If a site is built for desktops, there’s no need to panic. Google using a mobile user-agent, rather than a desktop one, won’t affect you in any way.

However, any sites with separate desktop and mobile sites, with issues such as different content, broken links, wrong schema, etc., on their mobile site will likely face problems and pay for it in the SERP rankings.

That’s because when Google use their mobile user-agent, any problems on a mobile site are going to stick out like a sore thumb, rather than what might be a squeaky clean and optimised desktop site.

Here’s what John had to say specifically:

“We have the deadline, I think, set for, we essentially had it set for March. I think there are some technical details with the last sites that we are working out. So we can switch them over in an optimal way. But at some point, it will just be switched over. If that is still in March or maybe in April or maybe even ends up in May, I don’t know. But we are going to be switching all of these sites, and if your site is ready, then that will just happen then. I think from what I heard from the team we are currently in a state where we are not automatically moving sites over anymore but rather saving all of these up for that last moment and switching a lot of those over at once.

“So if you improved your site for mobile-first indexing, then that will just happen. And if it is already showing the same content on mobile that it shows on desktop, then you won’t see any effect. It is not like you will have a ranking boost or any kind of improvement in indexing because of that.”

Want to check out the video? The transcript is taken from the 45:25 mark.

Microsoft vs Google: The Blog War Is Back

Finger Pointing

The Australian government finally passed legislation forcing digital platforms such as Google and Facebook to pay for local media outlets’ content.

Now it’s fair to say those two powerhouses didn’t exactly put out some good PR at the time, though Google has now introduced News Showcase in and effort to make amends and are looking to move on.

So far, so good? If you’d think the dust would settle now, Microsoft’s president, Brad Smith, has other ideas.

He decided to rile up Google by agreeing wholeheartedly with the legislation, even going so far as to say their endorsement had a significant influence on Google’s decision to back down.

Clearly annoyed, Google fired back and brought the recent hack on Microsoft’s SolarWinds to the fray:

“Microsoft’s newfound interest in attacking us comes on the heels of the SolarWinds attack and at a moment when they’ve allowed tens of thousands of their customers… to be actively hacked via major Microsoft vulnerabilities.”


Why does any of this matter? Watching two of the largest companies on the planet bounce tit-for-tat finger-pointing between each other might be entertaining, but there is a deeper layer to be found here.

Google’s retort appeared just as the US government’s House Judiciary antitrust subcommittee heard Brad Smith’s testimony on local news media outlet’s and whether they should arrange payment terms with Google and Facebook.

These hearings have been called after a report released in 2020 concluded that journalism, and by extension democracy, is harmed by tech companies utilising the data they collect from online news readership to control the digital advertising landscape. The crux is that those same media companies also rely on digital advertising, and the fear is that any oligopoly could cause them long-term financial damage.

We all know that Brad is not going to be sympathetic to Google’s perspective, and Google wanted to make it clear that Microsoft bring along some baggage of their own. Is this little spat over now? Possibly. But don’t be surprised if this old enmity continues.

And that’s all from me this month. Thanks for reading all the way down to here!

All the best,

Paul Holmes

Head of Sales Operations and Partner Development

Leave a Reply

Back to top