We’ve known for like two decades that Google does and can treat the same URL but with different cases differently. So domain.com/Apple vs domain.com/apple can be seen by Google as different URLs. But Google seems to be stricter about this rule when it comes to the robots.txt file.
Related Posts
Daily Search Forum Recap: September 17, 2021
Here is a recap of what happened in the search forums today…
5 Python scripts for automating SEO tasks
Python is a powerful programming language that has gained popularity in the SEO industry over the past few years. With its relatively simple syntax, efficient performance and abundance of libraries and frameworks, Python has revolutionized how many SEOs approach their work. Python offers a versatile toolset that can help make the optimization process faster, more accurate and more effective. This article explores five Python scripts to help boost your SEO efforts. Automate a redirect map. Write meta descriptions in bulk. Analyze keywords with N-grams. Group keywords […]
Microsoft: 5% Of BingBot Crawls Are New BingBot
Fabrice Canel from Microsoft said on Twitter last night that 5% of all URLs downloaded are over the new BingBot user agent. To take you back, in April, Bing announced a new user agent for its crawler, BingBot.


