An Explanation of Meta Robots Tags
Whilst robots.txt allows you to manage the accessibility of your site’s overall content to crawlers, it doesn’t indicate if the pages should be indexed. That’s where meta robots tags come in – they prevent unwanted webpages from being indexed. However, as there’s a common misconception that robots.txt can control indexation, many SEOs are a little […]