You find many bloggers and YouTubers about Blogger advising their followers to create bot files for their blogs, and each of them claims that his file is the best, so do I really have to add the bot file to my blog on Blogger?
And the answer is no! You don't have to add any Robots.txt file to your blog unless you want to hide certain links within the blog.
And the question is what should I do? Don't do anything just don't do the tags corresponding to the bot file in the settings and leave it as you found it (in grey) and blogger will create a bot file suitable for your blog.
https://seo1specialists.blogspot.com/robots.txt
Put your blog address in the place of https://seo1specialists.blogspot.com and see the result, you will find that there is a bot file that exists even though you have not added any file for your blog and it is the best bot file especially for new blogs.
Explanation of the default bot file for blogger:
- After we know what the default bot file for blogger is and that no file should be added, let us explain the meaning of the commands in this file for the sake of knowledge.
- Each command in the bot file consists of two parts, the first is user-agent followed by a specific name or *, this line specifies the name of the search engine program or bot, if the name is *, the command includes all programs.
- The second part is a set of commands such as Allow or Disallow, which is to give the search engine what to archive and what to ignore.
User-agent: Mediapartners-Google
Disallow:
- It means allowing the Google Adsense ad crawler to access all your website pages.
User-agent: *
Disallow: /search
Allow: /
- It means allowing all search engines to index your site, the second line means excluding search pages and label pages, and the third line is allowing all the site to be indexed.
- The last part is the type of sitemap that is allowed to be indexed.
We hope that the article has removed a little confusion about the location of the blogger's robot file, if you want to learn more, visit the next page of the official Google website.