Want to improve and secure your blog? looking for How to Add Robots.txt in Blogger Blogspot/Blog Guide, right?
Don’t worry, this guide will help you add custom Robots.txt file in blogger. Here I’ll guide you with how to create perfect Robots.txt file for your WordPress or Blogger Blog (website).
With the help of robots.txt, you can allow search engines / web crawlers to scan pages or posts. In other words, you can control from Robots.txt which specific area should not be scanned by the search engine and then it will not index in the search engine result app, so the user will not be able to see the page or post
Table of Contents
What is Robots.txt File?
First we’ll discuss what Robots.txt file is? As it’s name Robots file especially created text file used for web crawlers, search engine bots. It helps to communicate website with search engine bots, web crawlers.
The robots.txt file contains a set of commands written in an organized way with which you can exclude a particular path / area of the website, then the bot will not be allowed on that specific defined area or path and therefore the user specified path Can not reach or area of your website.
The robots.txt file will direct the web crawler which part of the website should not be scanned and which part should be scanned.
How to Create Custom Robots.txt file for Your Blog?
Yes, you can create custom Robots.txt file and add it on blogger or WordPress website. Robots.txt is also very important for your blog, if you’re new and updating daily blog on blogspot.
Follow this ultimate guide of How to Add Robots.txt in Blogger / Blogspot blog. Below you can create perfect custom Robots.txt file for your blog. It will help web crawlers to communicate with your website.
Follow the below steps to create custom Robots.txt file –
1) Open Notepad and paste below text in your notepad file
User-agent: * Disallow:
2) Now save your file with the name robots.txt.
Now let me explain what is meaning of these 2 lines –
1 – user-agent:* – We have used user-agents for web crawlers where * (star) means that all web crawlers are allowed to visit the blog. Or allowed for scanned my blog.
2 – Disallow: – With this tag web crawlers will not be allowed to visit particular area of the website. Like if I used
disallow: /2020/03/guide-to-start-a-blog/
Follow Ultimate Guide on – How to Start a WordPress Blog
Then users and web crawlers (search engine bots) will not be allowed to visit this particular page because we have declined to scan this page or post for search engines. Here you can also add your blog post or page URL.
3 – allow: With this, web crawlers or bots are allowed to scan this specific page like Sitemap.
Now let’s move on our next and final step is, how to add Robots.txt in Blogger.
[ss_click_to_tweet tweet=”Now I know What is Robots.txt file, and how it helps to my website to communicate with web crawlers or search engines. I can now create custom robots.txt file.” content=”Now I know What is Robots.txt file, and how it helps to my website to communicate with web crawlers or search engines. I can now create custom robots.txt file.” style=”default”]How to Add Robots.txt in Blogger?
Here we’re on final step i.e. how to add custom perfect Robots.txt file in Blogger. Follow the below given step-by-step guide of how to add robots.txt in blog.
1 => Visit Blogger.com
2 => Now login with your Google Account Email and Password.
3 => After Login go to Setting.
4 => There scroll down below or search for “Crawlers and indexing“.
5 => Now Select “Enable custom robots.txt“.
6 => Below it click on “Custom robots.txt“.
7 => And Click on Save Button.
8 => You’re done.
Finally, you can now add robots.txt your webmaster search console too. I hope this walk through will help you for how to add robots.txt in blogger.com.
Conclusion –
I hope you understand now what is Robots.txt and why it is used in blogger or wordpress website. If this guide help you then please share this article on social media platform will help needed to do it.
If you’ve any query or question on adding custom robots.txt in blogger then write/comment down below in the post. I’m available here to answer your question.
Related Post – Download These Top 5 Best Chrome Extension for Bloggers