| 04/24/2020 – Viswanath – Robots.txt |
The most important for a website, it’s ranking, and ranking will be possible when your website will be indexed on Google or other search engines, so you have to use Google search console to rank the website when we add our website in google search console, then Google’s Robots visit our website. And starting to scan, and also index our website content. (How to add Robots.txt WordPress Blogspot)
-
How to add Robots.txt WordPress Blogspot:
Table of Contents
But sometimes we have rust content on our website, and we want that is not
visible to anyone, we should hide it from others, and for this we use Robots.txt. Robots.txt file gives instructions to our website crawler Which file to index,
and which file to do not index.
| How to add Robots.txt WordPress Blogspot : |
But this does not mean that after you add Robots.txt on your website, a search engine’s robots cannot index Noindex items from your website, Robots.txt is just
an instruction, which helps Google crawlers know What work they have to do.
If you don’t understand now, let me explain by giving you an example, such as Thedailywebsite Robots.txt file, you can find Robots.txt of any website if you
want because no one can hide it.
| Robots.txt file of thedailywebsite : |
| User-agent: * Disallow: / cgi-bin / Disallow: /wp-admin/ Disallow: /archives/ Disallow: /*?* Disallow: /comments/feed/ Disallow: /refer/ Disallow: /index.php Disallow: /wp-content/plugins/ User-agent: Mediapartners-Google* Allow: / User-agent: Googlebot-Image Allow: /wp-content/uploads/ User-agent: Adsbot-Google Allow: / User-agent: Googlebot-Mobile Allow: / Sitemap: https://www.patti88.co.in/sitemap.xml |
How to add Robots.txt WordPress Blogspot
If you want, you can add this file to your website, but before that, you will have
to make some changes, such as the name of your website.
- Let us now understand Robots.txt well.
| User-agent: * Disallow: / cgi-bin / Disallow: / wp-admin / Disallow: / archives / Disallow: / *? * Disallow: / comments / feed / Disallow: / refer / Disallow: /index.php Disallow: / wp-content / plugins / |
This means that I have first allowed User-agent: * to visit my website, by this
any search engine can index my website, so User-agent: * has been written. After that I have written the Disallow code, this means that any search engine crawler
is attached to my website.
But the file that I have disallowed, will not index it. I have only those files here Disallow, which I do not want to show on my website, you can add any other files
of your website, like pages, videos, images, author, etc.
| User-agent: Mediapartners-Google * Allow: / User-agent: Googlebot-Image Allow: / wp-content / uploads / User-agent: Adsbot-Google Allow: / User-agent: Googlebot-Mobile Allow: / Sitemap: https://www.patti88.co.in/sitemap.xml |
How to add Robots.txt WordPress Blogspot
It means, I have to allow some things for my website: It is also an easy way, which Google robots can understand if I want. I can also remove them, it does not matter, but Adding code like this is good for a website indexing.
This makes it easy to understand, and never accidentally Google adds any Disallow page of our website.
| Now the point is that how do we add Robots.txt to our website? |
So, guys, this is a very easy way. If you use Blogger then it is very easy, you have to go directly to the dashboard of your Blogger, and click on the settings there, then click on Search Preference, after that, you will see many options.
You will see The option of Robots.txt will also appear, you have to click there, and add your Robots.txt, but Robots.txt is a bit different on Blogger because on blogger there are no plugins, then you remove these codes Should give.
For this, I have given the below Robots.txt for Blogger, you can use it.
How to add Robots.txt WordPress Blogspot
Blogger Robots.txt file..>>
User-agent: Mediapartners-Google By the way, Robots.txt can be created with the help of Plugin on WordPress, for Where you have to disable your author, tags, category, all these, if all this is Some people say that after doing so, the Robots.txt file of your website gets You do not have to do this, you have to go to the tools section of Yoast SEO, You have to remove that Robots.txt, and paste the Robots.txt that you have Sometimes it gets done within 24 hours, You can add a Robots.txt file from So friends, hope you have liked this article, help your friends, share this article because Sharing is Caring.
Disallow:
User-agent: *
Disallow: /search
Disallow: /b
Allow: /
Sitemap: https://www.patti88.co.in
How to add Robots.txt file to WordPress?
this, you have to install Yoast SEO Plugin, this plugin will be available for free,
after installing you have to go to its Search section.
enabled, then the problem of duplicate content can come on your website.
created, but it never happens, when I first started my website, then I saw many similar tutorials on youtube, then I apply it on my website, but this plugin
which generated Automatic Robots.txt removed my website from the index.
and you will find the file editor there, click on it, now you will see Robots.txt
of your website, which is automatically generated by the plugin.
How to add Robots.txt WordPress Blogspot
copied here, and click save. Just now your website’s Robots.txt file has been
added correctly, but don’t worry, it takes 1 month for a Robots.txt to be correct.
your Cpennel, by adding a txt file, but this is a bit risky.