Now Blogger has great feature for search engine visibility.Now we can visible our blog in search engines with robot.txt. To active this feature go to
Settings | Search Preferences, click
Edit next to "
Custom robots.txt" and select Yes. After this selection text box will be visible to give the crawler code for your site.
Note: which was mention in blogger site
Warning! Use with caution. Incorrect use of these features can result in your blog being ignored by search engines.
by default the robot.txt has following code,
User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search
Allow: /
Sitemap: http://www.example.com/feeds/posts/default?orderby=updated
User-agent: Mediapartners-Google is a robot from Google Adsense, leave it as is because if you mistakenly change that than the ads served will not fit with your content.
The next line is for all the robots and marked with an asterisk (*). On the default configuration, it is clear that the label of our blog is not indexed
Disallow: /search.
Keep in mind that a slash
(/) is as your homepage, so for example if you want the label to get indexed, do not just fill up with a slash like this
Disallow: / because that would be you do not allow the robot tracing your blog
Finally your code can be like bellow,
User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search?updated-min=
Disallow: /search?updated-max=
Disallow: /search/label/*?updated-min=
Disallow: /search/label/*?updated-max=
Allow: /
Sitemap: http://www.youtdomain.blogspot.com/feeds/posts/default?orderby=updated
After the changes, make sure everything is fit like what we want by visiting www.example.com/robots.txt. Replace the Example.com with your domain name.