To exclude specific collections, blogs, or pages, you can list conditions for them within a snippet. Then add the snippet to the theme.
Publish Insights 3 April 2021
Shopify Generates robot.txt File
At least 37.2% of website visits are from robots indexing content for third-party search engines. This is according to data from the year 2020. [1] A robot.txt file normally resides in the root directory of a website. This instructs robots (bots or spiders) of files or directories that should be excluded from indexing.
Shopify merchants do not have direct access to the root directory [and can now edit*] the robot.txt file. As with nearly any website, you can view it by typing /robots.txt after the domain name. [2]
To exclude specific collections, blogs, or pages with Shopify, list conditions for them within a snippet. Then add the snippet to the theme.liquid layout.
NoIndex, NoFollow, or NoArchive
noindex means that search engines should not index a web page and therefore should not be on the search engine’s result pages. It may follow the links therein to index other pages. nofollow means that search engines should not follow the links on that page.
noarchive instructs Googlebot not to show a cached link in search results. Google will not cache a noindex page. [3] Not all bots adhere to Google protocols.
Exclude Pinterest
While you are at it, you might as well disable Pinterest pinning from specific pages. For example, the blog page changes as you publish new posts. Therefore, this is not a good place for visitors to bookmark within Pinterest.
By default, Pinterest users can save any 200-pixel or larger image unless the page includes <meta name="pinterest" content="nopin" /> or the image tag contains nopin="nopin". [4] Verify your website on Pinterest to validate script.
Modify Liquid Code
Shopify is an e-commerce platform on which sellers build custom websites based on themes. Each theme adheres to similar underlying structure based on programming code called Liquid. [5]
Before making modifications to your Shopify theme, it is good to duplicate it. Within Admin, under the lefthand navigation, choose Online Store: Themes. Click the menu for the active theme, Actions: Duplicate. Work on the duplicate and preview before activating when done.
Caution: Modifying Liquid code may invalidate your theme for future updates by the developer. Anything you add will require manual migration to a different theme, with possible incompatibility.
Now you access the Liquid programming code with Actions: Edit code. Beneath the area called Snippets, click Add a new snippet. Name it bot-control. (Can be any name.) Then add the conditional statements to identify pages you wish to exclude. Examine your robots.txt file to see if they are already preceded by Disallow:.
Below is a sample snippet for modification. Code includes hidden formatting characters that can interfere with operation. Paste into a text editor and reveal invisibles before pasting into Shopify theme. Replace non-breaking spaces ( ) with regular space. Remove HTML line breaks (<br />). You may also need to replace HTML less-than (<) and greater-than (>) with actual characters.
{% comment %} // suppress bots // {% endcomment %}
{% assign nopin = ' <meta name="pinterest" content="nopin" />' %}
{% assign nobot = ' <meta name="robots" content="noindex, nofollow" />' %}
{% assign noindex = ' <meta name="robots" content="noindex" />' %}
{% capture bots -%}
{% if template == 'index' or template == 'collection' -%}{{ nopin }}
{% elsif template == 'search' or template == 'blog' -%}{{ nopin }}
{% elsif template == '404' -%}{{ nopin }}
{{ nobot }}
{% elsif template == 'article' and article.tags contains 'meta-nopin' %}{{ nopin }}
{{ nobot }}
{% endif -%}
{% endcapture -%}
{{ bots | strip_newlines -}}Refer to the robots.txt file to prevent redundancies. It already excludes customer accounts and search pages from indexing, but you may wish to exclude them from Pinterest if they have images larger that 199 pixels in height or width. You can control whether to exclude an article or product page by adding a tag.
Now add the snippet within the header area of theme.liquid under Layouts.
{%- render 'bot-control' -%}It is possible to add all the code to theme.liquid and forego the snippet, but most programmers prefer to keep the theme file as clean as possible.
If your web browser has developer mode, you can visit the pages to see the meta exclusion within the header. Satisfied? Close edited templates before exiting the code area. Then you’re done. Be aware that not all site crawlers adhere to NoFollow instructions; think of them as recommendations.
When you sign up for Google Search Console reports, you may receive emails warning you that directories or pages are no longer indexable. You will receive similar alerts if you subscribe to Ahrefs reports. This is a slight downside of including noindex in the header rather than the robots.text file. The warning should not affect ranking.
