Shopify robots.txt: Where It Is, What It Blocks and How to Edit It

Where is the robots.txt file on Shopify?

Open yourstore.com/robots.txt in a browser. Every Shopify store serves one, and you did not have to create it. Shopify renders it from a template called robots.txt.liquid, which is not included in any theme by default, so until you add your own copy you are running Shopify’s generated version.

Google’s documentation says a robots.txt file applies only to the host, protocol and port it is served from. Check the domain your customers actually use, and if you run separate domains or subdomains, check each one.

What does Shopify’s default robots.txt block, and why?

The live default was read from three Shopify stores on 28 September 2026 and all three share one structure. The main group, User-agent: *, applies to every crawler and does four jobs.

The reasoning is visible in the file itself, because Shopify labels the blocks with comments. Each rule is about where crawlers waste time or where a page is private, not a judgement about your products.

1. 01Private and transactional pages: /admin, the cart, /checkout and /checkouts/, /orders and /account, each also listed under a language prefix such as /*/cart/.
2. 02Shopify-internal endpoints: /services and /sf_* paths, plus the /cdn/wpm/ script files.
3. 03AJAX surfaces: /cart.js and /recommendations/products, which return data rather than pages.
4. 04Crawl traps: collection and blog URLs containing sort_by, URLs where tags are joined with a plus sign (/collections/*+*), collections with two or more filter parameters, language-picker duplicates, and preview parameters such as preview_theme_id.
5. 05A separate adsbot-google group, because Google’s ad crawler ignores the * group unless it is named. It lists the public product, collection, page and blog paths as allowed and repeats the checkout and order blocks.
6. 06A Sitemap line pointing to your /sitemap.xml at the bottom.

Should you edit it at all?

Most stores should not. Shopify’s developer documentation says the default file works for most shops, and that the default rules are updated regularly so that SEO best practice keeps being applied. If you freeze the file into plain text, those updates stop reaching you.

Shopify also treats the edit as unsupported. Its help page says Support can’t help with edits to robots.txt.liquid, that incorrect use can result in loss of all traffic, and that you can hire a Shopify Partner if you need a hand. Edit only for a reason you can name: a crawler you want out, a parameter URL you have seen wasting crawl in Search Console, or an AI crawler you have decided to admit or refuse.

How do you edit robots.txt.liquid on Shopify?

Add the template once, then change it a line at a time. On the theme you have published, the steps from Shopify’s help page are below. Changes are instant on your side, although crawlers do not react immediately.

Before you save, keep the loop that prints the default groups (the for group in robots.default_groups block that Shopify puts in the new file). That loop is what keeps your file in step with Shopify’s updates. Everything you add goes inside it, after the rule lines, or outside it below the closing tag.

1. 01In Shopify admin go to Online Store, then Themes.
2. 02On your live theme choose the three-dot menu, then Edit code.
3. 03In the Templates folder choose Add a new template and pick robots, then Create template.
4. 04Read the Liquid it generates. It loops over robots.default_groups and prints each group’s user agent, its rules and its sitemap line.
5. 05Add your change (examples in the next sections) and save.
6. 06Open yourstore.com/robots.txt and check that the change appears and that the default rules are still there.

How do you add a Disallow rule?

Inside the loop, test which group you are in and print your rule when it is the * group. Shopify’s own example blocks every crawler from URLs with a ?q= parameter. After the line that prints the rules, add an if block that checks whether group.user_agent.value equals *, and inside it output the text Disallow: /*?q=*.

The same pattern works for any pattern you have seen bloating the Pages report in Search Console. Match the shape of the URL, not one URL: /*?q=* catches every path that carries the parameter, while a rule for a single URL is better handled with a noindex tag on that page.

1. 01One parameter: Disallow: /*?q=* (Shopify’s documented example).
2. 02One folder: Disallow: /pages/old-campaign/
3. 03Rules are case-sensitive, so /Pages/ and /pages/ are different paths.
4. 04Keep the rule on its own line and inside the * group, so it sits with the defaults.

How do you remove a default rule?

Wrap the line that prints each rule in an unless test. Shopify’s example removes the rule that blocks /policies/: it skips any rule whose directive is Disallow and whose value is /policies/. Every other rule still prints.

Check your live file first. Shopify’s documentation uses /policies/ as its example, but the three current default files read on 28 September 2026 did not list it, so on a current store you may have nothing to remove. Remove a rule only when you have a reason and you have opened the URL it was blocking.

How do you block or allow a specific crawler, including AI crawlers?

Crawlers not in the default set get their own group, written as plain text outside the loop. Shopify’s examples use the discobot crawler: a User-agent: discobot line followed by Disallow: / blocks it, and Allow: / permits it. You can add an extra Sitemap line the same way.

One rule from Google’s documentation trips people up here. A crawler matches only one group, the most specific one that names it. If you write a group for a named bot and give it only Allow: /, that bot no longer reads the * group, so the cart, checkout and account blocks stop applying to it. Repeat the private-path blocks in any group you create.

For AI crawlers, OpenAI documents three. OAI-SearchBot is what surfaces sites in ChatGPT search, and OpenAI says it can take about 24 hours to adjust to a robots.txt change. GPTBot is used to crawl content that may be used in training. ChatGPT-User visits a page when a person asks ChatGPT about it, and OpenAI says robots.txt rules may not apply to those. So a block on GPTBot does not stop ChatGPT reading a page a shopper hands it. The argument for and against admitting AI crawlers is in Robots.txt and your AI buyer, and there is no need to repeat it here.

1. 01Refuse training only: User-agent: GPTBot followed by Disallow: /
2. 02Keep search visibility: leave OAI-SearchBot out of any block, or give it a group that repeats Disallow: /cart/, /checkout, /orders and /account.
3. 03Extra sitemap: a line reading Sitemap: followed by the full URL.
4. 04Stores on Shopify Markets with separate domains can test request.host in the loop to give one domain its own rules.

Which robots.txt mistakes take a Shopify store out of Google?

Nearly all the damage comes from a rule that is broader than intended. A Disallow: / under User-agent: * closes the whole store. A pattern such as Disallow: /collections/ or /products/ closes the pages that earn the traffic. Shopify itself warns that incorrect use can result in loss of all traffic, so treat every edit as a change to revenue.

The other errors are quieter. Blocking a page in robots.txt does not remove it from Google: Google says a disallowed page can still be indexed if other sites link to it, and it says robots.txt is not a mechanism for keeping a page out of Google. Worse, a blocked page cannot be crawled, so Google never sees a noindex tag on it. Use noindex to keep a page out, and leave the page crawlable so the tag is read.

1. 01Pasting a hand-written file over the Liquid. The rules stop updating, and Shopify says it can’t ensure best practice is applied.
2. 02Keeping an old workaround such as a Cloudflare-managed robots.txt. Shopify says to remove those before using the template.
3. 03Blocking a URL to hide it, when a noindex tag or a password is the tool for that.
4. 04Copying a file from another store or an old blog post, including rules for apps you do not use.
5. 05Forgetting the template belongs to a theme. It sits in that theme’s templates folder, so after a theme switch open /robots.txt and confirm your rules are still live.
6. 06Exceeding the 500 KiB limit Google reads, or breaking the file so it returns an error page. Google treats most 4xx responses as no robots.txt at all.

How do you test a robots.txt change?

Test in three places, in this order. First open yourstore.com/robots.txt yourself and read it top to bottom: the default groups should be intact, your addition should be where you meant it, and the Sitemap line should be present.

Second, use Google Search Console. The robots.txt report lists the files Google found for your top hosts, when it last crawled them, and any warnings or errors, and it lets you request a recrawl. For one specific URL, the URL Inspection tool tells you whether robots.txt blocks it. Shopify’s help page still points to Google’s robots.txt Tester, so check that it is available to you before relying on it.

Third, check what a product page delivers to a crawler. The free AI readiness checker fetches a product page the way a crawler does, without JavaScript, and reports among other things whether the page is allowed to be indexed. A blocked or noindexed product shows up there before it shows up in your traffic.

Allow time. Google caches robots.txt for up to 24 hours, and may keep an older copy longer when it can’t refresh it.

How do you undo a change?

Delete the robots.txt.liquid file from the theme’s Templates folder, or paste back the default loop. Because Shopify does not include the template in any theme by default, removing it returns the store to the generated file. Then reload /robots.txt to confirm, and use Search Console’s robots.txt report to ask Google to recrawl it.

If a store lost traffic after an edit, undo first and investigate second. Recovery still takes the time it takes Google to recrawl and reprocess the URLs, so an early undo is worth more than a careful diagnosis.

What should you check next?

Robots.txt is one gate among several. If pages are crawlable but still missing, read the guides on Google Search Console setup, on the sitemap that Google could not fetch, and on why Google says crawled, currently not indexed. If you are starting from the beginning, the beginner checklist for getting a Shopify store onto Google covers the storefront password and noindex tags that sit alongside robots.txt.

Frequently asked questions