Skip to the page
products and stores

How Depra AI reads your store catalogue

What DepraBot asks a Shopify or WooCommerce store for, in what order, how often, the limits of one scan, and how to allow or block it.

Updated

When you set up a Depra AI project for your own site, DepraBot checks whether the site is an online store. If it is, DepraBot reads the store's product list, so Depra AI knows which products are yours. DepraBot is the web reader behind Depra AI. The store read is in early access and is opened per workspace.

A Depra AI app for Shopify is coming to the Shopify App Store. When it is listed, it will read your products, collections and store pages. Until then, setup needs no Shopify app and no access token. This page says what DepraBot asks your store for, how often, and how to allow or block it.

Note: Depra AI never changes a store. DepraBot only reads.

How does Depra AI read a store's product list?Link to this section

How DepraBot reads the list depends on the platform your store runs on. It asks the platform's product API first. An API is an address that returns data for software to read. If no API lists products, DepraBot reads the store's sitemaps.

How DepraBot reads the product list, by kind of store
StoreWhat DepraBot asks for
On ShopifyThe Shopify Storefront API on the store's own myshopify.com address, without an access token. Then /products.json, when the Storefront API did not return the full list.
Not on ShopifyThe WooCommerce Store API, at /wp-json/wc/store/v1/products.
No API lists productsThe store's sitemaps, then up to 3 product pages listed in them.

A sitemap is the file in which a site lists its pages. To learn whether a store runs on Shopify, DepraBot asks for /meta.json, and then for /.well-known/ucp if needed.

In what order does DepraBot ask for files?Link to this section

One scan asks for these files, in this order, and skips the ones that do not apply.

  1. 1. robots.txt: for its rules and the sitemaps it names.
  2. 2. /meta.json, then /.well-known/ucp if needed: to see whether the store runs on Shopify.
  3. 3. The Shopify Storefront API: on the store's own myshopify.com address, without an access token.
  4. 4. /products.json: on a Shopify store, when the Storefront API did not return the full list.
  5. 5. The WooCommerce Store API: on a site that is not on Shopify.
  6. 6. The store's sitemaps: when none of the APIs above lists any products.
  7. 7. Up to 3 product pages: listed in those sitemaps.
  8. 8. The homepage: only when nothing above shows that the site sells products.

What are the limits of one scan?Link to this section

Every scan stays inside these limits.

Limits of one store scan
LimitValue
Files asked forAt most 80 in one scan, not counting robots.txt
Request rateAt most one request a second to any host
TimeNo new request starts after 2 minutes
RedirectsAt most 3 per file
Download sizeAt most 3 MB of a page, 2.5 MB of a JSON file and 10 MB of a sitemap
Product pagesUp to 3, taken from the sitemaps
Scans at onceOnly one scan runs for a project at a time
Headless browserNever used. A headless browser loads a page with its scripts, the way a normal browser does.

These numbers are published on About DepraBot.

When does a scan run?Link to this section

In an early-access workspace, a scan runs at three moments: at setup, once a week, and when someone starts one with Fetch.

  • At setup: when you set up the project for your site.
  • Once a week: scheduled between 00:30 and 02:30 UTC, which is 06:00 to 08:00 in India. The weekly scan does not run if you have told Depra AI that you do not sell products.
  • With Fetch: when someone at your brand starts a scan with Fetch.
Store scans you can start with Fetch, by plan
PlanFetch scans per project per day (early access)
Starter3
Growth3
Scale5
Enterprise5

The day is counted in UTC. Only one scan runs for a project at a time. The other plan limits are on plans and limits.

What if robots.txt blocks DepraBot?Link to this section

DepraBot obeys robots.txt, the file that tells bots what they may read. If your robots.txt keeps DepraBot out, a scan reads nothing past robots.txt itself. You can upload your product list as a file instead.

If your robots.txt answers with a server error, or does not answer at all, DepraBot treats the whole site as disallowed and comes back later. The one exception: if it read the file successfully in the previous 24 hours, it keeps using that copy.

Warning: DepraBot uses no proxies and never disguises itself. A firewall or bot filter may answer it with a 403 or 429 error, or with a challenge page that asks the visitor to prove it is human. DepraBot records each of these as a block.

How do I allow DepraBot in robots.txt?Link to this section

Add a group for DepraBot to your robots.txt. A group is a User-agent line with the rules under it. DepraBot looks for a group named DepraBot first, then the group for all bots, which is marked with a star. Under the robots.txt standard, a bot obeys the group that names it and falls back to the star group only when no group names it (RFC 9309 (opens in a new tab), checked 7 Oct 2026). So the DepraBot group decides, even when the star group blocks everything.

text
User-agent: DepraBot
Allow: /
robots.txt: allow DepraBot on the whole site.

The user agent token, the name robots.txt rules use for a bot, is DepraBot. Every request carries the user agent string DepraBot/1.0 (+https://depra.ai/about-bot). DepraBot obeys robots.txt as written in RFC 9309. It also honours a Crawl-delay line, which asks a bot to wait between requests, of up to 10 seconds.

To keep DepraBot out, use Disallow in the same group.

text
User-agent: DepraBot
Disallow: /
robots.txt: block DepraBot on the whole site.

DepraBot re-reads robots.txt within six hours, so a change takes effect within that time. If you want DepraBot kept out and cannot change robots.txt, email team@depra.ai with your domain. The team will add it to the exclusion list.

What does DepraBot never read?Link to this section

A store scan stays on your own store. It never reads the following.

  • Other sites: it reads only your own site, its subdomains and the store's own myshopify.com address.
  • A myshopify.com address typed into Depra AI: it is asked only for its robots.txt and /meta.json, unless that file names your site.
  • Redirects to another site: they are not followed.
  • Marketplaces and social sites: a marketplace or social site entered as a brand's website gets no request at all.
  • Anything behind a login: DepraBot does not sign in, fills in no forms and uses no access token. It cannot read orders or customers.
  • Images and media: no images, media, fonts, scripts or stylesheets are requested.
  • Pages robots.txt disallows: DepraBot obeys robots.txt for every page and file it asks for.

Nothing DepraBot reads is used to train AI models, and nothing is republished. The full policy is on About DepraBot.

Frequently asked questionsLink to this section

Do I need to install a Shopify app or share an access token?

No. Setup needs no Shopify app and no access token. DepraBot reads the Shopify Storefront API on your store's own myshopify.com address, then /products.json when the Storefront API did not return the full list.

Can DepraBot read my orders or customers?

No. DepraBot does not sign in, fills in no forms and uses no access token, so it cannot read orders or customers. Depra AI never changes a store, and DepraBot only reads.

How often does DepraBot scan my store?

A scan runs at setup, once a week and when someone at your brand starts one with Fetch. Fetch allows 3 scans per project per day on Starter and Growth, and 5 on Scale and Enterprise. The day is counted in UTC.

My robots.txt blocks DepraBot. Can Depra AI still get my product list?

Yes. You can upload your product list as a file instead. To let DepraBot in, add a DepraBot group with Allow: / to your robots.txt, and DepraBot re-reads robots.txt within six hours.

Is my store data used to train AI models?

No. Nothing DepraBot reads is used to train AI models, and nothing is republished. The full policy is on About DepraBot.

Keep reading

support

Still have a question?

The docs explain how Depra AI works, one topic per page. When they do not answer your question, ask the Depra AI team.

  • Help centre

    Short answers to common questions. Each help page answers one question.

    Open the help centre
  • Contact the team

    Write to team@depra.ai. The team aims to reply within an hour during working hours in India (IST).

    Contact us
  • Book a demo

    The Depra AI team sets up your account with you on a free call. Your trial still runs its full 3 days.

    Book a Demo

Be the brand AI recommends

Track your brand on ChatGPT, Gemini, Perplexity, Google AI Overviews and every other major AI engine, and get a ranked list of fixes that help AI name you.