Search to Table

Fix guide

A robots.txt for restaurant websites

Your robots.txt is missing, or it is blocking the search engines and AI assistants you want reading your site.

Why it matters

robots.txt is the first file a crawler asks for. One wrong line can hide an entire site from Google — we see "Disallow: /" left over from a site launch more often than you’d think.

It is also where you point crawlers at your sitemap so new pages get found quickly.

How to fix it

  1. Copy the file below.
  2. Save it as robots.txt at the root of your site, so it loads at yoursite.com/robots.txt.
  3. Most builders manage this file for you; if yours does, just make sure nothing important is disallowed.

The code

This is the fix our audit generated for a sample restaurant. Run the audit on your own site and the fix pack fills in your name, address, phone and hours for you. Anything in CAPITALS is a value we couldn’t find — fill it in, never guess.

txt
User-agent: *
Allow: /

# Let AI assistants read the site so they can recommend you.
User-agent: GPTBot
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: PerplexityBot
Allow: /

Sitemap: https://www.tacoselpuente.example/sitemap.xml

Where it goes: Save as robots.txt at the root of your site.

More fixes

Not sure which of these you need? Work through the checklist or run the free audit below.

Check your own site

Free score, plain-English report, and the code to fix it. About a minute.

No account. Nothing changes on your site.