Do I need a special file or format for AI agents?

No. There's no required agent file. Files like llms.txt are aimed at language models reading content, and their benefit for AI visibility is unproven. Agents operating a page rely on the page itself: its HTML, its labels and its forms. Clean, accessible markup does more for agents than any extra file you add to the root of your site.

Can AI agents see content loaded by JavaScript?

Agents inside a browser usually run JavaScript, so they can see content that crawlers miss. That doesn't make script-heavy pages safe. Content that loads late, sits behind interactions or renders into canvas is still harder to find and read. Server-rendered HTML works for crawlers, agents and slow connections alike, so it's the safer default for key facts.

Will AI agents fill in my lead form with fake data?

An agent fills a form with the details its user gives it, so a well-built form should receive real leads, sometimes faster. Some submissions may carry less context than a human would type. Ask for the few fields you truly need, use clear labels and consider an optional notes field so agent-submitted requests still give your sales team something to work with.

Can I tell if an AI agent visited my website?

Sometimes. Agents inside a user's browser can look like ordinary visits, and vendors identify their traffic in different ways. Referral data in analytics can show visits from AI products when a link is clicked, which is covered in our guide to tracking ChatGPT traffic in GA4. Check each vendor's current documentation for how its agent identifies itself.

Is an agent-friendly website the same as an accessible website?

Mostly, but not entirely. Accessible markup covers the biggest part: named controls, labeled forms, keyboard support and logical structure. Agent readiness adds a few extras, such as keeping facts in plain HTML, putting state in URLs, avoiding stacked pop-ups and making confirmations explicit. If you meet WCAG 2.2 AA, you've done most of the work.

All posts

Web design & development

Agent-Friendly Websites: Designing for AI Browsers Like Atlas and Comet

How to build an agent-friendly website for AI browsers like ChatGPT Atlas and Comet: semantic markup, labeled forms, plain navigation and how to test it.

By Mehdi HassanNov 2, 20269 min read

The short answer

An agent-friendly website is one an AI browser agent, such as the agents in ChatGPT Atlas or Perplexity Comet, can read and operate without a human stepping in. It uses real links and buttons with clear names, labeled form fields with autocomplete attributes, navigation that doesn't depend on hover, and key content in server-rendered HTML. Most of it is accessibility work you should do anyway.

Key takeaways

AI browser agents act on your site for a user, so they need to click, type and submit, not just read.
Accessible markup is the best agent markup: names, roles and labels tell an agent what each control does.
Forms need visible labels, standard autocomplete values and text error messages an agent can read.
Hover-only menus, div buttons, endless carousels and modal stacks are what trip agents up most.
Test with a real agent on three buyer tasks, then fix the step where it stalled.

Part of How to Build an AI-Search-Ready Website: Structure, Schema and Crawlability

What is an agent-friendly website?

An agent-friendly website lets an AI agent complete a task on it, such as finding a price, comparing plans or booking a demo, the same way a person would. The agent reads the page, decides what to click, types into fields and checks the result. If a control has no name, a menu only opens on hover or a form hides its errors, the agent stalls.

That's different from being visible in AI search. Our guide to AI-search-ready websites covers how answer engines crawl, read and cite your pages. Agents go a step further: they visit your site live, on a user's behalf, and act. A site can be cited well by ChatGPT and still be hard for an agent to use.

How do AI browsers like ChatGPT Atlas and Comet read a page?

AI browsers such as OpenAI's ChatGPT Atlas and Perplexity's Comet run an agent inside a real browser, so they see the rendered page rather than a crawler's snapshot. Agents of this kind generally combine what the page looks like with the structure underneath it: the elements, their text and the roles and names exposed to assistive technology. The exact method varies by vendor and changes often, so check each one's current documentation.

What the agent needs	What helps	What gets in the way
Know what a control does	Real button and link elements with text or an aria-label	Clickable divs, icon-only buttons with no name
Find the right page	Plain nav labels: Pricing, Features, Docs, Contact	Clever labels, mega menus that open only on hover
Read the facts	Content in server-rendered HTML and tables	Text inside images, canvas or script-only widgets
Fill in a form	Visible labels, input types, autocomplete values	Placeholder-only labels, custom date pickers
Know it worked	A text confirmation and a clear URL change	A silent toast that vanishes in two seconds

Why does accessible markup make a website agent-friendly?

Accessible markup makes a site agent-friendly because it already describes every element's name, role and state in a form software can read. WCAG 2.2 success criterion 4.1.2, Name, Role, Value, asks for exactly that so screen readers can operate a page. An agent working from the same structure benefits from the same discipline.

This is why we treat agent readiness as a side effect of good accessibility, not a separate project. Headings in order, one main landmark, lists marked as lists, tables built as tables and links whose text makes sense alone all help a screen reader user and an agent at the same time. Our overview of website accessibility for startups explains the WCAG Level AA target in full.

How do you design forms an AI agent can fill in?

Design forms for agents the way you'd design them for a careful, literal person: one visible label per field, the right input type, standard autocomplete values and errors written as text next to the field. The demo request and signup forms are the part of a marketing site agents are most likely to be asked to complete, so they matter most.

Labels: a visible label element tied to each input, not a placeholder that disappears when typing starts.
Input types and autocomplete: type set to email, tel or url, and autocomplete values such as email, given-name, organization and tel. WCAG 2.2 criterion 1.3.5, Identify Input Purpose, asks for this at Level AA, and MDN's autocomplete reference lists the values.
Choices: native selects, radio buttons and checkboxes instead of custom dropdowns built from divs.
Errors: a text message beside the field that says what's wrong and how to fix it, not just a red border.
Confirmation: a success message in the page or a thank-you URL, so the agent and the user can see it worked.

Then think about friction you add on purpose. CAPTCHAs and puzzle challenges stop bots, and they stop helpful agents too. WCAG 2.2 criterion 3.3.8, Accessible Authentication (Minimum), already discourages cognitive tests at login. Where you need spam protection, prefer invisible checks and rate limits over puzzles in front of a demo form.

The navigation patterns that trip up agents are the ones that hide content until a specific pointer action happens. Agents can hover and scroll, but every hidden step is another chance to guess wrong, time out or give up and report that the information wasn't there.

Hover-only menus: make menus open on click and keyboard too, and link key pages directly in the footer.
Facts in tabs and accordions only: fine for people, but keep the content in the HTML and give each tab a real name.
Endless carousels and infinite scroll: put important items, such as plans or case studies, in a static list.
State that isn't in the URL: if a pricing toggle switches to annual billing, reflect it in the URL or show both prices.
Stacked overlays: a cookie banner, a chat bubble and a newsletter pop-up on the same load can block the page entirely.

For example, suppose an agent is asked to book a demo

For example, suppose a user tells an AI browser agent: 'Book a demo with this hypothetical analytics startup for next Tuesday afternoon.'

On the first version of the site, the agent lands on the homepage, dismisses a cookie banner, then meets a newsletter pop-up. The nav's 'Get started' item opens a mega menu on hover only. The demo link inside it is an icon. The agent finds the form through the footer, but the fields use placeholders as labels and the company-size dropdown is a custom div list it can't open. It reports back that it couldn't finish.

On the fixed version, 'Book a demo' is a real button in the header, the form has labeled fields with autocomplete values and a native select, and the scheduler embed has named, keyboard-reachable time slots. The agent completes the booking and reads the confirmation text back to the user.

Should you block or allow AI browser agents?

For most marketing sites, allow them. An agent booking a demo or comparing your pricing is acting for a real buyer, and blocking it means that buyer hears about your competitors instead. The decision is different for logged-in products, checkout flows or anything with abuse risk, where you may want confirmation steps or rate limits.

Keep in mind that robots.txt was designed for crawlers, and an agent running inside a user's browser doesn't behave like a background crawler. How each vendor identifies its agent and which rules it honors varies, so check current vendor documentation rather than assuming. Our article on blocking or allowing AI crawlers covers the crawler side, and aggressive bot protection at your CDN can block agents even when robots.txt allows them.

How do you test whether your website works for AI agents?

Test by giving a real agent the tasks your buyers would give it, then watching where it stalls. Pair that with a keyboard and screen reader pass, because most agent failures show up there too, and a check that your key content is in the server-rendered HTML.

Write three buyer tasks
For example: find the price of the team plan, check whether you integrate with HubSpot, and book a demo. Use the wording a real buyer would.
Run them through an agent
Use an AI browser or an agent mode and note every step where it hesitates, picks the wrong control or reports missing information.
Check the HTML
View the page with JavaScript off or use a fetch tool to confirm pricing, features and forms are in the initial HTML. Server-side rendering for AI crawlers explains why it matters.
Do a keyboard and screen reader pass
Tab through the nav and forms and listen for unnamed buttons. Every gap a screen reader finds is likely a gap for an agent.
Fix, then rerun the same tasks
Keep the tasks fixed so you can see whether the changes worked, and repeat after major releases.

What should you fix first for AI agents?

Fix the paths that end in a lead: the route from the homepage to pricing and from any page to the demo or signup form. Those are the tasks an agent is most likely to be sent to do, and the ones where a stall costs you a buyer. The facts on pricing and features matter too; how AI agents change product discovery covers which facts agents look for.

Pros

The same fixes improve accessibility, SEO and conversion for human visitors
Native elements are less code to maintain than custom widgets
Clear forms and confirmations reduce failed submissions from everyone

Cons

Replacing custom menus and pickers can mean rebuilding components
Removing pop-ups and puzzles may need a new approach to email capture and spam
Agent behavior changes fast, so testing has to be repeated

If your site is built from custom components that fight native HTML, a rebuild may be cheaper than patching. We build marketing sites in Next.js with server-rendered content and accessible components, and our AEO website service covers the AI search side in the same project.

Frequently asked questions

Do I need a special file or format for AI agents?: No. There's no required agent file. Files like llms.txt are aimed at language models reading content, and their benefit for AI visibility is unproven. Agents operating a page rely on the page itself: its HTML, its labels and its forms. Clean, accessible markup does more for agents than any extra file you add to the root of your site.
Can AI agents see content loaded by JavaScript?: Agents inside a browser usually run JavaScript, so they can see content that crawlers miss. That doesn't make script-heavy pages safe. Content that loads late, sits behind interactions or renders into canvas is still harder to find and read. Server-rendered HTML works for crawlers, agents and slow connections alike, so it's the safer default for key facts.
Will AI agents fill in my lead form with fake data?: An agent fills a form with the details its user gives it, so a well-built form should receive real leads, sometimes faster. Some submissions may carry less context than a human would type. Ask for the few fields you truly need, use clear labels and consider an optional notes field so agent-submitted requests still give your sales team something to work with.
Can I tell if an AI agent visited my website?: Sometimes. Agents inside a user's browser can look like ordinary visits, and vendors identify their traffic in different ways. Referral data in analytics can show visits from AI products when a link is clicked, which is covered in our guide to tracking ChatGPT traffic in GA4. Check each vendor's current documentation for how its agent identifies itself.
Is an agent-friendly website the same as an accessible website?: Mostly, but not entirely. Accessible markup covers the biggest part: named controls, labeled forms, keyboard support and logical structure. Agent readiness adds a few extras, such as keeping facts in plain HTML, putting state in URLs, avoiding stacked pop-ups and making confirmations explicit. If you meet WCAG 2.2 AA, you've done most of the work.

Mehdi Hassan · CEO & Design Director

Copywriter, designer and developer, and founder of Designify Studio. Over five years, 60+ projects across 14+ industries, for startups that have raised $300M+.

Want this done for you? Website, $8,000.

A multi-page marketing site, written and designed by us, developed in Next.js and animated. $8,000, one-time, up to 6 pages. Book a free 30-minute call to see if it fits.

Schedule Call