Skip to content

How I Optimized This Site for Agent Engine Search Optimization

Updated Apr 29, 2026 · Published Apr 27, 2026/13 min read

A code-grounded walkthrough of how daviddacruz.dev was optimized for AI search and agent-style retrieval: content routing, schema by page type, internal links, robots policy, sitemap generation, llms.txt, and answer-first page structure.

This article used to lean too hard on naming the funnel around the work.

That is not the useful part.

The useful part is what was actually implemented on this site, how it works in code and content, and why those changes make the site easier for search engines, AI answer systems, and agent-style crawlers to interpret correctly.

The Goal Was to Make the Site Easier to Interpret

The core optimization goal was simple:

make the public site easier to understand without requiring a model to infer too much.

That meant reducing ambiguity in four places:

  • what each page is for
  • how pages relate to each other
  • what the site is claiming in machine-readable form
  • what support files exist for crawlers and answer systems

The site did not need a layer of gimmicks. It needed clearer structure.

That is why most of the work happened in content architecture, schema generation, route logic, and supporting crawl surfaces.

I Cleaned Up Page Roles Instead of Letting Pages Blur Together

One of the main problems on small expert sites is that page jobs start to blur. A page tries to define a topic, sell a service, answer objections, show proof, and act as a general article all at once.

That usually weakens retrieval.

On this site, the content system now distinguishes page jobs much more explicitly. The dynamic page route in app/pages/[...slug].vue loads content from the pages collection and then derives behavior from fields like:

  • pageType
  • schemaType
  • status
  • primaryKeyword
  • faq
  • relatedLinks
  • proofLinks

That matters because page meaning is not being inferred only from the final HTML. The site already knows, at render time, whether something is meant to behave more like:

  • a service page
  • a case study
  • a collection page
  • a general informational page

The result is that the site can stay consistent in how it presents the page to both people and machines.

This is a better implementation pattern than manually pasting ad hoc metadata into dozens of unrelated templates.

The Content System Now Turns Page Type Into Structured Output

The most important implementation detail is that page type is not just an editorial label. It drives output.

Inside app/pages/[...slug].vue, the route:

  1. resolves the public path
  2. maps it into the content collection path under /pages
  3. loads the content entry with queryCollection('pages')
  4. rejects unpublished content outside dev
  5. derives metadata, breadcrumb data, overview copy, and schema from the entry

That means the content model is doing real architectural work.

For example:

  • commercial pages default toward Service
  • case_study pages become Article
  • pillar, support_pillar, and case_study_hub pages become CollectionPage
  • pages with FAQ data automatically expose FAQPage

This matters for AI search because machine systems do better when the public structure is stable and repeated. If one service page behaves like a service, another behaves like a blog post, and a third behaves like an untyped landing page, the site becomes harder to classify.

On this site, that classification is more disciplined.

I Used Schema as an Output of Page Meaning, Not a Separate Layer

Schema is implemented here as a consequence of page meaning, not as a detached checklist.

The same page route generates:

  • WebPage
  • Service
  • CollectionPage
  • Article
  • BreadcrumbList
  • FAQPage

depending on what the content entry actually is.

That is the right direction for a site like this because it prevents two common failures:

  • adding schema that does not match the visible page
  • forgetting to update schema when page structure changes

The route uses defineWebPage() for the base page layer and then adds extra JSON-LD when the entry requires it. Service pages expose a Service object. Collection pages can expose an ItemList based on related links and proof links. Article pages get author, publisher, published date, and modified date.

There is also a global organization layer in app/app.vue, where the site defines the publisher identity for David Dacruz with canonical social references.

This is one of the bigger improvements on the site: the identity layer and the page-type layer now reinforce each other instead of living as disconnected snippets.

Why this matters:

  • the homepage and site shell establish who the publisher is
  • content entries establish what each page is
  • breadcrumbs establish where the page sits in the site
  • FAQ establishes question-answer structure where it exists

That stack makes the site easier to interpret as a coherent source rather than a loose pile of pages.

I Added Machine-Readable Support Files, but Kept Them Secondary

This site now exposes:

The important part is not that these files exist. The important part is that they support a site that is already structurally clearer.

The short llms.txt file is intentionally compact. It gives:

  • identity
  • primary offers
  • key pages
  • support files
  • canonical notes

That is enough for a retrieval system to get a fast summary of what the site is, what it offers, and where the higher-confidence pages live.

The file does not try to replace the site. It acts as an index card for the site.

That is the right use for it.

Too many discussions about AI-search optimization jump straight to llms.txt as if it were the main mechanism. It is not. It is a support layer.

The stronger layer is still the visible site structure plus the schema plus the crawlable public files.

I Made Crawl and Index Surfaces More Explicit

The robots implementation is one of the more concrete AI-search changes.

In server/routes/robots.txt.ts, the site:

  • allows standard crawling
  • blocks internal paths like /api/, /context/, and /.nuxt/
  • exposes llms.txt and llms-full.txt clearly to major AI crawler variants
  • declares the sitemap location explicitly

The AI-specific blocks include names like:

  • GPTBot
  • OAI-SearchBot
  • ChatGPT-User
  • PerplexityBot
  • ClaudeBot
  • Claude-SearchBot
  • Google-Extended

This is the actual pattern from the file:

User-agent: GPTBot
Allow: /
Allow: /llms.txt

User-agent: OAI-SearchBot
Allow: /
Allow: /llms.txt

User-agent: ChatGPT-User
Allow: /
Allow: /llms.txt

User-agent: PerplexityBot
Allow: /
Allow: /llms.txt

User-agent: ClaudeBot
Allow: /
Allow: /llms.txt

User-agent: Claude-SearchBot
Allow: /
Allow: /llms.txt

User-agent: Google-Extended
Allow: /
Allow: /llms.txt

I do not treat this as magical. A robots file will not make a weak page strong.

But it does remove unnecessary uncertainty. If the site is willing to expose useful public material, the policy should say so cleanly.

The sitemap side is also more deliberate than a static dump.

The generator in server/routes/sitemap.xml.ts tracks:

  • static pages
  • route priorities
  • change frequencies
  • source file paths
  • image metadata

and resolves lastmod from file modification times where possible.

That matters because the sitemap is not just a list. It is a maintained representation of what the site considers public and important.

There is also a separate sitemap-organization layer in app/utils/site-sitemap.ts, which groups important public destinations by section: primary links, services, industries, case studies, and resource pages.

That grouping helps reinforce the site structure in a human-readable way too.

Another major change was internal linking discipline.

This site already had useful content. The problem was not only missing material. The problem was that related pages needed clearer relationships.

The current structure does this more deliberately:

  • services connect to adjacent explanatory pages
  • case studies connect back to the broader service or topic pages
  • support content links into the stronger implementation pages
  • topic and section pages help group related material

You can see that pattern in places like:

The internal links are there to do real routing work, not to decorate the footer or inflate link counts.

Why this helps AI search:

  • a model can more easily see which page is broader
  • it can see which pages support or explain the main page
  • it can move from proof to explanation and back again
  • it gets more repeated confirmation of topic ownership

This is especially important on service-heavy or expert sites where a single page is rarely enough context on its own.

I Kept the HTML Answer-First

One of the simpler but more important improvements was content shape.

Pages on this site now lean more heavily toward answer-first intros and earlier resolution of the main question.

That sounds editorial, but it has technical consequences too.

In the page route, the rendered content is wrapped in a predictable content shell and exposed through a standard structure:

  • clear title
  • direct intro
  • overview section
  • article body
  • optional FAQ
  • optional related links and proof links

This does two useful things:

  1. it makes the visible page easier to skim and cite
  2. it reduces the amount of meaningless throat-clearing before the answer begins

That is one reason I have been removing long first-section explanation blocks elsewhere on the site. They waste the most valuable part of the page.

If the top of the page starts by describing the publishing strategy instead of the subject itself, both users and machines have to work harder than they should.

Why This Works Better Than Publishing More Generic AI-Search Posts

A lot of AI-search content on the web stays generic.

It says:

  • be quotable
  • add schema
  • publish FAQs
  • make pages clear

That advice is not wrong. It is just too abstract to be useful without implementation detail.

What was implemented on this site is more concrete:

  • content entries now carry explicit page-type information
  • the route logic turns that into stable schema and metadata
  • publisher identity is defined globally
  • robots policy exposes public support files intentionally
  • the sitemap is maintained as an actual public surface
  • llms.txt and llms-full.txt summarize the site for machine readers
  • internal links are organized around real topical relationships
  • the page copy begins earlier with the answer

None of these changes alone is the trick.

Together, they reduce ambiguity.

That is the main point.

If you want a shorter summary of how this site was optimized for agent engine search optimization, it is this:

I did not try to invent a special AI-search layer on top of a vague site.

I made the site itself clearer, then exposed that clarity through schema, crawl policy, support files, and internal links.

That is the part worth copying.

FAQ

Common questions.

What did you actually change on daviddacruz.dev for AI search?

The main changes were cleaner page jobs, schema generated from page type, stronger internal linking between related pages, explicit robots and sitemap support for AI crawlers, llms.txt and llms-full.txt, and more direct answer-first content structure.

Did you optimize this site mainly with llms.txt?

No. llms.txt was added as a support file, but the bigger work was structural: making pages easier to classify, routing related pages together, and generating cleaner machine-readable signals from the content model.

How is schema implemented on this site?

The site derives schema from page metadata and template logic. Service pages, collection pages, articles, breadcrumbs, and FAQ blocks are generated according to page type rather than being added manually one by one.

What matters most for AI search on a site like this?

Clear page ownership, explicit internal links, honest schema, crawlable support files, and direct answer-first copy matter more than publishing a large volume of generic AI-search content.

Is this article describing theory or the real site?

It is about the real implementation on daviddacruz.dev: the route structure, content collection logic, schema generation, sitemap behavior, robots policy, and public support files that are live on the site.

References

Authority sources.

Written by

David Dacruz

Digital architect in Ericeira, Portugal. 42 alumni. I write about building at the intersection of AI, web3, and what actually ships.