seo: rank first for "what is webjs" across site, repo, and Pages

[vivek7405]

Problem

Searching Google for "What is WebJs" does not surface this project first. The
current ranking is:

1. lubino.github.io/webJS/ (an unrelated client-side toolkit)
2. stephdz.github.io/WebJS/ (an unrelated Java framework, last published 2013-10-31, ~350 words)
3. moldstud.com/articles/p-what-is-webjs-and-how-is-it-used-by-developers
4. github.com/webjsdev/webjs (ours)
5. webjs.dev (ours, page 2)

The stephdz page wins on one thing only: an exact-match <title> ("WebJS - What is
WebJS ?") and <h1> ("What is WebJS?"). We have no page anywhere titled "What is
WebJs". webjs.dev has /why, /compare, and /articles, but nothing targeting
the flat definitional query.

Investigating that turned up four further live-site SEO defects, all verified against
production:

1. No favicon in Google results. app/layout.ts declares
<link rel="icon" href="/public/favicon.png" type="image/png" sizes="32x32">, but
website/public/favicon.png is actually 512x512. Google requires a favicon that is
square and a multiple of 48px (48x48, 96x96, 144x144). A favicon declared 32x32 is
below that floor and gets skipped. There is also no /favicon.ico at the origin root
(curl https://webjs.dev/favicon.ico returns 404), which is the universal fallback
Google looks for.

2. No <link rel="canonical"> on any page. Verified against /, /why, and
/articles/web-components-framework: zero canonical tags site-wide. The framework
supports it (packages/server/src/ssr.js L1719, via metadata.alternates.canonical),
but no website page sets it.

3. Cloudflare's Managed robots.txt is blocking every AI answer engine. The live
https://webjs.dev/robots.txt is not what app/robots.ts emits. Cloudflare prepends a
managed block that sets Content-Signal: search=yes,ai-train=no,use=reference and
Disallow: / for ClaudeBot, GPTBot, CCBot, Bytespider, Amazonbot,
Applebot-Extended, Google-Extended, meta-externalagent, and
CloudflareBrowserRenderingCrawler. For a framework whose entire positioning is
"AI-first", being unciteable by ChatGPT, Claude, and Perplexity is a direct own-goal on
exactly the surface where developers now ask "what is X". It also emits two
User-agent: * groups (Cloudflare's, then ours), so our own group is likely ignored by
crawlers that take the first match. Google Search indexing itself is unaffected
(Google-Extended governs Gemini training only, not Googlebot).

4. Stale title in the SERP. Google shows webjs: AI-first, web-components-first, no-build web framework for webjs.dev. Production currently serves
WebJs - The Web Framework for AI Agents (verified with a Googlebot user-agent), so this
is pure recrawl lag, not a live bug. Needs a Search Console reindex request, and the
canonical tags in (2) will help it settle.

Design / approach

Three surfaces get an answer to the query, because we do not care which of ours wins as
long as a WebJs-framework result is first.

A. webjs.dev/what-is-webjs (primary, highest long-term value). A bespoke page with
an exact-match URL slug, <title>, and <h1>. Definition front-loaded into the first
160 characters so the snippet stands alone. Carries SoftwareApplication +
BreadcrumbList + FAQPage JSON-LD, with the visible FAQ and the schema built from one
array so they cannot drift. Includes a deliberate disambiguation section naming
whatsapp-web.js, the 2013 Java WebJS, and the webJS toolkit. That section is the honest
answer to a real reader question and is the thing that makes this a genuinely better
result than the page it is trying to outrank.

B. GitHub repo (cheapest win, already ranks ~3rd). Google renders the repo SERP title
as GitHub - webjsdev/webjs: <description>. Make the description definitional so the
title itself answers the query, and add an early ## What is WebJs? section to the README
so the snippet body does too.

C. GitHub Pages at webjsdev.github.io/webjs/. A third SERP slot. Source of truth is
a single static index.html committed on main, published to an orphan gh-pages
branch by an Action, so there is no second surface to drift.

Honest caveat, recorded so nobody re-litigates it: github.io is on the Public Suffix
List, so webjsdev.github.io inherits zero authority from github.com. It starts
from nothing. GitHub's authority is why github.com/webjsdev/webjs ranks 5th; a new Pages
site gets none of it. C is therefore the weakest of the three levers despite being the
most intuitive. Ordering of effort should be A, then B, then C.

Implementation notes (for the implementing agent)

Where to edit:

• New page: website/app/what-is-webjs/page.ts. Reuse the shared class strings
  (KICKER / BTN / INSTALL / WIN / WINBAR / CARD) verbatim from
  website/app/why/page.ts L46-L53 so the three marketing pages stay one design system.
  Build the FAQ schema with faqJsonLd() from website/lib/faq.ts, and the code windows
  with highlight() from website/lib/highlight.ts plus the samples in
  website/lib/samples.ts.
• Sitemap: website/app/sitemap.ts L23, add /what-is-webjs to staticRoutes and
  give it a priority above the default 0.7 (it is a primary page).
• Internal links: website/lib/site-footer.ts (Resources column, ~L30) and the header
  nav in website/app/layout.ts L28-L37. The nav already carries 8 items, so weigh
  crowding before adding there. Internal links are what tell Google the page is primary.
• Favicon: website/app/layout.ts L71-L73. Correct the declared sizes to match the
  real 512x512 asset (use sizes="512x512", or emit a 192x192 raster). Order the PNG
  ahead of the SVG for crawlers that take the first usable icon. Add a root-served
  favicon.ico: the framework serves public/favicon.ico at /favicon.ico (same
  root-serving rule as sw.js / offline.html, see AGENTS.md "App layout").
• Canonical tags: add alternates: { canonical } to generateMetadata in
  website/app/why/page.ts, website/app/articles/[slug]/page.ts,
  website/app/compare/[slug]/page.ts, website/app/blog/[slug]/page.ts, and the root
  website/app/layout.ts. The framework already renders it
  (packages/server/src/ssr.js L1719-L1720).
• robots.txt: website/app/robots.ts. Explicitly Allow: / the AI crawlers we want
  (ClaudeBot, GPTBot, PerplexityBot, OAI-SearchBot, Applebot-Extended, CCBot). This
  alone will not fix it, because Cloudflare's managed block is injected at the edge and
  wins. The Cloudflare dashboard setting must also be turned off, under
  Security > Settings > "Manage robots.txt" / AI Scrapers and Crawlers, for the
  webjs.dev zone. Flag this to the user as a dashboard action, and note it applies to
  docs.webjs.dev and ui.webjs.dev too (verified: same managed block).
• README: add a ## What is WebJs? section immediately after the H1, before
  ## Why WebJs. Keep the first sentence a clean definition.
• Repo description: gh repo edit webjsdev/webjs --description "...".
• GitHub Pages: seo/github-pages/index.html plus .github/workflows/pages.yml
  publishing to an orphan gh-pages branch. Pages is currently not enabled on the
  repo (gh api repos/webjsdev/webjs/pages returns 404), so it needs enabling once.

Landmines / gotchas:

• The prose-punctuation hook (.claude/hooks/block-prose-punctuation.sh) blocks
  em-dashes, space-hyphen and space-semicolon pauses, and enforces WebJs capitalization
  in prose. It scans new content only. It also fires on shell command separators in
  Bash tool calls that look like prose, so use newlines rather than ; between commands.
• Invariant 9: no backticks inside html template bodies. Code samples must come
  through highlight() on a plain string constant, never inline.
• The website page must not import anything client-effecting, or the page module ships to
  the browser. highlight() already ships as a small dead module on the home page (see
  the comment at website/app/page.ts L5-L16) and that cost is accepted, so reusing the
  same pattern is fine.
• webjs.dev is behind Cloudflare in front of Railway (server: cloudflare,
  x-railway-edge: cdg1). Cloudflare bot rules 403 some non-browser user agents, which is
  why a plain fetch of the site can fail while Googlebot gets a 200. Test crawler-facing
  behaviour with an explicit -A user-agent, not a default curl.
• The root layout sets cacheControl: public, s-maxage=600, so an edge-cached response
  can mask a change for up to 10 minutes after deploy.
• After merge, confirm Railway actually redeployed the website service. A merge does not
  redeploy the live site by itself.

Invariants to respect: AGENTS.md invariants 8 (only the root layout writes the shell),
9 (no backticks in html bodies), 11 (prose punctuation and WebJs casing).

Tests + docs surfaces:

• website/test/ssr/seo-ssr.test.ts is the home for the JSON-LD and metadata assertions.
  Follow its existing shape: assert JSON-LD at the generateMetadata level, not by
  scraping HTML.
• Add a website/test/ssr/what-is-webjs-ssr.test.ts mirroring
  website/test/ssr/why-ssr.test.ts for the render smoke test plus title/og/twitter
  self-consistency.
• Include a counterfactual: the FAQ JSON-LD question set must match the visible FAQ array,
  so a test that fails if someone edits one without the other.
• Docs surfaces: this is website-only content, so no docs-site or AGENTS.md change is
  needed, but website/app/llms.txt/route.ts should list the new page.

Acceptance criteria

• webjs.dev/what-is-webjs returns 200 with <title>What is WebJs?</title> and a matching <h1>
• The page emits SoftwareApplication + BreadcrumbList + FAQPage JSON-LD, and the FAQPage entries match the visible FAQ exactly
• The page is in /sitemap.xml, in /llms.txt, and linked from the site footer
• <link rel="canonical"> is present on the home page, /why, /what-is-webjs, and every article, compare, and blog page
• The favicon sizes attribute matches the real asset dimensions, and https://webjs.dev/favicon.ico returns 200
• app/robots.ts explicitly allows ClaudeBot, GPTBot, PerplexityBot, OAI-SearchBot, and CCBot
• Cloudflare's managed robots.txt block is disabled for the zone, verified by curl https://webjs.dev/robots.txt no longer containing Content-Signal or ClaudeBot
• README opens with a ## What is WebJs? definition section
• The repo description is definitional
• webjsdev.github.io/webjs/ serves a static "What is WebJs?" answer page
• A counterfactual proves the FAQ-schema-matches-visible-FAQ test actually fires
• Tests cover the new page at the SSR and metadata layers
• A Search Console reindex is requested for webjs.dev and webjs.dev/what-is-webjs (user action, documented in the PR)