fix(website): improve crawlability and indexing - #467
Merged
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
github-actions
Bot
added
bug
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughCloudFront routing now preserves query strings, normalizes extensionless URLs, and serves 404 responses through ChangesCloudFront routing
Website SEO and localization
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant CloudFront
participant URLRewriteHandler
participant Viewer
CloudFront->>URLRewriteHandler: Forward URI and querystring
URLRewriteHandler->>URLRewriteHandler: Normalize path and serialize querystring
URLRewriteHandler-->>Viewer: Return 301 Location with normalized URL
Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
There was a problem hiding this comment.
Pull request overview
Improves Envilder website crawlability and indexing by making CloudFront serve real 404 responses (instead of index fallback), consolidating extensionless URLs to trailing-slash canonical URLs via a CloudFront Function redirect, and adding crawler discovery + SEO metadata (canonical/hreflang, localized descriptions, structured data, and a noindex 404 page) to the Astro site.
Changes:
- Update CloudFront distribution behavior: map 403/404 to a real
/404.htmlwith HTTP 404, and add viewer-request redirects for extensionless paths → trailing slash (plus legacy/sitemap.xmlredirect). - Add SEO/crawler signals to the website: robots.txt, llms.txt, canonical + hreflang links, localized meta descriptions, and JSON-LD structured data.
- Extend and adjust IaC tests/snapshots to cover redirects, query preservation, and 404 error responses.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/iac/lib/stacks/staticWebsiteStack.test.ts | Adds assertion coverage for CloudFront custom error responses returning real 404s. |
| tests/iac/lib/stacks/cloudfrontUrlRewrite.test.ts | Updates tests for redirect-based trailing-slash canonicalization and query preservation. |
| tests/iac/lib/stacks/snapshots/staticWebsiteStack.test.ts.snap | Updates synthesized stack snapshot for new CloudFront error/Function behavior. |
| src/iac/lib/stacks/staticWebsiteStack.ts | Switches CloudFront custom error responses to return /404.html with HTTP 404. |
| src/iac/lib/stacks/cloudfront-url-rewrite.js | Rewrites URL handling to redirect extensionless paths to trailing slashes and preserves query strings in redirects. |
| src/website/src/layouts/BaseLayout.astro | Adds canonical/hreflang, optional noindex, and JSON-LD structured data. |
| src/website/src/pages/404.astro | Introduces a dedicated 404 page rendered with noindex. |
| src/website/src/pages/docs.astro | Adds localized meta description support via BaseLayout prop. |
| src/website/src/pages/changelog.astro | Adds localized meta description support via BaseLayout prop. |
| src/website/src/pages/ca/docs.astro | Adds localized meta description support (Catalan). |
| src/website/src/pages/ca/changelog.astro | Adds localized meta description support (Catalan). |
| src/website/src/pages/es/docs.astro | Adds localized meta description support (Spanish). |
| src/website/src/pages/es/changelog.astro | Adds localized meta description support (Spanish). |
| src/website/src/i18n/types.ts | Extends translation types for page descriptions and 404 strings. |
| src/website/src/i18n/en.ts | Adds new SEO descriptions + 404 translations (English). |
| src/website/src/i18n/ca.ts | Adds new SEO descriptions + 404 translations (Catalan). |
| src/website/src/i18n/es.ts | Adds new SEO descriptions + 404 translations (Spanish). |
| src/website/public/robots.txt | Publishes robots.txt + sitemap discovery signal. |
| src/website/public/llms.txt | Publishes an LLM-friendly project summary and key links. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/iac/lib/stacks/cloudfront-url-rewrite.js`:
- Around line 47-78: Update getQueryString to percent-encode each query
parameter key and value with encodeURIComponent before joining them into the
rebuilt query string, while preserving key-only parameters and the existing
string-query handling.
In `@src/website/src/pages/404.astro`:
- Around line 7-8: Update the 404 page’s fixed `lang` selection so localized
missing routes derive their locale from the requested path or routing context,
falling back to English only when no supported locale is present. Use that
locale for `useTranslations` and localize the home link destination accordingly,
preserving the existing 404 layout.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 0cb87f12-721a-4686-bc50-14bff42e6b60
⛔ Files ignored due to path filters (1)
tests/iac/lib/stacks/__snapshots__/staticWebsiteStack.test.ts.snapis excluded by!**/*.snapand included bytests/**
📒 Files selected for processing (18)
src/iac/lib/stacks/cloudfront-url-rewrite.jssrc/iac/lib/stacks/staticWebsiteStack.tssrc/website/public/llms.txtsrc/website/public/robots.txtsrc/website/src/i18n/ca.tssrc/website/src/i18n/en.tssrc/website/src/i18n/es.tssrc/website/src/i18n/types.tssrc/website/src/layouts/BaseLayout.astrosrc/website/src/pages/404.astrosrc/website/src/pages/ca/changelog.astrosrc/website/src/pages/ca/docs.astrosrc/website/src/pages/changelog.astrosrc/website/src/pages/docs.astrosrc/website/src/pages/es/changelog.astrosrc/website/src/pages/es/docs.astrotests/iac/lib/stacks/cloudfrontUrlRewrite.test.tstests/iac/lib/stacks/staticWebsiteStack.test.ts
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/website/src/pages/404.astro`:
- Around line 7-8: Update the 404 page’s locale resolution around lang,
useTranslations, Navbar, Footer, and BaseLayout so it derives the requested
locale from the URL when visiting localized routes such as /es/404 or /ca/404
instead of always using defaultLang. Ensure the shared chrome and locale links
use that resolved locale while preserving the existing default-language
fallback.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 6e81fa5d-77eb-48a1-bc62-e6c1d1d08151
⛔ Files ignored due to path filters (1)
tests/iac/lib/stacks/__snapshots__/staticWebsiteStack.test.ts.snapis excluded by!**/*.snapand included bytests/**
📒 Files selected for processing (4)
src/iac/lib/stacks/cloudfront-url-rewrite.jssrc/website/src/layouts/BaseLayout.astrosrc/website/src/pages/404.astrotests/iac/lib/stacks/cloudfrontUrlRewrite.test.ts
🚧 Files skipped from review as they are similar to previous changes (3)
- src/website/src/layouts/BaseLayout.astro
- src/iac/lib/stacks/cloudfront-url-rewrite.js
- tests/iac/lib/stacks/cloudfrontUrlRewrite.test.ts
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Align the LLM reference with the llms.txt proposal. Route models to canonical docs, packages, examples, and current product boundaries. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Separate shipped interfaces and include a minimal map-file example. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Skip invalid query-object entries so redirects remain available. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Capture the intentional malformed-query guard in the synthesized Function. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
src/iac/lib/stacks/cloudfront-url-rewrite.js:53
getQueryStringwill throw a TypeError ifreq.querystringisnull/undefined(becausefor...incannot iterate nullish). Since this edge function already treats the querystring as potentially non-standard (string vs object + skipping malformed entries), add a null/type guard so redirects never fail at the edge.
function getQueryString(querystring) {
if (typeof querystring === "string") {
return querystring ? "?" + querystring : "";
}
var parts = [];
for (var key in querystring) {
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Improve Envilder indexability by returning real 404 responses, consolidating trailing-slash URLs, and publishing crawler discovery signals.
Harden CloudFront redirect query serialization and make the shared static 404 response locale-aware without changing its HTTP 404 status.
Changes
Testing
Related
N/A
Summary by CodeRabbit
/404.htmlwith a 404 status.llms.txtandrobots.txt(sitemap index reference).