feat: add sitemap and robots generation#443
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
WalkthroughThe changes introduce new modules for robots.txt and sitemap generation in the web application, add a constant for the site URL, and update the Next.js package version. The sitemap function dynamically includes both static and blog post routes, while robots.txt rules are explicitly defined for web crawlers. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant App (Next.js)
participant BlogData (getPosts)
participant Constants (SITE_URL)
Client->>App (robots.ts): Request /robots.txt
App-->>Client: Respond with robots.txt rules (using SITE_URL)
Client->>App (sitemap.ts): Request /sitemap.xml
App->>BlogData: getPosts()
BlogData-->>App: Blog posts data
App->>Constants: Retrieve SITE_URL
App-->>Client: Respond with sitemap including static and blog post URLs
Estimated code review effort2 (~15 minutes) Poem
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
✅ Deploy Preview for appcut ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
apps/web/src/constants/site.ts (1)
1-1: Consider using environment variables for deployment flexibility.While the hardcoded URL works for production, consider using an environment variable to support different deployment environments (development, staging, production). This would improve maintainability and deployment flexibility.
-export const SITE_URL = "https://opencut.app"; +export const SITE_URL = process.env.NEXT_PUBLIC_SITE_URL || "https://opencut.app";apps/web/src/app/robots.ts (1)
4-13: LGTM! Consider additional security paths.The robots.txt implementation is well-structured and follows Next.js best practices. The disallowed paths appropriately block Next.js internals and application-specific routes.
Consider whether additional paths should be blocked for security:
/api/routes (if they contain sensitive endpoints)/admin/or similar administrative paths- Any other private application routes
apps/web/src/app/sitemap.ts (1)
16-61: Review sitemap priorities and change frequencies.The static routes configuration looks comprehensive, but some priorities and change frequencies could be optimized:
- Contributors page (line 26):
changeFrequency: "daily"seems too frequent unless contributors are added daily- Roadmap page (line 33):
priority: 1matches homepage priority - consider if this is intentional- Why-not-capcut page (line 51):
priority: 1seems high for what appears to be a comparison pageConsider adjusting:
{ url: `${SITE_URL}/contributors`, lastModified: new Date(), - changeFrequency: "daily", + changeFrequency: "monthly", priority: 0.5, }, { url: `${SITE_URL}/roadmap`, lastModified: new Date(), changeFrequency: "weekly", - priority: 1, + priority: 0.8, }, { url: `${SITE_URL}/why-not-capcut`, lastModified: new Date(), changeFrequency: "yearly", - priority: 1, + priority: 0.7, },
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
bun.lockis excluded by!**/*.lock
📒 Files selected for processing (4)
apps/web/package.json(1 hunks)apps/web/src/app/robots.ts(1 hunks)apps/web/src/app/sitemap.ts(1 hunks)apps/web/src/constants/site.ts(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Redirect rules - appcut
- GitHub Check: Header rules - appcut
- GitHub Check: Pages changed - appcut
- GitHub Check: build (windows-latest)
🔇 Additional comments (2)
apps/web/package.json (1)
45-45: Next.js ^15.4.3 Verified and Ready
- Confirmed version 15.4.3 exists on npm and is the current latest release.
- No security advisories or breaking changes reported for this patch.
- Includes the metadata route type enhancements as expected.
No further action required.
apps/web/src/app/sitemap.ts (1)
8-14: We’ve confirmed thatgetPosts()is implemented inapps/web/src/lib/blog-query.tsas an async function callingfetchFromMarble, which already throws on non-OK responses. No additional error handling is needed here.No changes required.
| export default async function sitemap(): Promise<MetadataRoute.Sitemap> { | ||
| const data = await getPosts(); | ||
|
|
||
| const postPages: MetadataRoute.Sitemap = | ||
| data?.posts?.map((post) => ({ | ||
| url: `${SITE_URL}/blog/${post.slug}`, | ||
| lastModified: new Date(post.publishedAt), | ||
| changeFrequency: "weekly", | ||
| priority: 0.8, | ||
| })) ?? []; |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Add error handling for blog posts fetching.
The sitemap generation looks well-implemented, but consider adding error handling for the getPosts() call to ensure the sitemap still generates even if blog posts can't be fetched.
export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
- const data = await getPosts();
+ let data;
+ try {
+ data = await getPosts();
+ } catch (error) {
+ console.error('Failed to fetch posts for sitemap:', error);
+ data = null;
+ }
const postPages: MetadataRoute.Sitemap =
data?.posts?.map((post) => ({📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| export default async function sitemap(): Promise<MetadataRoute.Sitemap> { | |
| const data = await getPosts(); | |
| const postPages: MetadataRoute.Sitemap = | |
| data?.posts?.map((post) => ({ | |
| url: `${SITE_URL}/blog/${post.slug}`, | |
| lastModified: new Date(post.publishedAt), | |
| changeFrequency: "weekly", | |
| priority: 0.8, | |
| })) ?? []; | |
| export default async function sitemap(): Promise<MetadataRoute.Sitemap> { | |
| - const data = await getPosts(); | |
| + let data; | |
| + try { | |
| + data = await getPosts(); | |
| + } catch (error) { | |
| + console.error('Failed to fetch posts for sitemap:', error); | |
| + data = null; | |
| + } | |
| const postPages: MetadataRoute.Sitemap = | |
| data?.posts?.map((post) => ({ | |
| url: `${SITE_URL}/blog/${post.slug}`, | |
| lastModified: new Date(post.publishedAt), | |
| changeFrequency: "weekly", | |
| priority: 0.8, | |
| })) ?? []; |
🤖 Prompt for AI Agents
In apps/web/src/app/sitemap.ts around lines 5 to 14, the getPosts() call lacks
error handling, which could cause the sitemap generation to fail if fetching
posts errors out. Wrap the getPosts() call in a try-catch block, and in the
catch block, handle the error gracefully by logging it or returning an empty
array for posts so the sitemap can still generate without blog post URLs.
feat: add sitemap and robots generation
2 participants
Description
Adds the sitemap and robots generation to the project for google/seo reasons. I also quickly bumped nextjs.
Please delete options that are not relevant.
How Has This Been Tested?
Locally by navigating to the routes.
Checklist:
Summary by CodeRabbit