Skip to content

fix(nginx): noindex preview hostnames at the edge #2569

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vladfrangu merged 2 commits into from
May 25, 2026
+6 −0
Merged

fix(nginx): noindex preview hostnames at the edge #2569

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
efe37c3
fix(nginx): add X-Robots-Tag noindex on preview hostnames
vladfrangu May 25, 2026
88136f2
Merge branch 'preview-robots-canonical' into preview-noindex-header
vladfrangu May 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions nginx.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -810,6 +810,12 @@ server {
resolver 172.20.0.10;
server_name ~^(?<subdomain>[^.]+)\.preview\.docs\.apify\.com$;

# Block search indexing on every response from preview hostnames, including the
# apify.github.io proxy_passes (SDK/Client/CLI) that have no preview bucket and
# would otherwise serve production content under the preview hostname.
# `always` keeps the header on 4xx/5xx so error pages aren't indexed either.
add_header X-Robots-Tag "noindex, nofollow, noarchive" always;

# add trailing slashes to the root of GH pages docs
rewrite ^/api/client/js$ /api/client/js/ redirect;
rewrite ^/api/client/python$ /api/client/python/ redirect;
Expand Down
Loading