-
Notifications
You must be signed in to change notification settings - Fork 24
docs: Add uv guide #932
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
vdusek
wants to merge
6
commits into
master
Choose a base branch
from
docs/uv-guide
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+249
−0
Open
docs: Add uv guide #932
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
0426a58
docs: Add uv guide
vdusek 7efcf3e
docs: Clarify Dockerfile forward reference and simplify run command i…
vdusek 4421d77
docs: Retitle uv guide to describe what it is for
vdusek d5ab731
docs: Note in the uv guide that Actor templates don't support uv yet
vdusek 309b4fc
docs: Address review feedback on the uv guide
vdusek 560cc43
docs: renumber uv guide to 10
vdusek File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,188 @@ | ||
| --- | ||
| id: uv | ||
| title: Project management with uv | ||
| description: Manage your Actor's Python version, dependencies, and virtual environment with the uv package and project manager. | ||
| --- | ||
|
|
||
| import CodeBlock from '@theme/CodeBlock'; | ||
| import Tabs from '@theme/Tabs'; | ||
| import TabItem from '@theme/TabItem'; | ||
|
|
||
| import PyprojectExample from '!!raw-loader!./code/uv_project/pyproject.toml'; | ||
| import MainExample from '!!raw-loader!./code/uv_project/my_actor/main.py'; | ||
| import UnderscoreMainExample from '!!raw-loader!./code/uv_project/my_actor/__main__.py'; | ||
| import DockerfileExample from '!!raw-loader!./code/uv_project/Dockerfile'; | ||
|
|
||
| In this guide, you'll learn how to use [uv](https://docs.astral.sh/uv/) to manage your Apify Actor projects - from creating a new project, through running it locally, to building and deploying it on the Apify platform. | ||
|
|
||
| ## Introduction | ||
|
|
||
| [uv](https://docs.astral.sh/uv/) is an extremely fast Python package and project manager. It replaces the combination of pip, virtualenv, and similar tools with a single binary that manages your project's Python version, virtual environment, and dependencies. It records the project metadata in the standard [`pyproject.toml`](https://packaging.python.org/en/latest/guides/writing-pyproject-toml/) file and the exact resolved versions of all dependencies in a [`uv.lock`](https://docs.astral.sh/uv/concepts/projects/sync/) lockfile. | ||
|
|
||
| The [Python Actor templates](https://apify.com/templates/categories/python) declare their dependencies in a `requirements.txt` file, which is the default approach for Actors. Using uv instead brings a few advantages: | ||
|
|
||
| - The lockfile guarantees that the dependencies installed in the Actor's Docker image are exactly the ones you developed and tested against locally. | ||
| - Dependency installation during the Docker build is significantly faster than with pip, especially with a warm cache. | ||
| - During local development, a single tool manages your Python version, virtual environment, and dependencies, so the project behaves the same on every developer's machine. | ||
|
|
||
| :::info Actor templates don't support uv yet | ||
|
|
||
| The [Apify Actor templates](https://apify.com/templates) currently support only pip with `requirements.txt`. Adding uv-based templates is planned - follow [apify/actor-templates#350](https://github.com/apify/actor-templates/issues/350) for updates. | ||
|
|
||
| ::: | ||
|
|
||
| To follow along, install [uv](https://docs.astral.sh/uv/getting-started/installation/) and the [Apify CLI](https://docs.apify.com/cli/docs/installation) first. | ||
|
|
||
| ## Create a new project | ||
|
|
||
| Create a new uv project and add the Apify SDK to its dependencies: | ||
|
|
||
| ```bash | ||
| uv init my-actor --bare | ||
| cd my-actor | ||
| uv python pin 3.14 | ||
| uv add apify | ||
| ``` | ||
|
|
||
| The [`uv init`](https://docs.astral.sh/uv/reference/cli/#uv-init) command with the `--bare` option creates just the `pyproject.toml` project manifest. The `uv python pin` command writes the project's Python version to the `.python-version` file - uv automatically downloads that Python version if it's not installed on your machine. Finally, [`uv add`](https://docs.astral.sh/uv/reference/cli/#uv-add) records the dependency in `pyproject.toml`, resolves the exact versions of the whole dependency tree into `uv.lock`, and installs everything into the project's virtual environment in `.venv`. | ||
|
|
||
| The `uv add` command constrains the dependency to the latest version it resolved. You can edit the constraint as you see fit - this guide's example Actor allows any version of the SDK within the current major one: | ||
|
|
||
| <CodeBlock className="language-toml" title="pyproject.toml"> | ||
| {PyprojectExample} | ||
| </CodeBlock> | ||
|
|
||
| Note that the example has no `[build-system]` section. Without one, uv treats the project as a non-package ("virtual") project: it doesn't try to build and install the project itself, it only manages its dependencies. That's exactly what we want here - the Actor runs as a module straight from the source tree. | ||
|
|
||
| ## Add the Actor scaffolding | ||
|
|
||
| For the project to be runnable as an Actor, it needs two more pieces: the source code as a runnable Python package, and the `.actor/` directory with the [Actor configuration](https://docs.apify.com/platform/actors/development/actor-definition/actor-json). | ||
|
|
||
| Create a `my_actor` package with the Actor's source code: | ||
|
|
||
| <Tabs> | ||
| <TabItem value="my_actor/main.py" label="my_actor/main.py" default> | ||
| <CodeBlock className="language-python"> | ||
| {MainExample} | ||
| </CodeBlock> | ||
| </TabItem> | ||
| <TabItem value="my_actor/__main__.py" label="my_actor/__main__.py"> | ||
| <CodeBlock className="language-python"> | ||
| {UnderscoreMainExample} | ||
| </CodeBlock> | ||
| </TabItem> | ||
| </Tabs> | ||
|
|
||
| Don't forget to add an empty `my_actor/__init__.py` file, so that the directory is a regular Python package executable with `python -m my_actor`. | ||
|
|
||
| Then add the Actor definition to `.actor/actor.json`: | ||
|
|
||
| ```json title=".actor/actor.json" | ||
| { | ||
| "$schema": "https://apify.com/schemas/v1/actor.ide.json", | ||
| "actorSpecification": 1, | ||
| "name": "my-actor", | ||
| "title": "My uv Actor", | ||
| "description": "An Apify Actor with dependencies managed by uv.", | ||
| "version": "0.1", | ||
| "buildTag": "latest", | ||
| "dockerfile": "../Dockerfile" | ||
| } | ||
| ``` | ||
|
|
||
| The `dockerfile` field points to the project's `Dockerfile`, which doesn't exist yet - you'll create it in the [Use uv in the Dockerfile](#use-uv-in-the-dockerfile) section below. | ||
|
|
||
| The final project structure looks like this: | ||
|
|
||
| ```text | ||
| my-actor/ | ||
| ├── .actor/ | ||
| │ └── actor.json | ||
| ├── my_actor/ | ||
| │ ├── __init__.py | ||
| │ ├── __main__.py | ||
| │ └── main.py | ||
| ├── .python-version | ||
| ├── Dockerfile | ||
| ├── pyproject.toml | ||
| └── uv.lock | ||
| ``` | ||
|
|
||
| Make sure to commit `uv.lock` and `.python-version` to version control, so that every developer's machine works with identical dependencies and the same Python version. The Actor's Docker build gets its Python interpreter from the base image instead, so keep the base image tag (`apify/actor-python:3.14`) in sync with `.python-version`. | ||
|
|
||
| ## Run the Actor locally | ||
|
|
||
| If you've just cloned the project (or skipped `uv add` above), install the dependencies first: | ||
|
|
||
| ```bash | ||
| uv sync | ||
| ``` | ||
|
|
||
| The [`uv sync`](https://docs.astral.sh/uv/reference/cli/#uv-sync) command creates the `.venv` virtual environment (if it doesn't exist yet) and installs the locked dependencies into it. Then run the Actor with the Apify CLI: | ||
|
|
||
| ```bash | ||
| apify run | ||
| ``` | ||
|
|
||
| The [`apify run`](https://docs.apify.com/cli/docs/reference#apify-run) command automatically detects the virtual environment in `.venv` and uses it to run the Actor as a module (`python -m my_actor`), with the environment set up to emulate the Apify platform locally - for example, the Actor input is read from `storage/key_value_stores/default/INPUT.json`. | ||
|
|
||
| ## Use uv in the Dockerfile | ||
|
|
||
| On the Apify platform, the Actor runs as a Docker container built from the Dockerfile referenced in `.actor/actor.json`. The following Dockerfile installs the locked dependencies with uv on top of the [Apify Python base image](https://hub.docker.com/r/apify/actor-python): | ||
|
|
||
| <CodeBlock className="language-docker" title="Dockerfile"> | ||
| {DockerfileExample} | ||
| </CodeBlock> | ||
|
|
||
| A few details worth understanding: | ||
|
|
||
| - The uv binary is copied from its [official Docker image](https://docs.astral.sh/uv/guides/integration/docker/), pinned to a minor version line, so builds are reproducible and there is no need to install uv with pip. | ||
| - `uv sync --locked --no-dev` installs the dependencies exactly as recorded in `uv.lock` and skips development dependencies. If the lockfile is missing or out of sync with `pyproject.toml`, the build fails instead of silently resolving different versions. | ||
| - The dependencies are installed in a separate layer before the source code is copied, so editing your code doesn't invalidate the dependency layer, and rebuilds are fast. | ||
| - Putting `.venv/bin` first on `PATH` makes `python` resolve to the project's virtual environment, both during the build and when the Actor runs. | ||
|
|
||
| Also create a `.dockerignore` file and exclude at least `.venv`, `.git`, and `storage` from the Docker build context - the local virtual environment must never be copied into the image, since it's recreated by `uv sync` during the build. | ||
|
|
||
| ## Deploy to the Apify platform | ||
|
|
||
| Once the Actor works locally, log in and push it to the Apify platform: | ||
|
|
||
| ```bash | ||
| apify login | ||
| apify push | ||
| ``` | ||
|
|
||
| The [`apify push`](https://docs.apify.com/cli/docs/reference#apify-push) command uploads the project to the platform and builds the Docker image from the Dockerfile above. Thanks to the committed lockfile, the platform build installs exactly the dependency versions you ran locally. | ||
|
|
||
| ## Manage dependencies | ||
|
|
||
| Day-to-day dependency management goes through uv as well: | ||
|
|
||
| ```bash | ||
| # Add a dependency (records it in pyproject.toml and updates uv.lock). | ||
| uv add httpx | ||
|
|
||
| # Add a development-only dependency (skipped in the Docker build by --no-dev). | ||
| uv add --dev ruff | ||
|
|
||
| # Remove a dependency. | ||
| uv remove httpx | ||
|
|
||
| # Upgrade all dependencies to the latest versions allowed by pyproject.toml. | ||
| uv lock --upgrade | ||
| uv sync | ||
| ``` | ||
|
|
||
| Whenever the dependencies change, commit the updated `uv.lock` together with `pyproject.toml`. | ||
|
|
||
| ## Conclusion | ||
|
|
||
| In this guide, you learned how to use uv to manage Apify Actor projects. You can now create a uv project with the Apify SDK, run it locally with the Apify CLI, install the locked dependencies with uv in the Actor's Docker image, and deploy the whole project to the Apify platform with reproducible builds. If you have questions or need assistance, feel free to reach out on our [GitHub](https://github.com/apify/apify-sdk-python) or join our [Discord community](https://discord.com/invite/jyEM2PRvMU). Happy coding! | ||
|
|
||
| ## Additional resources | ||
|
|
||
| - [uv: Official documentation](https://docs.astral.sh/uv/) | ||
| - [uv: Working on projects](https://docs.astral.sh/uv/guides/projects/) | ||
| - [uv: Using uv in Docker](https://docs.astral.sh/uv/guides/integration/docker/) | ||
| - [Apify: Actor Dockerfile documentation](https://docs.apify.com/platform/actors/development/actor-definition/dockerfile) | ||
| - [Apify templates: Python](https://apify.com/templates/categories/python) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| # syntax=docker/dockerfile:1 | ||
| # First, specify the base Docker image. | ||
| # You can see the Docker images from Apify at https://hub.docker.com/r/apify/. | ||
| # You can also use any other image from Docker Hub. | ||
| FROM apify/actor-python:3.14 | ||
|
|
||
| # Add the uv binary from its official distroless image (pinned to the 0.11.x line). | ||
| COPY --from=ghcr.io/astral-sh/uv:0.11 /uv /uvx /bin/ | ||
|
|
||
| # Configure uv for container builds: | ||
| # - compile installed packages to bytecode, so the Actor starts faster, | ||
| # - copy packages instead of hardlinking, which avoids warnings with the cache mount, | ||
| # - never download a managed Python, always reuse the base image's interpreter, | ||
| # - put the project virtual environment first on PATH, so `python` resolves to it. | ||
| ENV UV_COMPILE_BYTECODE=1 \ | ||
| UV_LINK_MODE=copy \ | ||
| UV_PYTHON_DOWNLOADS=0 \ | ||
| PATH="/usr/src/app/.venv/bin:$PATH" | ||
|
|
||
| # Install dependencies into the project virtual environment (.venv) as a separate | ||
| # layer. The cache mount speeds up repeated builds, and the bind mounts make the | ||
| # project metadata available without copying it into the image. This layer is | ||
| # rebuilt only when uv.lock or pyproject.toml change - not on source code edits. | ||
| RUN --mount=type=cache,target=/root/.cache/uv \ | ||
| --mount=type=bind,source=uv.lock,target=uv.lock \ | ||
| --mount=type=bind,source=pyproject.toml,target=pyproject.toml \ | ||
| uv sync --locked --no-dev | ||
|
|
||
| # Next, copy the remaining files and directories with the source code. | ||
| # Since we do this after installing the dependencies, quick rebuilds will be | ||
| # really fast for most source file changes. | ||
| COPY . ./ | ||
|
|
||
| # Use compileall to ensure the runnability of the Actor Python code. | ||
| RUN python -m compileall -q my_actor/ | ||
|
|
||
| # Specify how to launch the source code of your Actor. | ||
| CMD ["python", "-m", "my_actor"] | ||
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| import asyncio | ||
|
|
||
| from .main import main | ||
|
|
||
| if __name__ == '__main__': | ||
| asyncio.run(main()) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| from apify import Actor | ||
|
|
||
|
|
||
| async def main() -> None: | ||
| async with Actor: | ||
| actor_input = await Actor.get_input() or {} | ||
| Actor.log.info('Actor input: %s', actor_input) | ||
| await Actor.set_value('OUTPUT', 'Hello from a uv-managed Actor!') |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| [project] | ||
| name = "my-actor" | ||
| version = "0.1.0" | ||
| description = "An Apify Actor with dependencies managed by uv." | ||
| requires-python = ">=3.14" | ||
| dependencies = [ | ||
| "apify>=3.0.0,<4.0.0", | ||
| ] |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.