-
Notifications
You must be signed in to change notification settings - Fork 17.3k
docs: clarify AssetAlias usage #67392
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -328,12 +328,18 @@ The shorthand for this is ``@asset.multi``: | |
|
|
||
| Dynamic data events emitting and asset creation through AssetAlias | ||
| ----------------------------------------------------------------------- | ||
| An asset alias can be used to emit asset events of assets with association to the aliases. Downstreams can depend on resolved asset. This feature allows you to define complex dependencies for Dag executions based on asset updates. | ||
| Use ``AssetAlias`` when a task must declare an asset dependency before the concrete asset URI is | ||
| known. The alias is listed in ``outlets`` as a stable name, and the task resolves it at runtime by | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What does "stable name" mean? What does "concrete" mean? |
||
| adding one or more concrete ``Asset`` objects through ``outlet_events`` or yielded ``Metadata``. | ||
| Downstream Dags can depend on the alias, and Airflow triggers them when events are emitted for the | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "trigger" is kind of an overloaded term in Airflow. Maybe we should say "...and Airflow initiates a Dag run when..." |
||
| resolved assets. | ||
|
|
||
| How to use AssetAlias | ||
| ~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| ``AssetAlias`` has one single argument ``name`` that uniquely identifies the asset. The task must first declare the alias as an outlet, and use ``outlet_events`` or yield ``Metadata`` to add events to it. | ||
| ``AssetAlias`` has one single argument ``name`` that uniquely identifies the alias. The task must | ||
| first declare the alias as an outlet, and then use ``outlet_events`` or yield ``Metadata`` during | ||
| execution to associate the alias with the concrete assets it produced. | ||
|
|
||
| The following example creates an asset event against the S3 URI ``f"s3://bucket/my-task"`` with optional extra information ``extra``. If the asset does not exist, Airflow will dynamically create it and log a warning message. | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
imo we should not use "i.e.". What about
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
as lone as we use both name and uri, I'm good 👀