Improve error message for Python decorator invoking function with same name as the task name

[eladkal]

Body

Given code:

from datetime import datetime
from airflow import DAG
from airflow.decorators import task
from time import sleep

default_args = {
    'owner': 'airflow',
    'start_date': datetime(2023, 2, 1)
}


with DAG('scheduler', schedule="@daily", catchup=True, default_args=default_args):

    @task()
    def sleep():
        sleep(3600)
    sleep()

It fails with:

TypeError: too many positional arguments

File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/task_runner.py", line 829 in run

File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/task_runner.py", line 1092 in _execute_task

File "/opt/airflow/task-sdk/src/airflow/sdk/bases/operator.py", line 408 in wrapper

File "/opt/airflow/task-sdk/src/airflow/sdk/bases/decorator.py", line 251 in execute

File "/opt/airflow/task-sdk/src/airflow/sdk/bases/operator.py", line 408 in wrapper

File "/opt/airflow/providers/standard/src/airflow/providers/standard/operators/python.py", line 212 in execute

File "/opt/airflow/providers/standard/src/airflow/providers/standard/operators/python.py", line 235 in execute_callable

File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/callback_runner.py", line 81 in run

File "/files/dags/scheduler.py", line 15 in sleep

File "/opt/airflow/task-sdk/src/airflow/sdk/bases/decorator.py", line 363 in __call__

File "/opt/airflow/task-sdk/src/airflow/sdk/bases/operator.py", line 513 in apply_defaults

File "/opt/airflow/providers/standard/src/airflow/providers/standard/decorators/python.py", line 58 in __init__

File "/opt/airflow/task-sdk/src/airflow/sdk/bases/operator.py", line 513 in apply_defaults

File "/opt/airflow/task-sdk/src/airflow/sdk/bases/decorator.py", line 239 in __init__

File "/usr/local/lib/python3.9/inspect.py", line 3045 in bind

File "/usr/local/lib/python3.9/inspect.py", line 2966 in _bind

The reason for this failure is because the python task has the exact name as the python function. My real code was more complex than the example I share here.. it took me several minutes to figure out what was the issue. Not sure if we can fix it but if we can improve the error message we should

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[giannistbs]

Hello! Can i work on this?

[ashb]

Hello! Can i work on this?

What is your plan?

[kiran2706]

@ashb - Can I take a shot at this?

My plan is to implement it by intercepting function calls in TaskDecorator.call, and inspecting the call stack to detect potential self-invocations caused by naming collisions between the function and the task object.
If detected, I'll log a clear warning or optionally raise an informative error to help users debug it faster.
Let me know if you have any suggestions before I start

cc: @potiuk

[vatsrahul1001]

I can repro this as well

[suman-himanshu]

I am able to reproduce this issue at my end.

[ashb]

@ashb - Can I take a shot at this?

My plan is to implement it by intercepting function calls in TaskDecorator.call, and inspecting the call stack to detect potential self-invocations caused by naming collisions between the function and the task object. If detected, I'll log a clear warning or optionally raise an informative error to help users debug it faster. Let me know if you have any suggestions before I start

Ideally though we do that without inspecting the callstack in "happy" path cases, as looking at the call stack is expensive.

[gitbita99]

@ashb - Can I take a shot at this?
My plan is to implement it by intercepting function calls in TaskDecorator.call, and inspecting the call stack to detect potential self-invocations caused by naming collisions between the function and the task object. If detected, I'll log a clear warning or optionally raise an informative error to help users debug it faster. Let me know if you have any suggestions before I start

Ideally though we do that without inspecting the callstack in "happy" path cases, as looking at the call stack is expensive.

Hi @ashb , I would like to work in this issue

Problem: When users shadow imports with task name, they get a cryptic TypeError: too many positional arguments.

Solution: Add exception handling in DecoratedOperator.__init__ to provide a helpful hint message - only when the error occurs (No overhead).

The fix catches the TypeError during signature binding, adds guidance / hint and returns it.

I have a working implementation ready. Could you please assign this issue to me?

[Gourigupta2004]

Hi @eladkal,

I would like to investigate and work on this issue. Please assign this to me

Thanks

[potiuk]

We do not assign issues (see contributing guides) - but feel free to work on it.