Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
116 changes: 116 additions & 0 deletions docs/signals.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# Signals

A **signal** runs the same labeling job on a repeating schedule: bind a
[job definition](job_definition_parameters.md) to an [audience](audiences.md)
and an interval, and Rapidata creates a new [job](understanding_the_results.md)
on every tick.

## What a signal is

A signal ties together three things:

- an **audience** — who labels the data,
- a **job definition** — the task that gets run,
- an **interval** — how often it fires, in hours.

Each firing creates one `RapidataJob`, identical to a job you create directly.
A signal is just a scheduler that keeps producing those jobs; every job runs the
same job definition against the same audience.

```mermaid
graph LR
S[Signal<br/>audience + job definition + interval] -->|every interval| J1[Job 1]
S -->|every interval| J2[Job 2]
S -->|every interval| J3[Job 3]
```

## Creating a signal

```py
from rapidata import RapidataClient

client = RapidataClient()

audience = client.audience.get_audience_by_id("aud_MU1GZYoESyO")

job_definition = client.job.create_compare_job_definition(
name="Prompt Alignment Job",
instruction="Which image follows the prompt more accurately?",
datapoints=[
["https://assets.rapidata.ai/flux_book.jpg",
"https://assets.rapidata.ai/mj_book.jpg"]
],
contexts=["A small blue book sitting on a large red book."],
)

signal = client.signals.create_signal(
name="Daily prompt alignment",
audience=audience,
job_definition=job_definition,
interval_hours=24,
)
```

`audience` and `job_definition` also accept id strings. By default the signal
uses the latest revision of the job definition at fire time; pin one with
`revision_number=...`. Set `is_public=True` to let others in your organization
read the signal.

## The jobs a signal creates

Every firing creates a `RapidataJob`:

```py
for job in signal.get_jobs(page_size=10):
print(job, job.get_status())

results = signal.get_jobs(page_size=1)[0].get_results()
```

A firing can be skipped (for example if the previous job hasn't finished). A
skipped firing creates no job and won't appear in `get_jobs()`.

## Triggering a job on demand

Fire one job immediately instead of waiting for the schedule:

```py
signal.trigger()
job = signal.wait_for_next_job(timeout=600)
print(job.get_results())
```

`trigger()` returns right away; the job is created in the background.
`wait_for_next_job()` blocks until the next firing has created its job and
returns it.

## Managing a signal

```py
signal.pause()
signal.resume()
signal.update(name="Hourly prompt alignment", interval_hours=1)
signal.delete()
```

Look signals up later:

```py
signal = client.signals.get_signal_by_id("signal_id")
signals = client.signals.find_signals(name="alignment")
```

## Property reference

| Property | Description |
|---|---|
| `id` | The signal's unique id. |
| `name` / `description` | Display name and optional description. |
| `audience_id` | The audience each job targets. |
| `job_definition_id` | The job definition each job is created from. |
| `revision_number` | Pinned job-definition revision, or `None` for "latest at fire time". |
| `interval_hours` | How often the signal fires, in hours. |
| `next_run_at` / `last_run_at` | Timestamps of the next and most recent firings. |
| `is_paused` | Whether the scheduler is currently skipping this signal. |
| `is_public` | Whether other users can discover and read it. |
| `created_at` | When the signal was created. |
2 changes: 2 additions & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ plugins:
- starting_page.md
- quickstart.md
- audiences.md
- signals.md
- job_definition_parameters.md
- understanding_the_results.md
- error_handling.md
Expand Down Expand Up @@ -120,6 +121,7 @@ nav:
- Overview: starting_page.md
- Quick Start: quickstart.md
- Custom Audiences: audiences.md
- Signals: signals.md
- Parameter Reference: job_definition_parameters.md
- Understanding Results: understanding_the_results.md
- Early Stopping: confidence_stopping.md
Expand Down
2 changes: 2 additions & 0 deletions src/rapidata/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@
RapidataJob,
RapidataJobDefinition,
RapidataJobManager,
RapidataSignal,
RapidataSignalManager,
ValidationSetManager,
RapidataValidationSet,
Box,
Expand Down
1 change: 1 addition & 0 deletions src/rapidata/rapidata_client/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
)
from .order import RapidataOrderManager, RapidataOrder
from .job import RapidataJob, RapidataJobDefinition, RapidataJobManager
from .signal import RapidataSignal, RapidataSignalManager
from .validation import ValidationSetManager, RapidataValidationSet, Box
from .results import RapidataResults
from .selection import (
Expand Down
10 changes: 10 additions & 0 deletions src/rapidata/rapidata_client/rapidata_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,9 @@
from rapidata.rapidata_client.datapoints._asset_uploader import AssetUploader
from rapidata.rapidata_client.job.rapidata_job_manager import RapidataJobManager
from rapidata.rapidata_client.flow.rapidata_flow_manager import RapidataFlowManager
from rapidata.rapidata_client.signal.rapidata_signal_manager import (
RapidataSignalManager,
)
from rapidata.rapidata_client.api.rapidata_api_client import (
optional_api_call,
mark_sdk_outdated,
Expand Down Expand Up @@ -108,6 +111,8 @@ def __init__(
audience (RapidataAudienceManager): The RapidataAudienceManager instance.
job (JobManager): The JobManager instance.
mri (RapidataBenchmarkManager): The RapidataBenchmarkManager instance.
signals (RapidataSignalManager): The RapidataSignalManager instance for managing
recurring audience-job schedules (signals) and observing their runs.
context (ContextManager): The ContextManager instance for shortening
datapoint contexts against a question.
"""
Expand Down Expand Up @@ -165,6 +170,11 @@ def __init__(
logger.debug("Initializing RapidataBenchmarkManager")
self.mri = RapidataBenchmarkManager(openapi_service=self._openapi_service)

logger.debug("Initializing RapidataSignalManager")
self.signals = RapidataSignalManager(
openapi_service=self._openapi_service
)

logger.debug("Initializing RapidataAudienceManager")
self.audience = RapidataAudienceManager(
openapi_service=self._openapi_service
Expand Down
2 changes: 2 additions & 0 deletions src/rapidata/rapidata_client/signal/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
from .rapidata_signal import RapidataSignal
from .rapidata_signal_manager import RapidataSignalManager
Loading
Loading