Skip to content

feat: smooth text streaming #12

Description

@ammario

Right now we display tokens as we get them, exposing the user to provider jitter.

My concept for fixing this is some kind of "smoothingBuffer" designed to always store
the last 250ms of tokens. There is an automatically adjusting dispersal rate that regulates the buffer to desired lookback.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Fields

No fields configured for Feature.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions