Data streams v2 - #1192

Merged

1egoman merged 110 commits into

mainfrom

data-streams-v2

Jul 28, 2026

Commits on Jul 28, 2026

feat: commit initial data streams v2 implementation
1egoman
committed
587fbbf
fix: Create data_streams_v2.md
1egoman
committed
374ab09
fix: remove ClientCapability from prelude
1egoman
committed
cbbc1bf
feat: allow data stream compress option to be toggled via livekit-ffi
1egoman
committed
bc365df
fix: run cargo fmt
1egoman
committed
41c1bc8
generated protobuf

github-actions[bot]
authored and
1egoman
committed
9c052df
fix: address toc-tou issue with decompressor state
1egoman
committed
620f493
fix: cleanup data stream v2 test fixture initialization
1egoman
committed
bdbd599
feat: make client capabilities proto deserialization use TryFrom
1egoman
committed
5c16795
fix: run cargo fmt
1egoman
committed
00354ae
fix: remove dead code
1egoman
committed
c9e4e7f
generated protobuf

github-actions[bot]
authored and
1egoman
committed
39b6cd8
fix: downgrade protocol to before data track schema metadata
1egoman
committed
19a491e
refactor: convert data stream open tuple to struct and add is_compressed / is_inline
1egoman
committed
50bbbe8
feat: add is_compressed / is_inline to ByteStreamInfo / TextStreamInfo
1egoman
committed
b96ab98
Revert "refactor: convert data stream open tuple to struct and add is_compressed / is_inline"
1egoman
committed
e8f034b
feat: explicitly check is_compressed / is_inline in e2e tests
1egoman
committed
0ea9d54
fix: add e2e test that tests compress=false path
1egoman
committed
fd190db
feat: add e2e feature on compressed / inline fields
1egoman
committed
225b858
feat: get random bytes e2e test to pass by using much higher quality random data
1egoman
committed
6694a90
refactor: move RemoteParticipantRegistry to participant/registry.rs
1egoman
committed
e59093b
refactor: restructure send_text / send_bytes
1egoman
committed
5c133c4
feat: throw error when remove participant disconnects halfway through sending a data stream
1egoman
committed
e389d01
fix: adjust send_text / send_bytes to make the disambiguation between can_compress / should_compress more clear
1egoman
committed
b0ad52f
fix: clean up incoming stream manager
1egoman
committed
7ea9d3d
fix: preload buffer length
1egoman
committed
0011cf3
fix: add reason labels on e2e test assertions
1egoman
committed
f87aa3e
refactor: depend on livekit_common::RemoteParticipantRegistry
1egoman
committed
f1344d1
fix: remove arc wrapping around reserved_topics
1egoman
committed
ecbcc9a
refactor: rename is_topic_internal -> is_internal_topic, is_stream_internal -> is_internal
1egoman
committed
0d0544c
refactor: split up livekit-data-stream crate into multiple smaller modules
1egoman
committed
5c36cd2
fix: run cargo fmt
1egoman
committed
780ecc3
fix: add missing OperationType reexport
1egoman
committed
6975730
refactor: make pseudo_random_text easier to understand
1egoman
committed
8837cf1
fix: run cargo fmt
1egoman
committed
743df04
fix: drop recv_reader
1egoman
committed
360633a
fix: add MaybeCompressed docstring
1egoman
committed
98372e4
refactor: reformat somewhat_compressible to use rand
1egoman
committed
68a548a
Revert "refactor: drop unused ClientCapability, RemoteParticipantRegistry, and CLIENT_PROTOCOL_DATA_STREAM_V2 from livekit-common"
1egoman
committed
eab5187
Revert "refactor: merge livekit-data-stream __e2e-test feature into test-utils"
1egoman
committed
9ab255c
fix: drop duplicate outgoing test helpers introduced by rebase merge
1egoman
committed
1d48d77
refactor: use livekit_common::enum_dispatch in data-stream info dispatch
1egoman
committed
cae7f36
feat: add async-compression and thread it through via the new MaybeCollectedAsyncReader
1egoman
committed
de52326
feat: add new local struct / enum layer in livekit-data-stream mirroring the protobufs
1egoman
committed
f63906e
refactor: port outgoing data stream manager to use new packet types
1egoman
committed
af003ac
feat: convert incoming data stream manager to actor
1egoman
committed
51fbba7
fix: address typos in test only data stream fields
1egoman
committed
aaaea58
refactor: rename {Incoming,Outgoing}StreamManager -> *DataStreamManager
1egoman
committed
b16f733
fix: run cargo fmt
1egoman
committed
bf9e252
docs: remove out of date data stream comments
1egoman
committed
cd6962d
refactor: re-organize data streams v2 changes to more closely match data tracks
1egoman
committed
a25c4fe
refactor: move INTERNAL_DATA_STREAM_TOPICS to reduce diff complexity
1egoman
committed
88f02fd
fix: swap custom random generation for standard `rand` crate usage
1egoman
committed
c734fd0
fix: rename identifier to have a unit
1egoman
committed
eeb7826
refactor: centralize data streams test random noise generation helpers
1egoman
committed
7343ffd
refactor: remove flate2 from livekit-data-streams
1egoman
committed
8aefca7
fix: remove remnants of __e2e-test livekit-data-stream feature
1egoman
committed
2012f6f
refactor: change variable names and reindent
1egoman
committed
909a20c
fix: remove prost typos
1egoman
committed
bda79b9
fix: add missing test_utils module
1egoman
committed
3517a3d
fix: run cargo fmt
1egoman
committed
b5effac
fix: ensure that data stream outgoing manager is shut down when forward task shuts down
1egoman
committed
7e101e6
fix: add docstring for data stream options id
1egoman
committed
4699a87
fix: broke apart matches! conditional
1egoman
committed
4274105
fix: remove pub(crate)
1egoman
committed
cfa020c
feat: add builder methods for StreamByteOptions / StreamTextOptions
1egoman
committed
ba4aed4
feat: port to livekit-data-stream builder pattern crate wide
1egoman
committed
fe0158e
feat: wire up progress events for data streams
1egoman
committed
36d3061
fix: bump year in copyright header
1egoman
committed
a4b374d
refactor: simplify needless variable assignment
1egoman
committed
2d0c141
fix: drop non needed comment
1egoman
committed
0d51f6b
fix: unify use statements
1egoman
committed
570251e
refactor: replace .drain().collect() with .split_off() to get rid of an extra allocation
1egoman
committed
e9b02fa
fix: remove pub on StreamId.0
1egoman
committed
7ac3bee
feat: add close_streams_with_error to get rid of extra allocation
1egoman
committed
8fe6ecd
refactor: rename close_streams_with_error -> close_matching_streams_with_error
1egoman
committed
0e6dcc6
feat: add max data stream size limit configurable on room level
1egoman
committed
d22d8f8
fix: ensure that decompress state tracks output bytes, not input bytes
1egoman
committed
244acb4
refactor: adjust StreamByteOptions / StreamTextOptions to use builders and remove derive Default
1egoman
committed
36c5422
feat: add e2e test to verify max data stream room length option works
1egoman
committed
eede1ec
refactor: rename on_ -> handle_
1egoman
committed
c6f0250
fix: run cargo fmt
1egoman
committed
6455914
docs: add livekit-data-stream AGENTS.md file
1egoman
committed
64baad0
feat: add data stream max size limit to inline payload path as well
1egoman
committed
6cbba5a
fix: use StreamByteOptions builder in example
1egoman
committed
39608c8
feat: add enforce_header_size unit tests
1egoman
committed
63fb1f2
fix: update remaining examples to use new_with_topic
1egoman
committed
93ed070
feat: add support for trailer attributes to data streams
1egoman
committed
ba37708
fix: update rpc tests to not set attributes field directly
1egoman
committed
f8acb93
fix: update data stream tests to use .attributes()
1egoman
committed
89df1b4
feat: add "CompressionType::Unrecognized" to handle cases where a newer compression mechanism was added in the future
1egoman
committed
8279dc5
refactor: add nested modules to incoming / outgoing managers
1egoman
committed
9dd1541
feat: add more test coverage to incoming / outgoing managers
1egoman
committed
1b74121
refactor: rename collect -> as_bytes
1egoman
committed
f5cb102
fix: clone inline_content when testing
1egoman
committed
bb27dfa
fix: get rid of deflate_raw_i32 / none_i32
1egoman
committed
5c8ef30
feat: add ability to override data stream sender identity
1egoman
committed
f3bbb67
fix: ensure that passed file name is respected
1egoman
committed
6372f78
feat: add RoomDataStreamOptions to livekit-ffi
1egoman
committed
a1f9971
feat: add support for data stream trailer attributes via livekit-ffi
1egoman
committed
55d7a54
feat: add temporary flag to disable advertising data streams v2 support
1egoman
committed
265b344
fix: run cargo fmt
1egoman
committed
ed8ea54
feat: enforce max payload size on every incoming path
1egoman
committed
49e54db
feat: add doc hidden
1egoman
committed
9eed87a
generated protobuf
github-actions[bot]
committed
be53982
fix: update livekit-protocol version and regenerate protobuf definitions
1egoman
committed
ddfa8e8
fix: avoid divide by 0 for zero length data streams
1egoman
committed
ad574c3
fix: commit more protocol updates
1egoman
committed
66497c5
generated protobuf
github-actions[bot]
committed
510ac03
fix: add path sanitization to ByteStreamReader::write_to_file
1egoman
committed
30d580e