Data streams v2 uniffi - #1286
Conversation
29ffd71 to
7ef488b
Compare
Changeset ✓This PR includes a changeset covering all affected packages:
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
|
My suggested approach here is to just open a draft PR to consumers e.g. swift and then polish it - that's what I did for data tracks before. Otherwise the reviewer must generate the bindings anyway, as it's very hard to predict the edge cases around concurrency, memory management, etc. I'd be super happy to take a look at both in parallel. |
f28049b to
d1c8eff
Compare
| /// Data streams v2 core from [`livekit-data-stream`]. | ||
| pub mod data_stream; |
There was a problem hiding this comment.
🟡 Pull request is missing the required changeset file
No changeset file was added for this change (new module registered at livekit-uniffi/src/lib.rs:19), even though the repository requires every pull request to include one listing the crates that need a version bump.
Impact: Release tooling will not record or version this new functionality.
Repository rule
AGENTS.md ("Documenting changes"): "Every PR needs a changeset" and "Changeset must list any crates which need to be bumped stemming from the change". The diff adds livekit-uniffi/src/data_stream/* and a new dependency in livekit-uniffi/Cargo.toml, but .changeset/ contains only pre-existing entries (fix-publisher-renegotiation-deadlock.md, fix_nvenc_dynamic_bitrate_updates.md, fix_uniffi_android_package_build.md).
Add a changeset (e.g. via knope document-change) listing livekit-uniffi.
Was this helpful? React with 👍 or 👎 to provide feedback.
| /// Handles an encoded [`livekit_protocol::DataPacket`] received over the data channel. | ||
| /// | ||
| /// Fire-and-forget: the packet is decoded and enqueued in order; processing happens on the | ||
| /// manager's run loop. Non-data-stream or undecodable packets are ignored. | ||
| pub fn handle_packet_received(&self, packet: Bytes) { | ||
| if let Some(event) = decode_data_packet(&packet) { | ||
| let _ = self.input.send(event.into()); | ||
| } | ||
| } |
There was a problem hiding this comment.
🟡 Readers can wait forever when the sending participant leaves mid-stream
There is no way for the host app to tell the receiver that a participant has left (no equivalent of the abort input used elsewhere, only handle_packet_received at livekit-uniffi/src/data_stream/incoming.rs:77), so a half-received stream is never ended and a pending read never returns.
Impact: If a sender disconnects in the middle of a stream, an app awaiting the stream contents hangs indefinitely and the partially received stream is retained.
The Rust room implementation aborts streams on disconnect; the FFI has no such entry point
The incoming actor supports ds::incoming::InputEvent::AbortStreamsFrom(identity), which the livekit crate sends when a remote participant disconnects (livekit/src/room/mod.rs:2269), causing the descriptor to be dropped and the reader's channel to error/close.
IncomingDataStreamManager only exposes handle_packet_received, so a foreign host cannot deliver that event. Consequently ByteStreamReader::read_all/next (livekit-uniffi/src/data_stream/incoming.rs:105-117) and the text equivalents (:141-153) await a chunk that will never arrive, while holding the reader's tokio mutex, and the manager keeps the open-stream descriptor alive.
Fix: expose a method such as handle_participant_disconnected(identity: String) that sends InputEvent::AbortStreamsFrom.
Prompt for agents
IncomingDataStreamManager in livekit-uniffi/src/data_stream/incoming.rs only exposes handle_packet_received, but the underlying incoming actor also accepts ds::incoming::InputEvent::AbortStreamsFrom(ParticipantIdentity), which the livekit crate sends on remote participant disconnect (see livekit/src/room/mod.rs:2269). Without an FFI entry point for it, foreign hosts cannot terminate in-flight streams whose sender left, leaving readers awaiting chunks forever and descriptors retained in the manager. Add a synchronous exported method that forwards an identity as InputEvent::AbortStreamsFrom, documented as required on participant disconnect.
Was this helpful? React with 👍 or 👎 to provide feedback.
I think this is going to be a lot cleaner, and mean that other platforms like swift can handle "internal" data streams in their own way
| #[uniffi::constructor] | ||
| pub fn new( | ||
| delegate: Arc<dyn IncomingDataStreamManagerDelegate>, | ||
| max_payload_byte_length: Option<u64>, | ||
| ) -> Arc<Self> { |
There was a problem hiding this comment.
🟡 Newly exposed constructors lack the required documentation
The two entry points foreign callers must use to create the stream managers are exported without any description (pub fn new at livekit-uniffi/src/data_stream/incoming.rs:55 and livekit-uniffi/src/data_stream/outgoing.rs:84), so the generated bindings' documentation has nothing explaining their arguments.
Impact: Users of the generated bindings get undocumented constructors, including no explanation of the optional size limit argument.
Repository rule
AGENTS.md ("API changes") requires: "New APIs should have idiomatic doc comments — All new functions and types should have at least a one-line description". Every other exported method in these two files carries a doc comment; only the #[uniffi::constructor] pub fn new items do not (livekit-uniffi/src/data_stream/incoming.rs:54-58, livekit-uniffi/src/data_stream/outgoing.rs:83-87).
| #[uniffi::constructor] | |
| pub fn new( | |
| delegate: Arc<dyn IncomingDataStreamManagerDelegate>, | |
| max_payload_byte_length: Option<u64>, | |
| ) -> Arc<Self> { | |
| /// Creates a manager, spawning its actor loop on the global runtime. | |
| /// | |
| /// `max_payload_byte_length` caps the size of any single incoming stream; the crate default is | |
| /// used when `None`. | |
| #[uniffi::constructor] | |
| pub fn new( | |
| delegate: Arc<dyn IncomingDataStreamManagerDelegate>, | |
| max_payload_byte_length: Option<u64>, | |
| ) -> Arc<Self> { |
Was this helpful? React with 👍 or 👎 to provide feedback.
| pub(crate) fn decode_data_packet(bytes: &[u8]) -> Option<ds::incoming::PacketReceived> { | ||
| let mut packet = proto::DataPacket::decode(bytes).ok()?; | ||
| let identity: common::ParticipantIdentity = packet.participant_identity.clone().into(); | ||
| let ds_packet = match packet.value.take()? { | ||
| proto::data_packet::Value::StreamHeader(header) => ds::Packet::Header { | ||
| header: header.into(), | ||
| encryption_type: common::EncryptionType::None, | ||
| }, | ||
| proto::data_packet::Value::StreamChunk(chunk) => { | ||
| ds::Packet::Chunk { chunk: chunk.into(), encryption_type: common::EncryptionType::None } | ||
| } | ||
| proto::data_packet::Value::StreamTrailer(trailer) => ds::Packet::Trailer(trailer.into()), | ||
| _ => return None, | ||
| }; | ||
| Some(ds::incoming::PacketReceived::new(ds_packet, identity)) |
There was a problem hiding this comment.
🟨 Incoming data-stream packets are treated as unencrypted regardless of the packet's encryption state
decode_data_packet hardcodes common::EncryptionType::None for every decoded header/chunk packet (livekit-uniffi/src/data_stream/common.rs:352-366), so the incoming manager's encryption-type consistency check (descriptor.encryption_type != encryption_type, livekit-data-stream/src/incoming/manager.rs:377-380) can never detect a mismatch, and encrypted payloads would be surfaced to the application as plaintext bytes. The behavior is documented as a follow-up ("the foreign side is expected to hand us already-decrypted packets"), but nothing in the FFI enforces or signals that contract.
Was this helpful? React with 👍 or 👎 to provide feedback.
Expose all the different error cases so consuming clients can get better quality errors.
This uses tokio for one (so it's not portable), but also had some security flaws (allowed paths like ../../evil.txt). So, remove it.
b0113f2 to
76f1e16
Compare
… `livekit-data-stream` (#1304) While working on #1286, I realized I made a bit of a mess of the "internal" data stream concept when extracting `livekit-data-stream` out into its own crate. This pull request attempts to address this. **Currently, what happens today:** `livekit-data-stream`'s `IncomingDataStreamManager` takes a list of `INTERNAL_DATA_STREAM_TOPICS`. These are topics used internally for v2 rpc requests / responses and messages associated with these data streams are not exposed to the user. They are filtered in two places: - Within `IncomingDataStreamManager` to conditionally expose both `InputEvent::ChunkReceived` and `InputEvent::TrailerReceived` for non internal topics - Within `RoomSession` to expose which `InputEvent::StreamOpened` events are exposed to the user. Doing this here meant that "internal" data streams could be intercepted at this level and fed into rpc / etc. This is messy - it's a little hard to follow and the filtering is split into these two places / some code duplication. What really made this untenable though was that exposing this internal state over uniffi in #1286 became particularly challenging, since you _do_ want to expose "internal" events downstream in this case - if you didn't, the uniffi consuming client couldn't handle rpc v2 / etc! **What this pull request does:** Now, `livekit-data-stream` knows nothing about "internal" or "not internal" data streams, and includes the raw `topic` of each stream in the associated event. This allows all the filtering to be centralized downstream in `RoomSession`. This fixes all of these previously mentioned problems and is a lot easier to reason about!
2 participants
Adds data streams v2 to the
livekit-unifficrate, and exposes it fairly similarly to how data tracks works.I've also for now added a python test script (It's the most complete bindgen set up in this project which I am familiar with) which exercises this api as an example of what it would look like in practice. I'll remove this before an eventual merge, but I thought it would be useful for reviewers:
$ python3 datastream_uniffi_test.py --- OUTGOING: PACKETS: [b'jP\n$4a501804-960c-4d54-94df-81784ffab6b1\x10\xc6\xa5\xb0\xaf\xf93\x1a\x04test"\ntext/plain(\x0bJ\x00Z\x0bhello world'] --- INCOMING: TEXT STREAM OPENED: alice CONTENTS: hello world