Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 10 additions & 11 deletions Common/README.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,20 @@
# lib-pixelstreamingcommon

The common library for the browser/client side of Pixel Streaming experiences. This library exposes common functionality for frontend applications of the pixel streaming ecosystem.
This library exposes common functionality for Pixel Streaming web applications. Examples include message protocols, logging, events, and various utilities.

For examples of how to implement this library.
- See [lib-pixelstreamingfrontend](/Frontend/library)
- See [lib-pixelstreamingsignalling](/Signalling)
- Also see [Wilbur](/SignallingWebServer) which uses the signalling library to implement a full signalling server.
For examples of how to implement this library, see:
- [lib-pixelstreamingfrontend](https://github.com/EpicGamesExt/PixelStreamingInfrastructure/tree/master/Frontend/library)
- [lib-pixelstreamingsignalling](https://github.com/EpicGamesExt/PixelStreamingInfrastructure/tree/master/Signalling)
- [Wilbur](https://github.com/EpicGamesExt/PixelStreamingInfrastructure/tree/master/SignallingWebServer) (the reference Pixel Streaming signalling server)

### Adding it to your project
`npm i @epicgames-ps/lib-pixelstreamingcommon-ue5.6`
`npm i @epicgames-ps/lib-pixelstreamingcommon-ue5.5`

## Documentation
The API is documented [here](docs).
The API is documented [here](https://github.com/EpicGamesExt/PixelStreamingInfrastructure/tree/master/Common/docs).

### How this library is built
The NPM packages supports:
- ES6 module usage
- CommonJS usage
### NPM package contents
- ES6 modules
- CommonJS
- Type definitions
- Source maps
2 changes: 1 addition & 1 deletion Common/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@epicgames-ps/lib-pixelstreamingcommon-ue5.7",
"version": "0.1.0",
"version": "0.1.1",
"description": "Common utilities library for Unreal Engine 5.7 Pixel Streaming",
"main": "dist/cjs/pixelstreamingcommon.js",
"module": "dist/esm/pixelstreamingcommon.js",
Expand Down
71 changes: 24 additions & 47 deletions Frontend/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Pixel Streaming Frontend

The **frontend** refers to the HTML, CSS, images, and JavaScript/TypeScript code that runs in web browsers and allows them to connect to Unreal Engine Pixel Streaming applications and interact with them. The frontend library is the foundation that developers can modify and extend to suit the needs of their Pixel Streaming experience.
The **frontend** refers to the HTML, CSS, images, and JavaScript/TypeScript code that runs in web browsers and allows them to connect to Pixel Streaming applications and interact with them. The frontend library is the foundation that developers can modify and extend to suit the needs of their Pixel Streaming experience.

The frontend consists of two packages:
The frontend consists of these packages:

1. [lib-pixelstreamingfrontend](/Frontend/library/): the core Pixel Streaming frontend for WebRTC, settings, input, and general functionality.
2. [lib-pixelstreamingfrontend-ui](/Frontend/ui-library/): the UI library that users can either optionally apply on top of the core library or build on top of.
Expand All @@ -11,82 +11,59 @@ The frontend consists of two packages:
These libraries are published as [NPM packages](/README.md#npm-packages) and support usage as ES6 modules, CommonJS, include type definitions and source maps.

## Docs
- [The Settings Panel](Docs/Settings%20Panel.md)
- [Customizing the Player Webpage](Docs/Customizing%20the%20Player%20Webpage.md)
- [HTML Page Requirements](Docs/HTML%20Page%20Requirements.md)
- [Player File Location and URL](Docs/HTML%20Page%20Requirements.md)
- [Customizing Player Input Options](Docs/Customizing%20Player%20Input%20Options.md)
- [Disabling User Input](Docs/Customizing%20Player%20Input%20Options.md)
- [Customizing the Player Widget Style](Docs/Customizing%20the%20Player%20Widget%20Style.md)
- [Accessing the Pixel Streaming Blueprint API](Docs/Accessing%20the%20Pixel%20Streaming%20Blueprint%20API.md)
- [Communicating from the Player Page to UE5](Docs/Communicating%20from%20the%20Player%20Page%20to%20UE5.md)
- [Using the emitCommand Function](Docs/Communicating%20from%20the%20Player%20Page%20to%20UE5.md)
- [Using the emitUIInteraction Function](Docs/Communicating%20from%20the%20Player%20Page%20to%20UE5.md)
- [Communicating from UE5 to the Player Page](Docs/Communicating%20from%20UE5%20to%20the%20Player%20Page.md)
- [Timing Out Inactive Connections](Docs/Timing%20Out%20Inactive%20Connections.md)
- [Frontend usage docs](/Frontend/Docs/)
- [Common API](/Common/docs/)
- [Signalling protocol](/Common/docs/Protocol.md)
- [Signalling API](/Signalling/docs/)

## Integrating the libraries into your project
The TypeScript libraries are provided as both an [NPM](https://www.npmjs.com/settings/epicgames-ps/packages) package and a [UMD](https://github.com/umdjs/umd) module (available via [unpkg](https://unpkg.com/)), making it easy to consume the libraries from either TypeScript code or plain JavaScript code using modern web development tools and workflows.
The TypeScript libraries are provided as both an [NPM](https://www.npmjs.com/settings/epicgames-ps/packages) package containing ES6 modules and a CommonJS, making it easy to consume the libraries from either TypeScript code or plain JavaScript code using modern web development tools and workflows.

## Usage from source

When developing your own Pixel Streaming experience the intent is you will start with this library and extend it through the use of
its public API. We have provided an example of this workflow in our [implementations/typescript](/Frontend/implementations/typescript), which is an implementation of the frontend libraries and contains a working example of how you can bundle/minify your final application JavaScript..
its public API. We have provided an example of this workflow in our [implementations/typescript](/Frontend/implementations/typescript), which is an implementation of the frontend libraries and contains a working example of how you can bundle/minify your final application JavaScript.

## Contributing

If part of the library is not exposed and you wish to extend it, please do so in your own branch and open a pull request with your change for our consideration.

## Developing

⚠️ Only NodeJS LTS 22.14.0 is officially supported, some newer versions on NodeJS **WILL BREAK YOUR BUILD** ⚠️
⚠️ Only [this](https://github.com/EpicGamesExt/PixelStreamingInfrastructure/blob/master/NODE_VERSION) NodeJS version is officially supported, other versions may **BREAK YOUR BUILD** ⚠️

### Prerequisites for local dev
- Install NodeJS LTS 22.14.0 on your system.
- Install npm globally using: `npm install npm -g` (yes this is required)
- Install NodeJS version specified above.
- Ensure `npm` is available in your terminal.

### Building the Library
### Building the frontend library

Changes to the library occur in the [/library](/Frontend/library) directory and require you to have NodeJS installed as part of your development environment.
Once you have NodeJS installed:

- `cd library`
- `npm install`
- `npm run build`
Once you have NodeJS installed, refer to the build instructions in [/Frontend/library/](/Frontend/library/)

### Building the UI-Library
### Building the frontend UI library

The user interface library is provided in [/ui-library](/Frontend/ui-library) directory. You can either use it or provide your own user interface. To build run:
- Follow the steps to build the library first
- `cd ui-library`
- `npm install`
- `npm run build`
The user interface library is provided in [/ui-library](/Frontend/ui-library) directory. You can either use it or provide your own user interface.

### Building the default UI
To build run refer to the build instructions in [Frontend/ui-library/](/Frontend/ui-library).

The default user interface is provided under [/implementations/typescript](/Frontend/implementations/typescript). To build run:
### Building the reference frontend (the web page shipped with the Pixel Streaming plugin)

- Follow the steps to build the libary and ui-library first
- `cd implementations/typescript`
- `npm install`
- `npm run build`
The default Pixel Streaming web player is provided under [/implementations/typescript](/Frontend/implementations/typescript).

This will produce `player.html` and `player.js` under the `SignallingWebServer/Public` directory - this is the default UI.
To build run refer to the build instructions in [/implementations/typescript/](/Frontend/implementations/typescript).

### Making your own UI
This will produce `player.html` and `player.js` under the `SignallingWebServer/Public` directory - these are the default Pixel Streaming web frontend.

We recommend studying [/ui-library](/Frontend/ui-library) and [player.ts](/Frontend/implementations/typescript/src/player.ts)/[player.html](/Frontend/implementations/typescript/src/player.html), or alternatively the sample React implementation in [implementations/react](/Frontend/implementations/react), then once you have copied and modified the [package.json](/Frontend/implementations/typescript/package.json) and `.ts` into your own `implementation/your_implementation` directory, the process is similar:
### Making your own UI

- `cd implementation/your_implementation`
- `npm run build`
We recommend studying [/ui-library](/Frontend/ui-library) and [player.ts](/Frontend/implementations/typescript/src/player.ts)/[player.html](/Frontend/implementations/typescript/src/player.html), or alternatively the sample React implementation in [implementations/react](/Frontend/implementations/react).

## Unit tests

The [/library](/Frontend/library) project has unit tests that test the Pixel Streaming functionality against a mocked connection. To run the tests manually, run:
- `cd library`
- `npm install`
- `npm run test`
Many of the libraries, such as [/frontend/library/](/Frontend/library), contain unit tests that test the Pixel Streaming functionality against a mocked connection. To run the tests manually study the `package.json` in the root of the library.

## Legal

Copyright © 2025, Epic Games. Licensed under the MIT License, see the file [LICENSE](/LICENSE.md) for details.
Copyright © 2025, Epic Games. Licensed under the MIT License, see the file [LICENSE](/LICENSE.md) for details.
21 changes: 15 additions & 6 deletions Frontend/implementations/typescript/readme.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,23 @@
## TypeScript Reference Frontend
## Reference Pixel Streaming web frontend (written in vanilla TypeScript)

This is the frontend we ship with the Pixel Streaming plugin. The reference frontend contains:
This is the stock look-and-feel Pixel Streaming player web page you are served out of the box with the shipped Pixel Streaming plugin. It is widely tested and is a fully featured Pixel Streaming experience.

1. The base `lib-pixelstreamingfrontend` library.
2. The reference ui plugin for the base library `lib-pixelstreamingfrontend-ui`.
**This is great starting point for building your UI or studying the Pixel Streaming feature set.**

![Frontend](https://raw.githubusercontent.com/EpicGamesExt/PixelStreamingInfrastructure/0aabae464daa95925cf6fa238ac18d0a5561a421/Frontend/implementations/EpicGames/docs/images/frontend.jpg)

The reference frontend uses:

1. The base `lib-pixelstreamingfrontend` library for all core streaming functionality.
2. The reference UI library `lib-pixelstreamingfrontend-ui` for all the UI.

Using these two libraries gives a fully functional (and customizable) Pixel Streaming experience.

This package is also a good example of how to include the frontend libraries as dependencies and bundle/minify the final application you ship.

### Key features of the reference frontend
- An info panel (screen right) that provides a UI for displaying live statistics to the user.
- A settings panel (screen right) that provides a UI for all the options inside [config.ts](/Frontend/library/src/Config/Config.ts).
- A settings panel (screen right) that provides a UI for all the [settings](https://github.com/EpicGamesExt/PixelStreamingInfrastructure/blob/master/Frontend/Docs/Settings%20Panel.md).
- A set of controls (screen left) to maximize the video, open the settings panel, open the info panel, and enter VR mode.
- Ability to display overlays that present information or errors to the user, or present prompts for the user to interact with.

Expand All @@ -22,6 +28,8 @@ npm install
npm run build
```

**Note:** You will need to build the `Frontend/library` and `Frontend/ui-library` first.

### Using the reference frontend
Building the reference frontend using the commands above will place it in the `SignallingWebServer/www` directory.
```
Expand All @@ -31,4 +39,5 @@ start.bat
# Navigate to http://localhost in your browser to see the reference frontend
```

***Note:* You can also run `start.bat --build` to build all the dependent libraries.
**Note:** You can also run `start.bat --build` to build all the dependent libraries.

6 changes: 6 additions & 0 deletions Frontend/library/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# @epicgames-ps/lib-pixelstreamingfrontend-ue5.6

## 0.1.1

### Patch Changes

- 7e5dfb7: Some versions of Firefox were unable to connect due the changes in PR#694 to overcome this issue and preserve the connectivity fixes from PR#694 we now assume the sdpMLineIndex is always 0 for bundle master media line. This change was tested on many browsers and restores connectivity with FireFox.

## 0.2.2

### Patch Changes
Expand Down
2 changes: 1 addition & 1 deletion Frontend/library/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@epicgames-ps/lib-pixelstreamingfrontend-ue5.7",
"version": "0.1.0",
"version": "0.1.2",
"description": "Frontend library for Unreal Engine 5.7 Pixel Streaming",
"main": "dist/cjs/pixelstreamingfrontend.js",
"module": "dist/esm/pixelstreamingfrontend.js",
Expand Down
21 changes: 10 additions & 11 deletions Frontend/library/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,21 @@

The core library for the browser/client side of Pixel Streaming experiences. **This library contains no UI.**

See [lib-pixelstreamingfrontend-ui](/Frontend/implementations/typescript) for an example of how to build UI on top of this library.
See the [reference Pixel Streaming web player](https://github.com/EpicGamesExt/PixelStreamingInfrastructure/tree/master/Frontend/implementations/typescript) for an example of how to build a Pixel Streaming web application, with custom UI, on top of this library.

### Key features
- Create a websocket connection to communicate with the signalling server.
- Create a WebRTC peer connection that displays the Unreal Engine video and audio.
- Handling of input from the user and transmitting it back to Unreal Engine.
- Opens a data channel connection sending and receiving custom data (in addition to input).
- Programmable and url specified settings.
- Create a WebSocket connection to communicate with the signalling server.
- Create a WebRTC peer connection that displays the Pixel Streaming video and audio.
- Handling of input from the user and transmitting it back to Pixel Streaming application.
- Opens a data channel connection for sending and receiving custom data (in addition to input).
- Programmable (or url specified) settings.

### Adding it to your project
`npm install @epicgames-ps/lib-pixelstreamingfrontend-ue5.5`
`npm install @epicgames-ps/lib-pixelstreamingfrontend-ue5.7`

### How this library is built
The NPM packages supports:
- ES6 module usage
- CommonJS usage
### NPM package contents
- ES6 modules usage
- CommonJS lib
- Type definitions
- Source maps

Expand Down
8 changes: 4 additions & 4 deletions Frontend/library/src/WebRtcPlayer/WebRtcPlayerController.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1485,13 +1485,13 @@ export class WebRtcPlayerController {
handleIceCandidate(iceCandidateInit: RTCIceCandidateInit) {
Logger.Info(`Remote ICE candidate information received: ${JSON.stringify(iceCandidateInit)}`);

// We are using "bundle" policy for media lines so we remove the sdpMid and sdpMLineIndex attributes
// from ICE candidates as these are legacy attributes for when bundle is not used.
// We are using "bundle" policy for media lines so we manually set the sdpMLineIndex attribute to 0 (our assumed bundle master media line)
// If we don't do this the browser may be unable to form a media connection
// because some browsers are brittle if the bundle master (e.g. commonly mid=0) doesn't get a candidate first.
// because some browsers are brittle if the bundle master doesn't get a candidate first.
// Note: This assumes we are using bundle and that the bundle master is the first media line (0).
const remoteIceCandidate = new RTCIceCandidate({
candidate: iceCandidateInit.candidate,
sdpMid: ''
sdpMLineIndex: 0
});

this.peerConnectionController.handleOnIce(remoteIceCandidate);
Expand Down
26 changes: 13 additions & 13 deletions Frontend/ui-library/README.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,23 @@
## lib-pixelstreamingfrontend-ui

A plugin library that add UI support to the `lib-pixelstreamingfrontend` library.
A library that adds UI support to the `lib-pixelstreamingfrontend` library.

This UI library depends on a the base library as a peer dependency, so you should include both as dependencies in your application
if you want to use this library.
### Adding it to your project

This UI library depends on a the frontend library, so to include this library in your project:

`npm i @epicgames-ps/lib-pixelstreamingfrontend-ue5.7`

`npm i @epicgames-ps/lib-pixelstreamingfrontend-ui-ue5.7`

### Key features
- Stock Pixel Streaming frontend
- Setting panel
- Default Pixel Streaming frontend look-and-feel
- Settings panel
- Info panel

### Adding it to your project
`npm i @epicgames-ps/lib-pixelstreamingfrontend-ue5.5`
`npm i @epicgames-ps/lib-pixelstreamingfrontend-ui-ue5.5`

### How this library is built
The NPM packages supports:
- ES6 module usage
- CommonJS usage
### NPM package contents
- ES6 modules
- CommonJS
- Type definitions
- Source maps

Expand Down
2 changes: 1 addition & 1 deletion Frontend/ui-library/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@epicgames-ps/lib-pixelstreamingfrontend-ui-ue5.7",
"version": "0.1.0",
"version": "0.1.1",
"description": "Reference frontend UI library for Unreal Engine 5.7 Pixel Streaming - gives the stock look and feel.",
"main": "dist/cjs/pixelstreamingfrontend-ui.js",
"module": "dist/esm/pixelstreamingfrontend-ui.js",
Expand Down
19 changes: 10 additions & 9 deletions Signalling/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@

# @epicgames-ps/lib-pixelstreamingsignalling

A library for developers wanting to implement tools that signal Pixel Streaming applications.
A library for developers wanting to implement tools and applications that signal Pixel Streaming applications.

## Building

Expand All @@ -13,19 +14,19 @@ npm run build

This will result in a `/dist` output directory.

**Note:** If you just want to get a signalling server up and running refer to ["Getting Started"](../README.md#getting-started)
**Note:** If you just want to get a signalling server up and running refer to ["Getting Started"](https://github.com/EpicGamesExt/PixelStreamingInfrastructure?tab=readme-ov-file#getting-started).

### How this library is built
The NPM packages supports:
- ES6 module usage
- CommonJS usage
### NPM package contents
- ES6 module
- CommonJS
- Type definitions
- Source maps

## Documentation

- [Protocol Notes](../Common/docs/Protocol.md)
- [Protocol Messages](../Common/docs/messages.md)
- [Protocol Notes](https://github.com/EpicGamesExt/PixelStreamingInfrastructure/blob/master/Common/docs/Protocol.md)
- [Protocol Messages](https://github.com/EpicGamesExt/PixelStreamingInfrastructure/blob/master/Common/docs/messages.md)

The API documentation is [here](https://github.com/EpicGamesExt/PixelStreamingInfrastructure/tree/master/Signalling/docs) and covers details of all exported components of the library.

The API documentation is [here](docs/) and covers details of all exported components of the library.

2 changes: 1 addition & 1 deletion Signalling/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@epicgames-ps/lib-pixelstreamingsignalling-ue5.7",
"version": "0.1.0",
"version": "0.1.1",
"description": "Basic signalling library for developers wishing to build applications that signal a Pixel Streaming application.",
"main": "dist/cjs/pixelstreamingsignalling.js",
"module": "dist/esm/pixelstreamingsignalling.js",
Expand Down
Loading