Standardized artwork kinds for GameStream app entries

[evertonstz]

Select Topic Area

Feature Request

Body

Summary

GameStream carries one artwork slot per app. Because there is only one, a host holding
good artwork has to damage it to fit. Moonshine pillarboxes square desktop icons onto
a portrait canvas and its own source comment says they get distorted. It also decodes
Steam's JPEG capsules and re-encodes them to PNG on every request. Its fix for missing
artwork had to funnel a 600x900 Steam library capsule and a square freedesktop icon
into the same config field, because only one field exists. Sunshine returns a
placeholder image rather than an error, which is why moonlight-qt inspects pixel
dimensions to guess whether it received real art at all.

None of that is a bug in any of those projects. It is what one artwork slot forces.

This proposal standardizes five artwork kinds (capsule, wide capsule, hero, logo,
icon) so a host can serve the artwork it already holds without reshaping it, and so
clients can use it. A kind names what an image is, not what shape it has: the host
declares each asset's real pixel dimensions and the client decides how to use them.
Today's box art request does not change, since it becomes the capsule kind, so
existing hosts and existing clients are unaffected.

Four independent host implementations (Sunshine, Apollo, Wolf and Moonshine) have all
converged on one PNG per app with the artwork type parameter ignored. That is what
makes this a gap in the protocol rather than a shortcoming in any one project.

The whole wire surface

Everything the extension adds, in one place. All of it is optional and additive, and a
host that implements none of it behaves exactly as it does today.

Endpoint	Added	Purpose
/serverinfo	ArtworkKinds	which kinds this host can serve; absent means legacy
/serverinfo	ArtworkScaling	whether the host will scale on request
/serverinfo	ArtworkRequestLimit	concurrent artwork requests the host accepts
/applist	Artwork with Asset children	per app: kind, width, height, tag, and index for heroes
/appasset	artworkKind	which kind to return; absent means capsule
/appasset	maxWidth, maxHeight	optional, honoured only if the host advertises scaling

No existing parameter changes meaning and nothing is removed. One existing parameter
gains a use: AssetIdx, hardcoded to 0 by every client for years, selects between an
app's heroes.

Three things are deliberately absent because they are client decisions rather than
protocol: where a logo sits over a backdrop, what a client stores on disk, and what
happens when a user has already assigned artwork locally.

What this costs each side

Worth stating plainly, because the split is uneven. A host declares which kinds it
has, per app and per asset, returns the requested kind, sets a Content-Type that
matches the bytes, and returns 404 instead of a placeholder when an asset is absent.
Optionally it emits ETag, relaxes PNG-only validation, and scales on request. That
is nine obligations, and it is most of the work in this document.

A client asks only for assets an app has declared, sends an Accept header, respects
the host's concurrency limit, includes any requested size in its cache key, and
ignores kinds it does not recognize. That is roughly an afternoon per client.

So hosts carry the cost. The reason it is still worth their while is the first
paragraph of this document: the artwork being reshaped, re-encoded and conflated today
is artwork hosts already have and already lose. A host that adopts this stops guessing
which grid cell some client is using, stops pillarboxing square icons, and can serve a
JPEG as a JPEG.

Where the design comes from

Nothing here is invented. Four existing artwork systems were read before drafting, and
three of them agree with each other:

• Jellyfin, whose ImageType covers Primary, Art, Backdrop, Banner, Logo, Thumb,
  Disc, Box, Screenshot, Menu, Chapter, BoxRear and Profile, and which carries
  ImageType, Width, Height, ImageTag and BlurHash per image. Its image
  endpoint accepts maxWidth, maxHeight, fillWidth, fillHeight, format and
  quality, which is the model behind optional host side scaling here.
• Kodi, with poster, fanart, banner, clearlogo, clearart, landscape, keyart, discart,
  thumb and spine.
• TMDB, which returns posters, backdrops and logos, each declaring aspect_ratio,
  width and height.
• Steam, which instead mandates fixed sizes: Library Capsule at 600x900, Library
  Header at 920x430 and Library Hero at 3840x1240.

Every name in the first three is a role, not a shape, and Jellyfin and TMDB both
declare real dimensions rather than requiring them. Steam is the exception, and it is
also the only one of the four that ships the sole client reading its artwork. Moonlight
is in the other group, with five clients and four hosts and no coordination between
them, which is why this proposal names roles and declares dimensions. See
How other artwork systems model this.

The five kinds follow SteamGridDB's taxonomy, read from decky-steamgriddb, because
that is where the curated artwork and the tooling to apply it already exist.

The current state described below was read from source in all four host
implementations (Sunshine, Apollo, Wolf and Moonshine) and all three maintained
clients (moonlight-qt, moonlight-android and moonlight-ios). Every file and line
reference in this document points at that reading rather than at documentation.

Motivation

With one slot, a host has to guess the client's layout and reshape images to match.
Moonshine resizes and pillarboxes every asset onto a 600x801 canvas chosen to fit one
specific client's grid cell, and its own source comment notes that square desktop
icons get distorted as a result. See
Hosts compensating for client layout. A host
cannot avoid this, because nothing in the protocol lets it say what an image is, and
nothing lets a client say what it wants.

The same slot caps what clients can build. Every Moonlight client renders a grid of
portrait tiles, because a portrait tile is the only thing the protocol can express. TV
clients such as tvOS and Android TV normally place a wide backdrop behind the focused
item with a transparent logo over it. Compact list layouts and "recently played" rows
want a square icon. Detail views want a wide image filling the header. None of those
are possible today at any client's discretion, because the artwork does not exist to
ask for.

The artwork itself is not the obstacle. A large curated corpus for exactly these kinds
already exists at SteamGridDB, hosts already scan Steam, Heroic, Lutris and
freedesktop entries that contain several kinds, and host side tooling to apply artwork
already exists. What is missing is any way for GameStream to say which kind it is
serving.

How this surfaced

This proposal came out of building a SwiftUI tvOS client, aiming for a library that
behaves like Steam Big Picture: a backdrop behind the focused item, a title treatment
composited over it, and shelf rows that are not portrait tiles.

Each of those layouts turned out to be blocked at the protocol rather than in the
client. Backdrops could not be populated at all, since no wide artwork exists to
request and upscaling a portrait capsule to fill a 16:9 area is not viable. Titles had
to be drawn as text, because no transparent logo is available to composite. Shelf and
list rows had to stay portrait, because cropping a capsule to 16:9 throws away most of
the image.

Since the only artwork available is a single portrait capsule, the ceiling on any
Moonlight client's library UI is set by the protocol. That is what prompted reading
the wire format, which in turn produced the finding below: the artwork type parameter
has been sent by every client, and ignored by every host, all along.

One slot, six names

The single artwork slot is named differently in every implementation, because there
is no shared concept for what it holds:

Implementation	Name for the one artwork slot
Sunshine	image-path / ctx_t::image_path
Apollo	image_path (inherited from Sunshine)
Wolf	icon_png_path
Moonshine	boxart
Moonlight clients	"box art" / boxArt
SteamGridDB	grid_p ("Capsule")

Wolf's name refers to what this proposal calls a capsule, while icon here means
something else entirely. Moonshine has a config field documented as "whether to
resolve desktop entry icons into Moonshine boxart paths"
(app_scanner/desktop.rs:30-32), collapsing both concepts into a single sentence.
Standardizing the taxonomy settles this drift as a side effect.

Current state

The unused artwork type parameter

All three actively maintained clients issue an identical request, with AssetType and
AssetIdx hardcoded:

GET https://<host>:<HttpsPort>/appasset?uniqueid=<id>&appid=<id>&AssetType=2&AssetIdx=0

Client	Location
moonlight-qt	app/backend/nvhttp.cpp:392-404 (NvHTTP::getBoxArt)
moonlight-android	app/src/main/java/com/limelight/nvstream/http/NvHTTP.java:720-723 (getBoxArt)
moonlight-ios	Limelight/Network/HttpManager.m:298-305 (newAppAssetRequestWithAppId:)

What each host does

No host reads either parameter:

Host	appasset handler	Reads AssetType?	Content-Type	Missing artwork
Sunshine	src/nvhttp.cpp:1237-1248	No	hardcoded image/png	default placeholder image
Apollo	src/nvhttp.cpp:1495-1524	No	hardcoded image/png	default placeholder image
Wolf	src/moonlight-server/rest/endpoints.hpp:340-369	No	hardcoded image/png	HTTP 404
Moonshine	moonshine-core/src/webserver/mod.rs:451-539	No	hardcoded image/png	HTTP 400

Sunshine and Apollo both reduce to a single lookup:

auto app_image = proc::proc.get_app_image((int) util::from_view(get_arg(args, "appid")));
headers.emplace("Content-Type", "image/png");

Wolf reduces to one optional field on moonlight::App
(src/moonlight-protocol/moonlight/data-structures.hpp:17-22):

struct App {
  const std::string title;
  const std::string id;
  const bool support_hdr;
  std::optional<std::string> icon_png_path;
};

AssetType is therefore a pre-existing, already-transmitted, currently inert
extension point, which means this proposal does not have to invent a new request
channel.

No artwork metadata in applist

Sunshine emits three fields per app (src/nvhttp.cpp:973-981):

app.put("IsHdrSupported"s, video::active_hevc_mode >= 3 ? 1 : 0);
app.put("AppTitle"s, proc.name);
app.put("ID", proc.id);

Wolf builds its list from the same moonlight::App shape, and Moonshine's app_list
likewise carries no artwork fields, so a client cannot know what artwork exists
without speculatively requesting it.

PNG on the wire

Sunshine's validate_app_image_path (src/process.cpp:530-560) rejects any extension
other than .png and quietly substitutes DEFAULT_APP_IMAGE_PATH. Wolf's field is
named icon_png_path.

Moonshine is the interesting case. It decodes any format via
image::load_from_memory, then unconditionally re-encodes to PNG
(webserver/mod.rs:528, ImageFormat::Png), so its PNG constraint applies to the
wire rather than to the source file.

This matters because a 1920x620 hero as PNG is far heavier than the same image as
WebP or JPEG, and hero is the kind most likely to be large. Moonshine already runs a
full image pipeline inside the request path, so for that host emitting WebP is close
to a one line change.

Hosts compensating for client layout

All four hosts face the same constraint: with a single artwork kind, a host cannot
know what shape the client needs, so any source image that is not already portrait
has to be forced into a portrait slot.

Moonshine is cited here because it is the only implementation that wrote that
constraint down in code, which makes it the clearest available evidence, not because
it behaves worse than the others. Its appasset handler normalizes every image
server side (webserver/mod.rs:522-525):

// Moonlight displays box art at a fixed 200x267 pixel area using stretch mode.
// Icons that don't match this ratio (e.g. square desktop icons) get distorted.
// Fit the image into a 600x801 canvas (same ratio as 200:267), preserving aspect ratio, centered.
let asset = fit_to_boxart(asset);

fit_to_boxart (webserver/mod.rs:1055-1078) scales to fit BOXART_WIDTH = 600 by
BOXART_HEIGHT = 801 and centres the result on a transparent canvas. Given one
artwork kind this is the right call, since it prevents exactly the stretching the
comment describes. What it also makes visible is the cost.

A host is obliged to encode a specific client's layout: 200x267 is moonlight-qt's
image size (app/gui/AppView.qml:108-109), and other clients use other sizes, so the
host is compensating for a layout it cannot query and cannot negotiate. Two artwork
kinds are demonstrably in play already, because the comment itself distinguishes box
art from square desktop icons, which are the capsule and icon kinds of this proposal
sharing one slot. Non portrait sources also lose usable resolution: Moonshine's
scanners ingest Steam, Heroic, Lutris and freedesktop .desktop entries, and
resolve_icons (app_scanner/desktop.rs:30-32) routes square desktop icons into the
boxart field, where they end up pillarboxed onto a portrait canvas.

Given an icon kind, a host could serve a square icon as a square icon and leave the
layout decision to the client, which is the only place it can be made correctly.

Caching without invalidation

Clients cache one file per app, keyed by host UUID, and fetch only when the file is
absent:

Client	Cache path	Fetch condition
moonlight-qt	<boxart>/<uuid>/<appId>.png (boxartmanager.cpp, getFilePathForBoxArt)	file missing or zero-byte
moonlight-ios	<Caches>/<uuid>/<appid>.png (AppAssetManager.m:22-35)	file missing (AppAssetManager.m:49)

There is no ETag, no Last-Modified and no version field, so changing artwork on the
host does not propagate to clients.

Because the four hosts disagree about how to signal absence (a placeholder image, a
404, or a 400, per What each host does), moonlight-qt resorts
to inspecting image dimensions to detect placeholders
(app/gui/AppView.qml:91-106):

if (!model.isAppCollectorGame &&
    ((sourceSize.width === 130 && sourceSize.height === 180) || // GFE 2.0 placeholder image
     (sourceSize.width === 628 && sourceSize.height === 888) || // GFE 3.0 placeholder image
     (sourceSize.width === 200 && sourceSize.height === 266)))  // Our no_app_image.png
{
    isPlaceholder = true
}

This is a symptom of "artwork absent" not being expressible, and it is already a
latent bug. isAppCollectorGame is a GFE concept, so against Sunshine, Wolf, Apollo
and Moonshine the guard is false and the size check always runs; any genuine artwork
that happens to be exactly 628x888, 130x180 or 200x266 is silently classified as
missing. In games-on-whales/wolf#140 a user authoring test artwork for
icon_png_path produced it at exactly 628x888, one of the three sizes checked.

Adding four more kinds without addressing this would multiply the problem by five.

The four request cap

Both clients cap artwork fetches at four concurrent requests, explicitly to protect
the host. In moonlight-qt, boxartmanager.cpp sets
m_ThreadPool.setMaxThreadCount(4) with the comment "4 is a good balance between
fast loading for large app grids and not crushing GFE with tons of requests". In
moonlight-ios, AppAssetManager.m:20 sets MAX_REQUEST_COUNT = 4.

Any proposal that turns one request per app into five will be rejected on load
grounds. Two things address that: an app declares which assets it has, so clients do
not probe, and the host states the concurrency it will accept rather than having a
client's number written into the specification. See
Per app artwork assets and
Request concurrency.

Related reports

A search of all moonlight-stream repositories, LizardByte/Sunshine,
games-on-whales/wolf and hgaiser/moonshine found no existing issue, pull request
or discussion proposing additional artwork kinds. No repository in the
moonlight-stream organization has GitHub Discussions enabled and none has a
CONTRIBUTING.md, so this proposal is filed as an issue.

The trackers do contain the downstream symptoms. hgaiser/moonshine#40, "Fix missing
boxart for Steam games and manual applications", does two things at once: it reads
Steam's library_600x900.jpg with a fallback to library_capsule.jpg, and it
resolves XDG icon directory icons into the same boxart field for apps that lack
one. A 600x900 Steam library capsule and a square freedesktop icon are different
kinds, and the fix has to funnel both into one field because only one field exists.
Note also that library_600x900 is exactly SteamGridDB's canonical grid_p
dimension, and that it is a JPEG which Moonshine then re-encodes to PNG on every
request.

games-on-whales/wolf#140, "Game icons", is a user unable to work out what
icon_png_path expects, who supplied a 628x888 portrait image for a field named
"icon". The naming drift described in One slot, six names is
producing real confusion.

To be explicit, neither project is doing anything wrong and nothing above is a
criticism of either implementation. Both are working around a protocol that gives
them one slot and no vocabulary. The reason for citing them is that the workarounds
are converging independently, which is what a missing protocol primitive tends to
look like.

Prior art

How other artwork systems model this

Four systems already solve this problem, and three of them agree with each other.

System	Type names	Data carried per image	Clients
Jellyfin	Primary, Art, Backdrop, Banner, Logo, Thumb, Disc, Box, Screenshot, Menu, Chapter, BoxRear, Profile	ImageType, Width, Height, ImageTag, BlurHash, Size	many, third party
Kodi	poster, fanart, banner, clearlogo, clearart, landscape, keyart, discart, thumb, spine	none declared	many skins
TMDB	posters, backdrops, logos	aspect_ratio, width, height, file_path	many, third party
Steam	Library Capsule, Library Header, Library Hero, Library Logo	sizes mandated, not declared	one, first party

Every name in the first three columns is a role. Not one is a shape. Jellyfin says
Primary, not Portrait, and Backdrop, not Wide. Jellyfin and TMDB additionally
declare each image's real dimensions to the client, so a client can choose without
decoding anything first.

Steam is the exception that proves the rule. It mandates exact sizes:

Steam asset	Size
Library Capsule	600x900
Library Header	920x430
Library Hero	3840x1240

Steam can do that because Steam ships the only client that consumes them. Moonlight
is in the other group, with five clients and four hosts and no coordination between
them.

Jellyfin also lets the client ask for the size it wants, and its image endpoint
accepts width, height, maxWidth, maxHeight, fillWidth, fillHeight,
quality, format, blur, backgroundColor, tag and imageIndex. That is the
model behind Optional host side scaling.

The SteamGridDB taxonomy

From decky-steamgriddb/src/constants.ts (ASSET_TYPE, DIMENSIONS, MIMES):

SGDB key	Readable	Dimensions offered	MIME types
grid_p	Capsule	600x900, 342x482, 660x930, plus square options	PNG, JPEG, WebP
grid_l	Wide Capsule	460x215, 920x430	PNG, JPEG, WebP
hero	Hero	1920x620, 3840x1240, 1600x650	PNG, JPEG, WebP
logo	Logo	no fixed dimensions	PNG, WebP
icon	Icon	square, 8 to 1024	PNG, ICO

SteamGridDB also tags artwork with styles (alternate, white_logo, no_logo,
blurred and material for grids and heroes; official, white, black and
custom for logos). That is a concern for whatever populates a host, not for serving,
so it stays out of scope here.

Why kinds name roles, not shapes

The first draft of this proposal defined each kind by a nominal aspect ratio. That
does not survive the evidence, because the ratios in the wild do not agree:

Artifact	Ratio
SGDB grid_p 512x512 (option)	1:1.000
moonlight-qt no_app_image 200x266	1:1.330
moonlight-qt draw area 200x267	1:1.335
Moonshine canvas 600x801	1:1.335
SGDB grid_p 342x482 and 660x930	1:1.409
GFE 3.0 box art 628x888	1:1.414
Steam Library Capsule 600x900	1:1.500

There are two clusters. Authoring standards sit between 1:1.409 and 1:1.500, while
Moonlight's own rendering sits at 1:1.335. Mandating any single figure would put
either the entire existing GameStream artwork corpus or two current host
implementations out of spec, and would force hosts to reshape or reject most of what
SteamGridDB serves, including its square capsule options.

So a kind names a role and the host declares each asset's real dimensions.

Variation in dimensions does reach the client under this model. That is already true
today, and is currently handled by stretching: moonlight-qt sets its draw area to
200x267 regardless of the source image and declares no fillMode, so QML falls back
to Image.Stretch. Declaring dimensions does not introduce the variation, it tells a
client about variation it already receives, so it can letterbox or crop deliberately
and reserve layout space before an image arrives.

Proposal

Artwork kinds

A kind describes the artwork itself. The example uses are illustrative rather than
binding, because where an image is drawn is a client decision and the five Moonlight
clients want different answers.

Kind	What the artwork is	Example use	SGDB equivalent
capsule	Portrait, no fixed ratio, key art with the title legible. This is today's box art.	grid tile	grid_p
wide-capsule	Landscape, roughly twice as wide as tall, focused on branding	list rows, horizontal grids	grid_l
hero	Much wider than tall, cinematic, usually free of title text, meant to sit behind other elements	detail header, focused item backdrop	hero
logo	Transparent title treatment with no background	drawn over a hero	logo
icon	Square. For apps that are not games it is often the only artwork that exists.	compact lists, shortcuts	icon

capsule must behave identically to the current AssetType=2&AssetIdx=0 response. It
is named here only to complete the taxonomy.

Capability negotiation

A host implementing this proposal adds up to three elements to /serverinfo:

<root status_code="200">
  ...
  <ArtworkKinds>capsule,wide-capsule,hero,logo,icon</ArtworkKinds>
  <ArtworkScaling>1</ArtworkScaling>
  <ArtworkRequestLimit>4</ArtworkRequestLimit>
</root>

If ArtworkKinds is absent the host is legacy, and clients must not send
artworkKind requests. Clients must ignore kinds they do not recognize, so the list
can grow later.

That element carries real weight rather than being a convenience. A legacy host given
a hero request silently returns the portrait capsule (see
What each host does), which a client would then draw into a
wide slot. Since the response carries no indication of which kind it satisfies,
negotiation is the only way to avoid that.

Per app artwork assets

Each app lists the assets it actually has, so a client never probes for artwork that
does not exist:

<App>
  <IsHdrSupported>1</IsHdrSupported>
  <AppTitle>Hollow Knight</AppTitle>
  <ID>881448767</ID>
  <Artwork>
    <Asset kind="capsule" width="600" height="900" tag="7f3a91c2"/>
    <Asset kind="logo" width="1280" height="512" tag="1b40de77"/>
    <Asset kind="hero" width="3840" height="1240" tag="a0c81e35" index="0"/>
    <Asset kind="hero" width="1920" height="620" tag="55f2ab19" index="1"/>
  </Artwork>
</App>

width and height are the dimensions of the stored asset, before any scaling the
host may apply on request. tag is an opaque token that changes when that asset
changes; hosts may compute it from file modification time and size. An app with no
artwork omits the Artwork element entirely, and the common case of one capsule
costs the single request it costs today.

Only hero may appear more than once, since rotating or per scene backdrops are the
one case where several images of the same kind are genuinely useful. Multiple heroes
carry an index, and every other kind appears at most once.

This one list replaces the separate availability and revision fields an earlier draft
carried: availability is the presence of an Asset entry, and versioning is the
per asset tag.

Requesting an asset

The existing endpoint gains one parameter:

GET /appasset?uniqueid=<id>&appid=<id>&AssetType=2&AssetIdx=0&artworkKind=hero

AssetType=2 is kept verbatim so the request stays valid against legacy hosts and no
client needs a second code path. An absent artworkKind means capsule, matching
today's behaviour exactly. An unknown artworkKind on a supporting host must produce
HTTP 400, never a substituted image of some other kind.

AssetIdx finally has a meaning: it selects among an app's heroes, matching the
index from the asset list. For every other kind it must remain 0.

Optional host side scaling

A host that advertises ArtworkScaling accepts a requested bound and returns the
asset scaled to fit it:

GET /appasset?uniqueid=<id>&appid=<id>&AssetType=2&AssetIdx=0&artworkKind=hero&maxWidth=1280

Aspect ratio is preserved and assets are never scaled up. A host that does not
advertise scaling ignores these parameters and returns the stored asset, which is what
Sunshine and Wolf do today by streaming a file straight off disk. Moonshine already
decodes and re-encodes on every request, so for it this is nearly free.

Clients caching a scaled response must include the requested bound in their cache key,
since the same tag describes different bytes at different sizes.

Content types

Hosts must set a Content-Type reflecting the bytes they actually serve, replacing
the hardcoded image/png described in What each host does.
Clients should send an Accept header:

Accept: image/webp,image/png,image/jpeg

Permitted types per kind follow the table in
The SteamGridDB taxonomy. Hosts should relax PNG-only
validation to match, which is where the bandwidth saving for heroes comes from.

Absence semantics

A supporting host must return HTTP 404 when the requested asset does not exist, and
must not substitute a placeholder image.

This settles a genuine three way split. The four hosts currently signal absence as a
placeholder image (Sunshine and Apollo), a 404 (Wolf), or a 400 (Moonshine). Wolf
already matches this rule. Moonshine's 400 is close, though 404 is the more precise
signal, since a missing asset is not a malformed request. Sunshine and Apollo need a
behaviour change, but only on the path that carries artworkKind, so their existing
placeholder behaviour stays intact for older clients.

Once a host advertises support, clients can retire the dimension inspection described
in Caching without invalidation.

Caching and revalidation

Hosts should emit ETag and Last-Modified on /appasset responses, and should
honour If-None-Match and If-Modified-Since with 304 Not Modified.

An asset's tag in the asset list changes whenever that asset changes. That is the
whole of its meaning; what a client stores, evicts or re-fetches on the strength of it
is the client's own business.

Request concurrency

A host may advertise ArtworkRequestLimit, the number of concurrent artwork requests
it will accept. Clients should not exceed it. If the element is absent, clients should
assume 4, which is what moonlight-qt and moonlight-ios already use.

An earlier draft required clients to cap requests at four. That number belonged to two
specific clients rather than to the protocol, and the constraint is the host's to
describe, so it now travels in the direction of the party being protected.

[ReenigneArcher]

Depends on LizardByte/roadmap#153 and LizardByte/roadmap#87

[jorys-paulin]

I actually had the same kind of idea and even investigated how the original GameStream server worked, but I couldn't determine for sure the logic used by the original software. GeForce Experience's server being offline probably doesn't help analyze how the software is supposed to run, and my reverse engineering efforts with no experience didn't yeild any results.
Your message is quite long detailed and thought out (my printer tells me it would take fourteen pages to print!), but I feel it's over-engineered for something that's a small part of the GameStream protocol. I think we could get away with just using other AssetTypes and letting the client opt-in if and how they want. All the concerns of scaling, caching, erros, content types and sizes are primordial for a hosted public service, but I don't think it's needed for a self-hosted setup.

[evertonstz]

Fair on the length 😄. Fourteen pages was the wrong opening move for what it is.

You are also right about most of what should go. Host side scaling, ETag and conditional GET, and the advertised concurrency limit are all things a public image service needs and a LAN host does not. Cutting them takes the host side from nine obligations down to about three.

The one piece I do not think survives is the opt-in itself. Sunshine, Apollo, Wolf and Moonshine all read only appid from /appasset and ignore AssetType completely, so a client asking for AssetType=5 today gets the box art back with a 200. It cannot tell whether that is a hero from a new host or a capsule from an old one, so probing quietly puts the wrong image in the wrong slot. Something still has to declare what exists, though it can be one field in /applist instead of the three I wrote.

That gets harder rather than easier at the scale of LizardByte/roadmap#153, since GameDB and ThemerrDB bring screenshots, video references and theme songs alongside cover art. Probing a dozen integers per app is worse than reading one list.

[jorys-paulin]

Heh, honestly clients could just take the cover as "fallback", maybe blurring it to make it look nice-er.
The GameStream server also returned the cover when I tried other asset types, but can't confirm that's intended behavior.

[hgaiser]

Haha yeah the length of the first post kinda scared me off from replying initially. Small disclaimer: I did not read it entirely still so I hope this was not covered in your post :p.

My stance from Moonshine's perspective has always been that I feel like a guest using someone else's API. Being mentioned here as "one of the implementations" feels both weird and cool at the same time 🤷

That said though, I do feel that the interface of Moonlight feels outdated (no offense intended). If there would be an initiative to make it more "Steam Big Picture"-like I would support that. Part of that would be to retrieve more media, ie hero + box cover + others.

I do wonder if retrieving that media should be the responsibility of the server or the client. In the current API the full responsibility is with the server, but the client has better knowledge over what it needs and in what shape. I'd hope there's some game DB, where we can send an ID from that DB to the client and the client is responsible for retrieving media.

Obviously there's a big flaw with that approach. If there's a game which has no ID in that DB you would need to set some media manually on each client.

This is not a complete review, more of a random thought that I wanted to share :)

[evertonstz]

No need for the disclaimer, that's fair and it's on me. I will probably edit the original post to be smaller so its more bitable first 😄

The guest feeling is mutual from the other side. I hit the same wall building a SwiftUI tvOS client (https://github.com/evertonstz/moonlight-tvos), I had lots of ideas for the library screen, almost none of them possible, because the API stops at one portrait image, only genuinelly new thing I was able to pull was a "all apps" tab, and IMO it already worked wonders for the usability. So yes to the Big Picture direction.

On whether that's the server's job or the client's, you're right that the client knows what shape it needs, but I'd be careful about moving the whole job over. The missing ID case isn't rare in Moonshine's case specifically, since desktop entries and Lutris or Heroic apps often have none. Moving to clients would also require internet access, that is a problem for LAN only clients. Plus API credentials, since art sources like IGDB returns 401 without them, so every client would be storing and shipping keys. Often the host already has better artwork than a lookup would give you, which your own scanners prove by reading Steam library files and .desktop icons.

Jellyfin for example does both. ProviderIds sits right next to ImageTags, so a client can enrich from an external database while the server still serves whatever it has. That seems like the right shape here too.

[ReenigneArcher]

I don't think anyone really wants to read this LLM slop whether here in feature requests, github issues, or where ever. I think it's much better to describe your idea in your own words. Leave the technical nonsense for the implementation.

[evertonstz]

Fair enough. I think I should have disclaimed the text was translated by an LLM model. To be fair, I proofreaded it and I think its pretty faithfull to my original Portuguese research I was doing for my personal client+host implementation. What I kinda disagree is leaving the tech part out, if the interest is creating a contract for a protocol everyone will eventually reach for, at least how I have ever done this is nailing an RFC out so everyone can work in their own implementation when they have time. That's why it was bought here, btw, an ack from the biggest server is a greenlight for smaller hosts and clients to start working over it before your own implementation start.

If this kind of discussion should not be done here or this type of development is not how you do things, say the word and I can close it asap.

[hgaiser]

Jellyfin for example does both. ProviderIds sits right next to ImageTags, so a client can enrich from an external database while the server still serves whatever it has. That seems like the right shape here too.

I think that's a decent approach. I do see that client-only has major drawbacks.

I don't think anyone really wants to read this LLM slop whether here in feature requests, github issues, or where ever. I think it's much better to describe your idea in your own words. Leave the technical nonsense for the implementation.

I think you're being a bit harsh. The intention is good and there is a case to be made for improving how media is handled. To me your reply discourages contributions. It was already mentioned and agreed that the first post was perhaps a bit lengthy, let's move past that I'd say.

[jorys-paulin]

GeForce Experience has its own CMS system and database, where each supported game has a CmsId in addition to an app's per-computer Id, this could be re-used in additon to something like CmsSource as an optional way to link an app to an external source, though re-using this specific fields means only one source can be linked to per app.