Skip to content

DigitalArsenal/space-data-network

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

451 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Space Data Network (SDN)

A decentralized peer-to-peer network for exchanging standardized space data using Space Data Standards, built on IPFS and libp2p.

CI Go Version TypeScript License Built on IPFS


Mission

Enable decentralized, global collaboration on space situational awareness and space traffic management.

As space becomes increasingly congested with satellites, debris, and new actors, the need for transparent, real-time data sharing has never been greater. Space Data Network removes barriers to collaboration by:

  • Eliminating single points of failure - No central server that can go down or be blocked
  • Enabling permissionless participation - Anyone can join and contribute data
  • Ensuring data integrity - Cryptographic verification of all shared data
  • Reducing latency - Direct peer-to-peer data exchange without intermediaries
  • Promoting interoperability - Standardized formats everyone can use

Overview

Space Data Network enables real-time sharing of space situational awareness data between organizations, satellites, and ground stations. Built on IPFS/libp2p with FlatBuffers serialization, SDN provides:

  • Standardized Data Exchange - All Space Data Standards schemas supported
  • Decentralized Architecture - No central server required
  • Real-time PubSub - Subscribe to data streams by type (OMM, CDM, EPM, etc.)
  • Cryptographic Verification - Ed25519 signatures on all data
  • Cross-Platform - Server (Go), Browser (TypeScript), Desktop, Edge Relay support

Current UI Surfaces

  • / is the browser-first SDN UI.
  • /webui is the upstream-style IPFS WebUI.
  • /admin is reserved for admin and auth flows.

The SDN browser path uses sdn-js plus the generic async capability surfaces from space-data-module-sdk and the existing hd-wallet-wasm and hd-wallet-ui identity stack. It does not use a helper service or the old broker/bootstrap browser flow.

Marketplace Direction

Marketplace discovery is driven by the canonical PLG manifest from spacedatastandards.org. There is exactly one signed listing per PLUGIN_ID + VERSION, and schema changes must land upstream before they are consumed here.


Quick Start

Install the Server

# Download latest release
curl -sSL https://digitalarsenal.github.io/space-data-network//install.sh | bash

# Or build from source
git clone https://github.com/DigitalArsenal/space-data-network.git
cd space-data-network
npm run install:wasmedge
npm run server:build

Source builds of the Go server host standalone WASM artifacts through WasmEdge, so space-data-network installs and wires the native WasmEdge SDK as part of its own build and test entrypoints.

Build the JavaScript SDK (Source)

cd space-data-network/sdn-js
npm install
npm run build

Run a Full Node

# Initialize configuration
./spacedatanetwork init

# Start the node
./spacedatanetwork daemon

Browser Usage

import { SDNNode, SchemaRegistry } from './path/to/sdn-js/dist/esm/index.js';

// Create and start a node
const node = new SDNNode();
await node.start();

// Subscribe to Orbital Mean-Elements Messages
node.subscribe('OMM', (data, peerId) => {
  console.log(`Received OMM from ${peerId}:`, data);
});

// Publish data
const ommData = { /* your OMM data */ };
await node.publish('OMM', ommData);

CI and Local Checks

  • Local CI (same checks as GitHub CI):
./scripts/ci-local.sh quick
  • Full local CI (includes encryption tests):
./scripts/ci-local.sh full
  • Pushes run local CI automatically via .husky/pre-push. To bypass intentionally:
SKIP_LOCAL_CI=1 git push

Architecture

+-------------------------------------------------------------------+
|                      Space Data Network                           |
+-------------------------------------------------------------------+
|                                                                   |
|   +-----------+      +-----------+      +-----------+             |
|   | Full Node |<---->| Full Node |<---->| Full Node |             |
|   |   (Go)    |      |   (Go)    |      |   (Go)    |             |
|   +-----+-----+      +-----+-----+      +-----+-----+             |
|         |                  |                  |                   |
|         |     DHT + PubSub |                  |                   |
|         |                  |                  |                   |
|   +-----+-----+      +-----+-----+      +-----+-----+             |
|   |Edge Relay |      |Edge Relay |      |Edge Relay |             |
|   |   (Go)    |      |   (Go)    |      |   (Go)    |             |
|   +-----+-----+      +-----+-----+      +-----+-----+             |
|         |                  |                  |                   |
|         |  Circuit Relay   |                  |                   |
|         |                  |                  |                   |
|   +-----+-----+      +-----+-----+      +-----+-----+             |
|   |  Browser  |      |  Desktop  |      |  Browser  |             |
|   |   (JS)    |      |   (App)   |      |   (JS)    |             |
|   +-----------+      +-----------+      +-----------+             |
|                                                                   |
+-------------------------------------------------------------------+

Downloads

Full Node

Platform Architecture Download
Linux amd64 spacedatanetwork-linux-amd64
Linux arm64 spacedatanetwork-linux-arm64
macOS amd64 spacedatanetwork-darwin-amd64
macOS arm64 spacedatanetwork-darwin-arm64
Windows amd64 spacedatanetwork-windows-amd64.exe

Edge Relay

Platform Architecture Download
Linux amd64 spacedatanetwork-edge-linux-amd64
Linux arm64 spacedatanetwork-edge-linux-arm64

Desktop Application

Platform Download
macOS SpaceDataNetwork.dmg
Windows SpaceDataNetwork-Setup.exe
Linux SpaceDataNetwork.AppImage

JavaScript SDK

cd sdn-js
npm install
npm run build

Identity & HD Key Derivation

Every SDN node derives its cryptographic identity from a BIP-39 mnemonic using SLIP-10 hierarchical deterministic key derivation with the standard BIP-44 Bitcoin derivation path (coin type 0).

BIP-39 Mnemonic → PBKDF2 → 512-bit Seed → SLIP-10 Master Key
    ├── m/44'/0'/0'/0'/0'  →  Ed25519 Signing Key (also libp2p PeerID)
    └── m/44'/0'/0'/1'/0'  →  X25519 Encryption Key

Why BIP-44?

SDN reuses the standard BIP-44 HD wallet path structure with Bitcoin's coin type 0:

  • Wallet-native identity. The BIP-44 path structure lets users derive SDN signing and encryption keys from the same mnemonic they use for cryptocurrency wallets. One seed, many independent key trees.
  • Multi-account. The account' segment enables one mnemonic to manage multiple SDN identities (operator, sensor, analytics service), each with independent key pairs.

The xpub (extended public key) serves as the master network identity. Anyone with the xpub can derive the node's public signing and encryption keys without access to private key material.

Managed Node Identity

  • The node mnemonic under data/<node>/keys/mnemonic is the single root secret for a managed node.
  • The SDN node identity and the managed IPFS identity are both derived from that root instead of being stored as separate long-lived key sources.
  • /api/directory/nodes and /api/directory/users expose the local FlatSQL-backed EPM directory index used by the SDN dashboard and shared runtime adapters.

Built on IPFS

Space Data Network is built on the InterPlanetary File System (IPFS) stack:

Technology Purpose
libp2p Modular P2P networking
Kademlia DHT Distributed peer discovery
GossipSub Publish/subscribe messaging
Circuit Relay NAT traversal
Kubo IPFS reference implementation

SDN extends IPFS with space-specific optimizations:

  • FlatBuffers for zero-copy performance
  • Schema-validated data (Space Data Standards + OrbPro control schemas)
  • Topic-per-schema PubSub
  • SQLite storage with FlatBuffer virtual tables

Components

Component Description Language
sdn-server Full node and edge relay server Go
sdn-js Browser/Node.js SDK TypeScript
desktop Desktop application TypeScript
schemas FlatBuffer schema definitions FlatBuffers
plugin-demo Plugin development guide, WASM API reference, integration tests C / JS
kubo IPFS reference implementation Go

Canonical module-delivery records live in spacedatastandards.org, and SDN consumes the shared runtime helpers from space-data-module-sdk. SDN does not ship a separate module-spec tree anymore.

The live licensing/module-delivery flow is carried by the SDS families LCH, LPF, LGR, LWK, LMR, PLG, and REC.

Browser and Node apps use public sdn-js package surfaces for marketplace purchase and encrypted module delivery:

  • @spacedatanetwork/sdn-js for SDNNode, requestModuleGrant, fetchEncryptedModuleBundle, requestEncryptedModuleBundle, and MODULE_DELIVERY_PROTOCOL_ID
  • @spacedatanetwork/sdn-js/ui for PLG listing discovery, grant content-key unwrap, encrypted bundle decrypt, SDK browser harness load, and module invoke
  • @spacedatanetwork/sdn-js/storefront for createStorefrontClient, purchase request types, payment enums, grant status, and purchase status

The documented third-party flow is in sdn-js/examples/purchase-encrypted-wasm-delivery.ts: discover a PLG listing, purchase it through the storefront client, request the encrypted WASM bundle through SDNNode, unwrap and decrypt locally, load with the SDK browser harness, and invoke the module.

Run the focused module-delivery compatibility checks:

npm run test:module-delivery

Server Packages

Package Description
internal/sds FlatBuffer builders for all SDS schemas with fluent API
internal/vcard EPM to vCard/QR code bidirectional conversion
internal/pubsub PubSub topics and PNM-based tip/queue system
internal/storage SQLite storage with FlatBuffer support

Supported Standards

SDN supports all Space Data Standards:

Category Standards
Orbit OMM, OEM, OCM, OSM
Conjunction CDM, CSM
Tracking TDM, RFM
Catalog CAT, SIT
Entity EPM, PNM
Maneuver MET, MPE
Propagation HYP, EME, EOO, EOP
Reference LCC, LDM, CRM, CTR
Other ATM, BOV, IDM, PLD, PRG, REC, ROC, SCM, TIM, VCM

Use Cases

Conjunction Assessment

node.subscribe('CDM', (cdm, peerId) => {
  if (cdm.COLLISION_PROBABILITY > 1e-4) {
    alertOperator(cdm);
  }
});

Orbital Data Exchange

  • OMM - Mean orbital elements (TLE-equivalent)
  • OEM - Precise ephemeris state vectors
  • OCM - Comprehensive orbit characterization

Coordination

  • MPE - Maneuver notifications
  • LDM/LCC - Launch coordination
  • ROC - Reentry predictions

Network Architecture

SDN uses a two-tier peer topology for maximum reach and reliability:

┌─────────────────────────────────────────────────────────────────┐
│                    FULL NODES (Open Internet)                    │
│                                                                  │
│    ┌──────────┐      ┌──────────┐      ┌──────────┐             │
│    │Full Node │◄────►│Full Node │◄────►│Full Node │             │
│    │  (Go)    │      │  (Go)    │      │  (Go)    │             │
│    └────┬─────┘      └────┬─────┘      └────┬─────┘             │
│         │                 │                 │                    │
│         │    DHT + GossipSub + Relay        │                    │
│         │                 │                 │                    │
├─────────┼─────────────────┼─────────────────┼────────────────────┤
│         ▼                 ▼                 ▼                    │
│                 LIGHT PEERS (Behind NAT/Firewall)                │
│                                                                  │
│    ┌──────────┐      ┌──────────┐      ┌──────────┐             │
│    │ Browser  │      │ Desktop  │      │Corporate │             │
│    │  (JS)    │      │  (App)   │      │   Node   │             │
│    └──────────┘      └──────────┘      └──────────┘             │
└─────────────────────────────────────────────────────────────────┘

Full Nodes

  • Run on servers with public IP addresses
  • Participate in DHT routing and peer discovery
  • Relay traffic for firewalled peers via Circuit Relay
  • Pin and store content for the network
  • Requirements: Public IP, ports 4001 (libp2p), 8080 (HTTP API)

Light Peers

  • Connect through relay nodes when behind NAT/firewalls
  • Can subscribe to data, publish messages, verify signatures
  • Cannot contribute to DHT routing
  • Includes: browsers, mobile apps, desktop apps, corporate networks

Run a Full Node

Help strengthen the network by running a full node:

./spacedatanetwork daemon --relay-enabled --announce-public

Content Addressing

All data on SDN is content-addressed using cryptographic hashes (CIDs):

Feature Description
Tamper-proof Hash changes if data is modified - tampering is immediately detectable
Permanent references CIDs never change - reference specific data versions forever
Deduplication Same data = same hash - network automatically deduplicates
Selective pinning Choose what to store locally - pin critical data for availability

Data Flow

SDN uses a layered data flow architecture: FlatBuffersFlatSQLPNMPLOG/PLHDSubscriptions

Publisher                                      Subscriber
    │                                               │
    │  1. Build FlatBuffer (OMM, CDM, etc.)         │
    │  2. POST /api/v1/data/publish/{schema}        │
    │     → Validate + store in FlatSQL (SQLite)    │
    │     → Compute SHA-256 CID                     │
    │     → Append PLOG entry (hash-chained log)    │
    │                                               │
    │  3. Broadcast PNM via GossipSub ──────────────│──→ Receive PNM
    │     (lightweight notification: CID + schema)  │
    │                                               │  4. Verify signature
    │  5. Publish PLHD (log head) ──────────────────│──→ Check tip/queue config
    │                                               │
    │                                               │  6. If autoFetch: fetch CID
    │  ◄────────────────────────────────────────────│     from publisher or any peer
    │     (SDS exchange protocol)                   │
    │                                               │  7. Store in local FlatSQL
    │                                               │  8. Fire onMessage(schema, data, peerId)

FlatSQL Storage

All data is stored in SQLite with per-schema tables: sds_{schema}(cid, data, peer_id, signature). Records are indexed by NORAD ID, epoch day, and entity ID for fast queries. Content is addressed by SHA-256 CID — same data always produces the same hash.

Publication Logs (PLOG/PLHD)

Each publisher maintains a per-schema hash-chained log for efficient incremental sync:

  • PLOG — Append-only log entries with sequence numbers, chain links, and Ed25519 signatures
  • PLHD — Lightweight log head announcements broadcast when the log advances
  • Subscribers compare HEAD_SEQUENCE against last_synced to determine the delta
  • Full chain verification: recompute hashes + verify signatures

PNM Tip/Queue System

Publish Notification Messages (PNM) decouple content storage from notification. Nodes configure auto-fetch, auto-pin, and TTL per-source AND per-schema:

Setting Description
Per-schema defaults E.g., always fetch CDM (conjunction data)
Per-source overrides E.g., trust data from partner organizations
Per-source+schema E.g., special handling for OMM from trusted source

Encrypted Messages

SDN supports end-to-end encryption for private data:

Mode Algorithm Use Case
ECIES X25519 + ChaCha20-Poly1305 Per-message encryption
SessionKey AES-256-GCM Bulk streaming
Hybrid Plaintext header + encrypted payload Routable encrypted data

Streaming Subscriptions

Three delivery modes: Single (on-demand), Streaming (real-time), Batch (periodic). Subscriptions support schema/peer filtering, rate limiting, and priority routing.

Browser Nodes (sdn-js)

The sdn-js SDK turns any browser or Node.js process into a full SDN peer. A browser with the same mnemonic as a server node has the same cryptographic identity — users ARE their HD wallet keys.

See docs/docs.html and plugin-demo/ for complete architecture documentation.

See sdn-server documentation for configuration details.


Data Marketplace

SDN includes an optional commercial layer for monetizing space data:

How It Works

  1. Provider publishes premium data product (high-precision ephemeris, analysis, etc.)
  2. Per-customer encryption - Data encrypted with each customer's public key (ECIES)
  3. Customer pays via credit card through integrated payment gateway
  4. Access granted - Customer receives and decrypts data with their private key

Features

Category Options
Data Products High-precision ephemeris, conjunction analysis, historical archives, real-time feeds
Plugin Marketplace Analysis algorithms, visualization tools, format converters, custom propagators
Payment Options Credit cards (Stripe), subscriptions, usage-based billing, enterprise invoicing

Technical Details

  • Encryption: ECIES with X25519 key exchange + AES-256-GCM
  • Payment Gateway: Stripe integration for credit card processing
  • Revenue Distribution: Automated splits between data providers and platform
  • Metering: Usage tracking for consumption-based billing

The marketplace operates on top of the free, open network. Core SSA data exchange remains free and open - the commercial layer is opt-in for premium products.


Plugin harness smoke test

Run an end-to-end check that validates loading a licensing plugin from a local workspace into SDN.

npm run plugin-harness -- /path/to/private-repo

You can also use the command with any plugin workspace path:

npm run plugin-harness -- /path/to/repo

Options:

  • --repo (or positional first arg): path to the plugin workspace
  • --admin-addr: admin endpoint used for verification (default 127.0.0.1:5010)
  • --artifact-dir: path to existing encrypted artifacts when --skip-build is set
  • --skip-build: use existing artifacts in the staging directory
  • --keep-workspace: keep temporary workspace for debugging
  • --derivation-secret: optional derivation secret override (64 hex chars)

This command is key-management agnostic on the CLI:

  • It derives the keypair internally for normal runs.
  • A fixed test public key is read from PLUGIN_KEY_SERVER_ARTIFACT_PUBLIC_KEY_HEX when set.
  • For --skip-build, it requires PLUGIN_KEY_SERVER_ARTIFACT_PUBLIC_KEY_HEX and --artifact-private-key-file <path>.

The command uses the standardized plugin task:

npm run build:key-server

It then copies/decrypts the generated encrypted artifact, boots SDN with a temporary plugin catalog, and verifies:

  • /api/v1/plugins/manifest reports the module-delivery plugin id as running
  • /api/v1/plugins/<plugin-id>/bundle returns 200 and non-empty WASM payload

Private repo setup for plugin harness tests

This harness runs against private repos as long as the repo is reachable and follows the plugin workspace contract.

  1. Clone/fetch private repo using your normal auth path (SSH key or token-based HTTPS).
  2. Confirm workspace layout includes:
    • package.json
    • scripts/build-plugin-release.js (or PLUGIN_HARNESS_BUILD_HELPER_SCRIPT override)
    • npm run build:key-server succeeds (or configure PLUGIN_HARNESS_BUILD_COMMAND)
  3. Export one of the artifact public key env vars used for staging:
    • PLUGIN_KEY_SERVER_ARTIFACT_PUBLIC_KEY_HEX (preferred)
    • For --skip-build, pass --artifact-private-key-file <path> pointing to the matching private key file
  4. Run:

npm run plugin-harness -- /path/to/private-plugin-repo

5. The harness validates the plugin lifecycle and plugin API endpoints in SDN.

Use `--skip-build` when reusing staged artifacts already in CI:

```bash
export PLUGIN_KEY_SERVER_ARTIFACT_PUBLIC_KEY_HEX=<public_hex>
npm run plugin-harness -- /path/to/private-plugin-repo --skip-build --artifact-dir /path/to/Build/plugin/licensing-server --artifact-private-key-file /secure/artifact-private-key.hex

If your private repo has a custom auth requirement, run the harness in that authenticated shell context so Git can access dependencies and source.

Development

Prerequisites

  • Go 1.21+
  • Node.js 18+
  • Emscripten (for WASM)

Build from Source

git clone https://github.com/DigitalArsenal/space-data-network.git
cd space-data-network

# Build server
cd sdn-server
go build -o spacedatanetwork ./cmd/spacedatanetwork
go build -tags edge -o spacedatanetwork-edge ./cmd/spacedatanetwork-edge

# Build JavaScript SDK
cd ../sdn-js
npm install
npm run build

Run Tests

# Go tests
cd sdn-server && go test ./...

# JavaScript tests
cd sdn-js && npm test

Local Admin Dev Wallet

npm run admin:dev, config/dev.yaml, and config/dev-docker.yaml all use the tracked local-only wallet in config/dev-wallet.env.

That file contains the mnemonic, xpub, and derivation path for the local dev admin identity. It is meant only for local development, and the production deploy script refuses to deploy if that xpub appears in a production config.

Suite Version Pinning

The suite-wide version contract lives in suite.versions.json.

That manifest is the canonical source for:

  • the suite release version
  • the pinned spacedatastandards.org version
  • the pinned hd-wallet-wasm and hd-wallet-ui versions
  • the pinned IPFS WebUI version
  • the current advertised SDN protocol flag

Generated readers are checked into:

When the manifest changes, regenerate those files with:

npm run generate:versions

And verify repo-wide consistency with:

npm run check:versions

Documentation

Full documentation is available at docs.digitalarsenal.github.io/space-data-network or locally at docs/docs.html.

To preview the docs locally, start a webserver from the docs/ directory:

cd docs && python3 -m http.server 8080

Then open http://localhost:8080.

Topics covered:

  • Getting Started & Quick Start
  • Full Node Setup & Configuration
  • Edge Relay Deployment
  • JavaScript SDK Reference
  • REST & WebSocket API
  • Schema Reference (all Space Data Standards)
  • Data Flow Architecture — FlatBuffers, FlatSQL, PNM, PLOG/PLHD, streaming, encryption
  • Wallet Identity — HD key derivation, TOFU binding, multi-account
  • Browser Nodes — sdn-js as a full network peer
  • Plugin System — WASM API, host functions, lifecycle

Links


Contributing

See CONTRIBUTING.md for guidelines.

License

MIT License - see LICENSE for details.


Built for the space community

About

No description, website, or topics provided.

Resources

Contributing

Stars

Watchers

Forks

Packages

 
 
 

Contributors