Earth Observation has a data problem. Not a shortage of data — quite the opposite. The exponential growth of EO missions has created a glut of imagery, telemetry, and sensor data that current ground infrastructure was never designed to handle at scale. Satellites acquire; ground stations receive; centralised processing centres store and serve. The pipeline is linear, sequential, and — at the edges where it matters most — fragile.
At CloudFerro, we've been working with ESA's Φ-lab to ask a different question: what if the data pipeline didn't have to be linear at all?
The Problem with Centralised Pipelines
The dominant model for EO data handling today is architecturally straightforward: a satellite downlinks its data to one of a limited number of ground stations, which forwards it to centralised processing and archive infrastructure. It works. But it carries costs that compound as mission density increases.
Bandwidth bottlenecks arise when high-resolution imagery converges on a small number of ground station receivers. During disaster events — when satellite data is most urgently needed — this creates a cruel irony: the higher the demand, the more the pipeline strains.
Latency is structural. Near-real-time (NRT) delivery windows of two to four hours are standard, not exceptional. For time-critical applications — flood mapping, wildfire monitoring, humanitarian response — that latency has real consequences.
Single points of failure are everywhere. A ground station outage, a network interruption, a lost downlink window — any of these can delay or lose data that cannot be recaptured.
And as satellite constellations grow, these problems scale with them. More satellites means more data, more competition for ground station access, and more surface area for failure.
The question ESA's Φ-lab programme asked us to explore was whether distributed, peer-to-peer storage technologies — the same class of technologies that transformed how the internet handles large-scale data distribution — could be applied to this domain.
The EDFS Approach
The ESA Distributed File System (EDFS) is a software prototype developed under the Disruptive Technologies for Decentralized Storage and Provenance of EO Data programme (Theme 2), funded by ESA under Contract No. 4000148100/25/I-NS. It is a joint effort by CloudFerro as Prime Contractor and DuoBit as Sub-Contractor.
At its core, EDFS is a private, distributed storage network in which each node — whether a satellite or a ground station — participates actively in data propagation. Rather than a satellite waiting for a downlink window to transmit data to a single receiver, it pushes data into the network, from which any authorised node can retrieve it. The network handles replication, integrity, and routing automatically.
The technology stack is built on IPFS — the InterPlanetary File System — a well-established open-source protocol originally developed for terrestrial peer-to-peer content distribution, now repurposed for orbital constraints.
Why IPFS?
After a systematic evaluation of the distributed storage landscape — covering BitTorrent, Storj, Sia, Arweave, Filecoin, Hypercore, Holochain, and Ethereum Swarm — IPFS emerged as the most viable foundation for this application. Its MIT licence, active development community, and content-addressed architecture make it both practically deployable and technically suited to EO data characteristics.
IPFS stores data as Merkle-DAGs, where every block carries a Content Identifier (CID) derived from its cryptographic hash. This design provides deduplication and integrity by construction: any corruption of data yields a different CID, making tampering or bit rot immediately detectable. For EO data — where provenance and integrity are mission-critical — this is not a convenience feature. It is a fundamental property.
Data transfer runs over libp2p using Bitswap, a wantlist-driven block exchange protocol that requests and delivers content-addressed blocks between peers. Discovery, encryption, and NAT traversal are handled at the transport layer, meaning nodes can exchange data across heterogeneous network topologies — including the intermittent, bandwidth-constrained links characteristic of satellite communication.
IPFS Cluster sits above the data layer as the control plane for persistence and placement. It maintains a global pinset across all nodes, enforces replication targets, and keeps the distributed state consistent even over intermittent links — using either Raft or CRDTs depending on consistency requirements. For EDFS, this means the system continuously reconciles desired versus actual replication state across the constellation, autonomously recovering from node outages, connectivity losses, and orbital gaps.
One component is entirely custom-built: the EDFS Name Server, a MQTT-based service that provides mapping between IPFS Content Identifiers and human-readable file names. Standard IPFS operates purely on CIDs; the name server adds a practical layer that allows the system to be used with conventional file naming conventions, and provides a clean abstraction boundary that allows the storage engine to be swapped and compared during experiments.
Three Scenarios Where This Changes the Equation
The EDFS design is grounded in three operational use cases drawn from real EO mission requirements.
Use Case 1: Rapid Disaster Response
A satellite passes over a major disaster event — a flood, an earthquake, a wildfire. Within seconds of acquisition, the imagery is chunked and injected into the EDFS network. Both satellite nodes and ground-station nodes participate in propagation. As soon as any chunk reaches a ground node, it is forwarded over the fastest available terrestrial links to other ground nodes — distributing the bandwidth load across the entire network rather than concentrating it on a single downlink.
The target is a measurable reduction in time-to-availability relative to the standard NRT baseline of two to four hours. In a centralised architecture, that window is bounded by when the satellite next passes over a ground station. In a distributed architecture, it is bounded by how quickly the network can propagate chunks between nodes already in range — a substantially shorter constraint.
Use Case 2: Continuous LOS Relay for LEO Satellites
A satellite in low Earth orbit has limited ground communication windows — a narrow pass, a spinning attitude, an intermittent link. Under current architectures, data acquired between passes simply waits. Under EDFS, the satellite uses any peer in line-of-sight to forward chunks onward: satellite-to-satellite, then satellite-to-ground, relayed across the constellation until it reaches a ground node.
The same mechanism applies in reverse: corrective files, mission commands, or updated parameters can be uploaded into the EDFS network and delivered to the target satellite the next time it is in line-of-sight with any peer that holds the data — not necessarily a ground station.
This use case demonstrates a target of ≥95% delivery success rate under degraded connectivity — conditions that would cause conventional downlink to fail entirely.
Use Case 3: Situation-Aware Data Routing
Small satellites carry constrained hardware: limited disk, CPU, RAM, and — critically — power. Standard IPFS, designed for terrestrial infrastructure, is resource-heavy by default. EDFS introduces energy and resource awareness into routing decisions.
Using orbital parameters to predict line-of-sight windows, the system routes data preferentially through nodes that maximise bandwidth efficiency while minimising power expenditure. A custom priority scoring function — incorporating data urgency, link quality, and node resource state — drives routing decisions dynamically. The indicative target for energy consumption reduction relative to baseline is ≥20%.
This is not theoretical optimisation. It reflects a practical reality of small satellite operations: a node that exhausts its power budget cannot participate in the network. Energy-aware routing is a prerequisite for reliable operation in constrained orbital environments.
The Simulation Framework
Validating a distributed storage system for space applications cannot be done on real hardware in the early stages of research. EDFS uses a Kubernetes-based simulation environment in which each satellite is represented as a pod — a containerised compute unit with defined resource limits, orbital parameters, and connectivity profiles.
The framework is built for programmable, repeatable experimentation. Key components include:
- World Controller — manages each node's environment: throttles bandwidth, injects latency, cuts connections to simulate LOS interruptions or hardware failures.
- Experiment Executor — orchestrates scenario execution according to a declarative experiment definition, driving the World Controller and simulation agents.
- Simulation Agent — models satellite behaviour: generating files, injecting data into EDFS, requesting retrievals.
- Metrics Collector and Grafana Dashboards — capture per-node and system-wide telemetry in real time: network usage, CPU and memory load, disk I/O, replication success times, and latency.
- Chaos Mesh — fault injection engine, implementing realistic connectivity failures driven by orbital movement parameters.
Each experiment is software-defined in a YAML configuration specifying satellite layout, orbital parameters, hardware limits, connectivity schedules, and planned fault events. Experiments are re-runnable with parameter sweeps across node count, file size, connectivity quality, and acquisition timing — producing trend analyses rather than single-point measurements.
The experiment definitions and sample configurations are published openly at github.com/duobitx/yass-experiments, in line with ESA's open-source commitment.
The simulation infrastructure itself runs on CloudFerro's Kubernetes cluster, with persistent volumes provided via OpenStack Cinder — a deployment environment directly relevant to operational EO infrastructure.
What We Are Measuring
The experimental evaluation framework targets four KPI groups:
Availability and replication — replication success time as a function of node count; total replication completion time across the constellation under normal and degraded conditions.
Latency — end-to-end time from data acquisition to availability on a ground node, measured against the standard NRT baseline.
Bandwidth utilisation — effective aggregated use of multiple satellite and ground links versus the single-node downlink baseline.
Energy and resource efficiency — CPU, disk I/O, and network usage as proxies for power consumption on resource-constrained nodes; compression gains and adaptive behaviour under hardware limits.
A note on honesty: power consumption estimates are derived from computational proxies, not measurements on real flight hardware. The simulated environment provides a controlled basis for comparing architectures and informing design decisions — not a substitute for in-orbit validation. We document this clearly, because reproducibility and intellectual honesty are prerequisites for scientific credibility.
Why This Matters Beyond the Prototype
The implications of EDFS extend beyond the specific use cases we are testing.
Data provenance — the ability to trace the origin, movement, and processing history of a piece of data — is increasingly critical in EO applications. Content-addressed storage, as implemented in EDFS, provides provenance by construction: the CID of a data block is a cryptographic commitment to its content and history. Any transformation of the data produces a new CID, creating an immutable audit trail without additional instrumentation.
Constellation scalability — as constellations grow from tens to hundreds of satellites, centralised ground architectures face quadratic scaling pressure. Distributed architectures, by contrast, gain capability as nodes are added: more nodes means more propagation paths, more storage, and more resilience. The network becomes stronger as it grows.
Resilience in contested environments — distributed storage has no single point of failure by design. Data that has been replicated across multiple nodes survives the loss of any individual node — including ground stations, which are themselves vulnerable infrastructure.
The technology we are building on is not experimental in the terrestrial context. IPFS underpins real-world content distribution systems at scale. What EDFS explores is the translation of those capabilities to the orbital environment — with its specific constraints of intermittent connectivity, limited energy budgets, and the physical realities of orbital mechanics.
Where We Are Now — Experiments Complete, Results In
After months of development and simulation runs, the EDFS experimentation phase is complete. All five use cases have been executed across the full parameter sweep — constellation sizes from 1 to 200 satellites, replication factors 1 through 5, and fault injection ranging from intermittent connectivity to complete satellite destruction.
The headline finding is unambiguous: when a satellite captures critical imagery far from any ground station and is then completely destroyed before establishing a downlink window, EDFS recovers and delivers the file through surviving relay peers — achieving 100% delivery across the large majority of viable configurations. The point-to-point baseline (TUS) delivered nothing in every single one of 15 such variants. This is not a marginal improvement; it is a structural difference.
At scale, the results are counter-intuitive. Larger constellations deliver faster, not slower. In Use Case 1 (Rapid Disaster Response), EDFS delivered in 122 seconds at 100 satellites — compared to roughly 800 seconds at small constellations — because a denser relay pool means some node is almost always already near a ground station. TUS timed out entirely at the same constellation size.
The cost is real and quantifiable. EDFS uses roughly 13–15 times more memory per node (~180–260 MiB vs ~13–15 MiB for TUS) and substantially more CPU and network bandwidth — the price of content-addressing and bitswap relay. For small, fault-free, direct-downlink deployments, that overhead buys nothing. For large, fault-prone constellations where producers may fail before reaching ground contact, it buys everything.
With the Final Review scheduled for 15 June at ESRIN and a live demonstration at the Φnnovation Summit 2026 on 25 June, the team is finalising the full results package.