UC5 — General Failure (EDFS only)

Abstract

Twenty per cent of the constellation produces a small, finite batch of images each. The whole constellation — both the producing satellites and every ground station — is hit by a continuous stream of non-terminal hardware faults (no Destroy). We measure how long it takes for every produced image to reach at least one ground station, and check whether EDFS is eventually able to deliver them all in spite of the fault stream. EDFS only; TUS is out of scope here.

UC5 sits between UC2 (every sat produces one file, faults active) and UC4 (one terminal failure on one producer). It's the long-duration stress test of EDFS's eventual consistency under sustained but non-catastrophic degradation.

Relationship to UC2. UC5 is essentially the same experiment as UC2 — the same constellation, the same continuous stream of non-terminal hardware faults (Destroy is excluded in both), measuring the same time-to-ground KPI — only with a different parametrization: a sparser, time-distributed workload (only ~20% of satellites produce, each a small finite batch of images over time, instead of every satellite producing one file at t=0) and a stricter 100% completeness bar. The one practical consequence worth calling out is that UC5 is EDFS-only, so there is nothing to compare against: there is no point-to-point (TUS) baseline here, because the eventual whole-set convergence under sustained relay has no point-to-point analogue (TUS would simply lose files under the fault stream). UC5 is therefore an absolute characterization of EDFS across the RF axis rather than a head-to-head.

Success Condition

The run ends successfully once a single ground station holds every image produced in the run.

Detailed description

Out of the sat_count satellites in the constellation, exactly 20% are designated producers. Each producer's agent writes N=5 images spaced by IMAGE_INTERVAL (default 2m), then exits. The total number of distinct files in flight per run is therefore floor(0.2 * sat_count) * 5 — finite and countable, as required.

While the run is in progress, the operator stamps a per-fsNode random fault schedule on every fsNode (producers, relays and ground stations alike). Allowed fault types: NetworkBandwidthReduced, NetworkFailure, DiskFull, DiskFailure. Destroy is explicitly excluded — the whole point of UC5 is to study eventual delivery under transient degradation; a destroyed producer of a file with RF=1 (no copies yet) would be UC4's loss-mode and would muddy the metric.

The seven ESTRACK ground stations receive on all files (their agent does not exit on first RECEIVED — it keeps logging until the run ends).

The run ends when (a) every file has been received by at least one GS, or (b) maxDuration fires. Both outcomes are reportable.

Main goal

Show that EDFS eventually delivers every file under sustained non-catastrophic faults, and quantify "how long is eventual?":

• delivery_success_rate = (distinct files delivered to ≥1 GS) / files_produced, with files_produced = floor(0.2·sat_count)·5. Target ≥ 95% (at RF ≥ 3).
• Time-to-all-delivered when the run terminates by (a).
• The trade-off between RF and total energy / network spent: higher RF should improve delivery_success_rate at the cost of more cross-talk on inter-satellite links.

KPI:

• delivery_success_rate ≥ 95% at RF ≥ 3 for every tested sat_count.
• Time-to-all-delivered grows sub-linearly with the number of files in flight.

Parameters

Assumptions

• The inter-ground-station network is broadband and disturbance-free. Every pair of ground stations is treated as permanently linked, independent of orbital line-of-sight: the link is fixed at 10 Gbit/s with zero packet loss, and only its propagation delay scales with the great-circle distance between the two stations. Satellite links, by contrast, exist only while the two nodes are in line-of-sight and their bandwidth degrades with distance. The only event that can disturb a ground-station-to-ground-station link is a hardware failure injected as a HardwareEvent (e.g. NetworkFailure or NetworkBandwidthReduced) on one of the two stations.

Additional metrics

Beyond yass_file_delivery_seconds:

• delivery_success_rate — count(distinct file names delivered to ≥1 estrack GS) / total_files_produced, with the deterministic total_files_produced = floor(0.2·sat_count)·5. Source: events-csv/file_delivered.csv (numerator = distinct name where target starts with estrack). Do not use yass_file_received_total{is_target_gs="true"} — that label does not exist on the metric and it counts receive events, over-counting repeated GS receipts (> 1.0).
• "Eventual completion" plot: cumulative delivery_success_rate as a function of wall-clock time. We expect a smooth monotonic climb that asymptotes at or close to 1.0 for RF ≥ 3.
• yass_edfs_replica_completeness heatmap per CID — over a long stress-test run we expect "full replicas" to grow steadily as DHT pickups absorb each file.
• Per-fsNode resource cost over the long run: yass_container_cpu_millicores, yass_container_memory_bytes, yass_network_tx_bytes_total.
• Total network cost: sum(yass_network_tx_bytes_total) aggregated over the whole constellation. Headline trade-off metric for the RF sweep — the byte volume should grow roughly linearly with RF while the completion time should drop.
• Fault-overlay correlation: yass_hardware_event_active as a state timeline so we can visually correlate fault windows with stalled per-file progress.