The Stellar Development Foundation has achieved a 9x speedup in Horizon reingestion by leveraging the Composable Data Platform (CDP) with precomputed ledger metadata, reducing full history reingestion from approximately 66 days to 7 days.
This blog post details how the Stellar Development Foundation optimized Horizon reingestion performance using CDP components. Reingestion is the process of fetching and reprocessing historical Stellar ledger data, which is essential for setting up new servers, auditing, and recovering from operational pauses. The previous architecture relied on Captive Core, which was time-consuming and resource-intensive. The new CDP-based architecture leverages precomputed ledger metadata stored in Google Cloud Storage, eliminating Captive Core startup time and ledger replay overhead. Benchmarking shows that CDP achieves superior parallelization (16 workers vs. 4 for Captive Core) and reduces full network history reingestion from an estimated 66 days to 7 days—a 9x improvement. The Horizon db reingest command now supports reingestion from the datastore with configurable parameters for bucket address and data schema.
