George Kudrayvtsev demonstrates how the Composable Data Platform (CDP) and Galexie enable efficient backfilling of Stellar RPC with historical ledger data in just 12 minutes—a task that would take over an hour using traditional captive core replay. The prototype required only 80 lines of Go code and showcases CDP's flexibility for building composable ingestion pipelines.
A Stellar Platform Team engineer experimented with the newly released Composable Data Platform (CDP) and Galexie to add backfilling capabilities to Stellar RPC. Unlike Horizon's deep history management, RPC is designed as a lightweight live gateway that doesn't backfill historical data. By leveraging CDP's data lake model and the BufferedStorageBackend from the ingest library, the engineer successfully backfilled 17,280 ledgers (approximately one day of data, ~28GB) in about 12 minutes—compared to over an hour using captive core replay. The implementation required minimal code and demonstrates CDP's power for efficient, parallel ledger data ingestion. The article highlights how CDP's flexibility enables developers to track payments, DEX activity, contract events, and other ledger insights without the overhead of repeated exports.
