Stellar Development Foundation released an updated analytics dataset with 5-minute refresh intervals and open-sourced its data pipeline tooling, including stellar-etl and Apache Airflow orchestration, enabling developers to build real-time Stellar data pipelines.
The Stellar Development Foundation announced a new version of its analytics dataset that updates every 5 minutes instead of daily, along with open-sourced tooling for building custom data pipelines. The release includes stellar-etl, a command-line tool written in Go that extracts, transforms, and loads data from Stellar history archives and core, plus Apache Airflow DAGs for orchestrating complex data workflows with automatic retries and detailed logging. The infrastructure supports real-time use cases like market analysis and orderbook tracking. Developers can deploy the pipeline using custom Kubernetes setups or Google Cloud Composer, with full documentation and examples provided on GitHub.
