mirror of
https://github.com/autistic-symposium/blockchain-data-engineering-toolkit.git
synced 2025-04-27 19:26:06 -04:00
1.8 KiB
1.8 KiB
⛓🛠 blockchain data engineering
👉 this repository contains some pf my blockchain engineering projects such as scalable event scanners and infrastructure setups for on-chain analysis and machine learning models training.
🛠 here is a high-level system design chart for a possible blockchain intelligence data platform (all deployed in kubernetes):
in this repo
- token-scanner-api:
- a mvp for a scalable event scanner cli and api for ethereum, through indexing and parsing blocks events. this is the first step for training machine learning models on the chains (e.g., high-frequency trading with deep learning).
- check my mirror post on building a scalable event scanner for ethereum.
- apache arrow
- rlp enconding
- spotify's luigi
- google's or-tools