⛓🛠 blockchain data engineering

👉 this repository contains some pf my blockchain engineering projects such as scalable event scanners and infrastructure setups for on-chain analysis and machine learning models training.

🛠 here is a high-level system design chart for a possible blockchain intelligence data platform (all deployed in kubernetes):

---

in this repo

• token-scanner-api:
  • a mvp for a scalable event scanner cli and api for ethereum, through indexing and parsing blocks events. this is the first step for training machine learning models on the chains (e.g., high-frequency trading with deep learning).
  • check my mirror post on building a scalable event scanner for ethereum.
• apache arrow
• rlp enconding
• spotify's luigi
• google's or-tools

---

external resources

• go-outside-labs ml-hft-agents
• go-outside-labs orchestration-toolkit
• google biquery article on blockchain public datasets
• paradigm's data portal