mirror of
https://github.com/autistic-symposium/tensorflow-for-deep-learning-py.git
synced 2025-05-12 11:42:14 -04:00
3.7 KiB
3.7 KiB
Curated Resources on ETL, Machine Learning, and ML Pipelines
The morale of this repository is the fact that Machine learning involves tasks that include data sourcing, data ingestion, data transformation, pre-processing data for use in training, training a model and hosting the model.
Three conceptual steps are how most data pipelines are designed and structured:
- Extract: sensors wait for upstream data sources.
- Transform: business logic is applied (e.g. filtering, grouping, and aggregation to translate raw data into analysis-ready datasets).
- Load: processed data is transported to a final destination.
Learning References
Courses and Lists
- Data science resources.
- Lorte data pipelining.
- Incubator Airflow data pipelining
- Udemy's Airflow for Beginners.
- Awesome Airflow Resources.
- Airflow in Kubernetes.
- Astronomer: Airflow as a Service.
- Data pipeline samples.
- Awesome Scalability: a lot of articles and resources on the subject.
- Coursera's Big Data Pipeline course.
- Industrial Machine Learning Talk.
Enterprise Solutions
- Netflix data pipeline.
- Netlix data videos.
- Yelp data pipeline.
- Gusto data pipeline.
- 500px data pipeline
- Twitter data pipeline.
- Coursera data pipeline.
- Cloudfare data pipeline.
- Pandora data pipeline.
- Heroku data pipeline.
- Zillow data pipeline.
- Airbnb data pipeline.
- Walmart data pipeline.
- Robinwood data pipeline.
- Lyft data pipeline.
- Slack data pipeline.
- Remind data pipeline.
- Wish data pipeline.
- Databrick data pipeline.