mirror of
https://github.com/autistic-symposium/tensorflow-for-deep-learning-py.git
synced 2025-05-12 03:34:59 -04:00
3.8 KiB
3.8 KiB
Curated Resources on ETL, Machine Learning, and ML Pipelines
The morale of this repository is to cover resources for deploying Machine learning
in production environments, a task that includes data sourcing, data ingestion, data
transformation, pre-processing data for use in training, training a model, and hosting
the model.
Three conceptual steps are how most data pipelines are designed and structured:
- Extract: sensors wait for upstream data sources.
- Transform: business logic is applied (e.g. filtering, grouping, and aggregation to translate raw data into analysis-ready datasets).
- Load: processed data is transported to a final destination.
Tools & Code Samples
- Data science resources.
- Incubator Airflow data pipelining
- Awesome Airflow Resources.
- Airflow in Kubernetes.
- Lorte data pipelining.
- Astronomer: Airflow as a Service.
- AWS Data pipeline samples.
MOOCs
Tutorials & Articles
2019
Enterprise Solutions
- Netflix data pipeline.
- Netlix data videos.
- Yelp data pipeline.
- Gusto data pipeline.
- 500px data pipeline
- Twitter data pipeline.
- Coursera data pipeline.
- Cloudfare data pipeline.
- Pandora data pipeline.
- Heroku data pipeline.
- Zillow data pipeline.
- Airbnb data pipeline.
- Walmart data pipeline.
- Robinwood data pipeline.
- Lyft data pipeline.
- Slack data pipeline.
- Remind data pipeline.
- Wish data pipeline.
- Databrick data pipeline.