Use cases: Prediction, classification and labeling, quntification of risk and uncertinty, feedback based recommendation Examples: Sales forecasting, Vegas odds, insurance risk, credit fraud detection
- Least Squares
- Sequential
- Penalized Regression - glmnet - R Package
- Ridge
- LASSO
- Group LASSO, Elastic Net ...
- Gaussian Process regression - Gaussian Processes for Machine Learning Python code
- Decision Tree - (R package - Generalized Boosted Regression Model) ( R package - random forest) (Python package with GPU acceleration - lightGBM ) (R and Python package with GPU acceleration - xgboost)
- Support vector machines (GPU accelerated code)
- Neural Networks
- K-Nearest Neighbors - Fast KNN
- Decision Tree - (R package - Generalized Boosted Regression Model) ( R package - random forest) (Python package with GPU acceleration - lightGBM ) (R and Python package with GPU acceleration - xgboost)
- Factor Analysis
- Analysis of Feature Importance - XGBoost explainer (R Package) (randomForestExplainer)
- Linear discriminant analysis
- Discriminant analysis of principal components (DAPC)(R package - adegenet)
Use cases: Structure Discovery, grouping/labeling when no labels are known, implicit recommendation, improve supervised methods Examples: Youtube Recomendations, google translate
- PCA - Practical Guide to Principal Component Analysis (PCA) in R & Python
- T-SNE - Comprehensive Guide on t-SNE algorithm with implementation in R & Python (R Package)
- Vector Embeddings/Distributed Representations - An Intuitive Understanding of Word Embeddings: From Count Vectors to Word2Vec (Python Package- gensim) (List of embedding methods for text)
- Matrix Factorization - (Python-Pytorch package - Spotlight)
- Recommendation Systems - (Introduction to Recommender System. Part 1 (Collaborative Filtering, Singular Value Decomposition))
- Autoencoders
- Mixture Models - mixtools - R Package
- Dirichlet Process Models Python package
- DBSCAN (R package - dbscan)
Use cases: A/B/C... tests, maintain equalibrium or setpoint, explore/exploit optimization Examples:Website optimization, content personalization, auto pilot
- PID Algorithm - PID controller (Python Code)
- Bandit Algorithms - Introduction to Multi-Armed Bandits (Python Code)
-
Introduction to Various Reinforcement Learning Algorithms. Part I (Q-Learning, SARSA, DQN, DDPG)
-
Introduction to Various Reinforcement Learning Algorithms. Part II (TRPO, PPO)
- Voting
- Averaging
- Stacked ensembling/Blending
Be careful when you ensamble, "Here there be dragons."
- Competing in a data science contest without reading the data
- The reusable holdout: Preserving validity in adaptive data analysis