Machine Learning

Intro

Machine learning is so chic that every programmer even non-programmer starts to learn. After several months of online courses, everyone becomes self-proclaimed data scientist. The managers hold high hopes and deploy data scientists to machine learning this or that. In no time, people run into cul-de-sac, things don't work so well outside of the realm of iris dataset! If you have been to my other repositories like quant trading or graph theory, you must have seen me bashing reckless applications of machine learning. Stop selling AI snake oil! Don't get me wrong. I ain't no machine-learning-sceptic. I see great potential in machine learning but I am merely cynical to the current overstatement of artificial intelligence where it is frankly nowhere in sight.

The most popular supervised learning has very rigid requirement in both data quality and data quantity. Reinforcement learning is a drain on existing hardware. On the contrary, unsupervised learning is something I mess around frequently. It greatly boosts my work efficiency by dimension reduction, although I struggle to interpret the substantial meaning of the clustering pattern from time to time. In short, machine learning is no panacea. Its strongest suit is classification with discrete answers. When it comes to predicting stock price tomorrow or computing basic reproduction number yesterday, we still have to take the conventional path.

This repository is based upon the course material by Stanford University. Professor Andrew Ng may not teach the most comprehensive lectures but he has inspired millions to study data science. This repository attempts to replicate every algorithm mentioned in the course as well as the popular ones outside of the course. The experienced coders urge us not to reinvent the wheel but I firmly believe we never truly understand how a wheel works until we reinvent it. If you only learn OPTICS from some articles on towardsdatascience.com, you would've skipped DBSCAN since OPTICS does not require the key input ε. Well, by reinventing the wheels, you would come to senses that this is purely quid pro quoi. The introduction of new input ξ is crucial to determine the clustering. Yet, few people talk about it. In that sense, data modelling is not really scientific and will never be that way. Machine learning is a state of art where you fine tune the parameters to create discrete answers to the real-life problems. I sincerely hope this repository can help you see that.

---

Algorithms

Supervised

• Approximate Bayesian Computation

• Coordinate Descent (Lasso Regression/Elastic Net Regression)

• Generative Learning (Gaussian Discriminant Analysis)

• Gradient Descent (Batch/Stochastic/Mini-batch/Multiplicative Update)

• Least Squares (Linear Regression/Ridge Regression)

• Naïve Bayes (Multivariate/Multinomial/TF-IDF/KL Divergence)

• Newton Method (Logistic Regression)

• Support Vector Machine (Binary/Multiclass/DAG/SMO)

• Tree-based Learning (Decision Tree/Random Forest)

• Instance-based Learning (K Nearest Neighbors)

Unsupervised

• Centroid-based Model (K Means)

• Density-based Model (DBSCAN/OPTICS)

• Distribution-based Model (Gaussian Mixture Model/Naïve Bayes Mixture Model)

• Expectation Maximization (Factor Analysis/Dawid-Skene Model/Platt-Burges Model)

• Matrix Completion (ISTA/FISTA/ADMM)

• Principal Component Analysis (Singular Value Decomposition/Low Rank Approximation/Latent Semantic Indexing)

• Recommender System (Alternating Least Squares/Funk SVD/SVD++/Non-negative Matrix Factorization/Slope One/KNN with Baseline/Koren Neighborhood Model/Koren Integrated Model/Co-clustering)

• Signal Processing (Independent Component Analysis)

Applications

1. Reverse Engineering project

Creating a visualization from data is easy. In Tableau, it's only one click. What happens if you want to extract data from a visualization? A simple google search yields a few reverse engineering tools, yet they share the same malaise – they only work with single curve and require a lot of clicks. This project addresses these issues by incorporating unsupervised learning into image processing. Multiple curves are separated by different color channels with clustering techniques. Data can be easily extracted via computing coordinates of each pixel. A simple conversion from resolution scale to axis scale approximates the coordinates to the original spreadsheet. Voila, no more ridiculous subscription to Statista 😲

For more details, please refer to the read me page of a separate directory or machine learning section on my personal blog.

2. Wisdom of Crowds project

Every now and then, we read some bulge brackets hit the headline, “XXX will reach 99999€ in 20YY”. Some forecasts hit the bull’s eye but most projections are as accurate as astrology. Price prediction can be easily influenced by the cognitive bias. In the financial market, there is merit to the idea that consensus estimate is the best oracle. By harnessing the power of ensemble learning, we are about to leverage Dawid-Skene model and Platt-Burges model to eliminate the idiosyncratic noise associate with each individual judgement. The end game is to reveal the underlying intrinsic value generated by the collective knowledge of research analysts from different investment banks. Is wisdom of crowds a crystal ball for trading?

For more details, please refer to the read me page of a separate directory or machine learning section on my personal blog.