Skip to content

Experiments with artificial neural networks on small datasets.

Notifications You must be signed in to change notification settings

thomberg1/SmallDatasetsAnalysis

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

38 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Experiments on Small Datasets with Deep Learning

This repository contains experiments that use neural networks to learn classifiers using small datasets.

The datasets are:

Breast Cancer Wisconsin Data Set (reference to UCI ML Repository description)
Diabetes Data Set (reference to UCI Machine Learning Repository description)
Ionosphere Data Set (reference to UCI Machine Learning Repository description)
Mushroom Data Set (reference to UCI Machine Learning Repository description)

For each dataset there are three groups of notebooks:

Data Analysis performs basic data analysis, visualizations of the data sets, outlier detection and feature engineering.
ML model search uses traditional sklearn Grid Search to select a traditional sklearn estimator (e.g. Linear Regression) and allows tuning hyper-parameters.
ML classifier trains the final estimator and validates its performance using metrics and visualizations.
DL model search uses traditional sklearn Grid Search to select a MLP implemented in pytorch and allows tuning hyper-parameters.
DL classifier trains the final estimator and validates its performance using metrics and visualizations.

The notebooks implement the following data analysis, model selection and tuning flow for both traditional ML estimators and NNs:

Alt text


Notes:
• All process steps have been configured for a 12 core CPU and a 12Mb GPU running OSX and need to be re-configured if executed on significantly different HW (especially the n_jobs parameter of sklearn functions).
• All functions make heavy use of python multi-processing and might hang is a sub-process fails. Use the verbose parameter to get log information from sklearn for debugging purposes.
• Sklearn GridSearchCV is used but RandomizedSearchCV and BayesSearchCV can be plugged in but did not perform in this experiment.
• A wrapper called Classifier is used to plug pytorch NNs into the sklearn functions. The code is in the lib directory. A supported version of similar functionality can be found here.
• The focus of the notebooks is to understand the behavior of traditional ML models versus neural networks if one uses small datasets (i.e. overtraining, overfitting, runtime, metrics). Only minimal feature engineering and individual tuning has been done.
• If code has been included form other places the URL of the source blog, repository, etc. has been included above the code.

About

Experiments with artificial neural networks on small datasets.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published