Model_Simulation_Notes.txt

CENTRAL IDEA: Neural Networks can be used to approximate highly nonlinear functions very efficiently
	-Most important aspects of successful training and use:
		-Very good understanding of data 
			-What is it that you're solving for?
			-What kinds of input features do you have access to? (Not too many, as this may result in noise)
	-Most useful when relationships between parameters in equation are unknown, but large amounts of data exist

Probability Density Estimation Using Artificial Neural Networks (2000): B
	-Probability Density Functions (pdf): important computational problem with applications all over physics
	-Training data comes from non-parametric technique for pdf estimation (least amount of parameters as possible)
		-Train data across many parameter values

Modelling and Prediction of Hourly NOx and NO2 concentrations in London (1998): A-
	-Nitrogen Oxides (NO) are emitted into urban atmosphere primarily from vehicle exhausts
	-Much effort in attempting to forecast pollutants
	-Training data:
		-Input data: measurements of meteorological conditions
		-Output: pollutant concentration
	-Architecture: two hidden layers with 20 nodes each
	-Activation function: hyperbolic tangent (output layer used identity function??)
	-6 meteorlogical variables used for input
	-All data normalized between -1 and 1 (just like activation output)
	-This model worked considerably better than previous models 

Neural Network Modelling of Coastal Algal Blooms (2003): A+
	-Major algal blooms around Hong-Kong and South China cause many problems for environment
	-Early warning to such phenomena would allow fish farmers to help wildlife
	-NNs represent opportunity because of ability to predict complicated patterns
	-Activation function: sigmoid (thus output data must be normalized between 0 and 1; could this be avoided with ReLu?)
	-Training data:
		-Monthly/biweekly water quality data 
		-Data from 1982-1992 used to train; data from 1993-2000 used to test
		-Based on simplest number of input variables known to affect algal dynamics (nine variables)
	-Architecture: one hidden layer with 3 neurons

Neural Network Modelling of Wave Overtopping at Coastal Structures (2007): B-
	-For safety assessment of coastal structures, reliable predictions of wave overtopping are required
	-Each data point described by number of parameters that represent hydraulic/structural information
	-Data preprocessed to remove bad outliers 
	-Training data:
		-Total database consisted of 8372 tests
		-15 total parameters used 
		-All inputs and outputs scaled 
	-Cost function: Root-mean-squared
	-Architecture: 1 hidden layer with 20 neurons

Neural Network Model for Bankruptcy Prediction (1990): A
	-Generalization: method for using neural networks for predictions
	-Sample: firms that went bankrupt between 1975 and 1982 (129 firms, 65 bankrupt, 64 non-bankrupt)
	-Training data:
		-74 firms data (last financial statements issued before firms declared bankruptcy)
		-Input: 5 fincancial ratios 
	-Architecture: 1 hidden layer with 5 nodes
	-Output: scale between 0 and 1; firms below .5 classified as bankrupt, firms above .5 non-bankrupt
	-Converged after 191,400 iterations

Neural Network for Predicting Features of Osmotically Dehydrated Pumpkin (2017): A-
	-Pumpkin is very sensitive to spoilage, so it must be dried or frozen
	-Color of food product is most important quality parameter considered by consumers
	-Input: three variables (sucrose concentration, solution temperature, immersion time)
	-Output: three variables, three networks (color changes, shrinkage, texture)
	-Architecture: 1 hidden layer with 4 neurons 
	-Activation function: tangent sigmoid transfer function
	-27 sets of data used (three of each input variable)
	-Input and output normalized between 0 and 1

Applications of Artificial Neural Networks (2017): C+
	-Data mining: extracting knowledge from large amounts of data
	-Accuracy increases with number of training cases
	-NNs represent one type of data mining

Convolutional Neural Network with AdaBoost (2017) : A
	-Basic idea: use AdaBoost with strong classifiers (CNN)
	-Works very well by allowing CNNs to vote on output values

Multi-Level Attention Network (2017): A
	-Combination of CNN and RNN to classify a picture
	-Concept of AI Attention one of most exciting in current research
		-Gets us closer to fully functional AI
		-Allows AI to identify images (used to be impossible)