Add validation and training sets to NNPredictor

[JBlaschke]

Use validation data to test the "tolerance" of the forward model. We can also use the training loss function to check if the NNPredictor needs more gradient descent steps.