Do I include the validation set in final training?

by Gonkulator   Last Updated January 22, 2019 22:19 PM

For optimizing an unsupervised neural network with 1 hidden layer, I use the training set for training and the validation set for optimizing the number of neurons in the hidden layer (for example by running a grid search of many options and comparing the resulting errors each architecture returns). Having obtained the optimal architecture, how do I approach the final evaluation step?

1) do I train the optimal model on the training set alone, followed by evaluation on the test set

2) do I train the optimal model on the training AND validation set combined, followed by evaluation on the test set



Answers 1


Since you tuned your model, you don’t need to (and you shouldn’t) separate the validation set because it means you are throwing your data away. Moreover, consider you do cross-validation, which fold(s) would you choose to ignore and what is your training set really?

Note: How do you optimize and calculate errors in an unsupervised problem?

gunes
gunes
January 22, 2019 22:04 PM

Related Questions