For optimizing an unsupervised neural network with 1 hidden layer, I use the training set for training and the validation set for optimizing the number of neurons in the hidden layer (for example by running a grid search of many options and comparing the resulting errors each architecture returns). Having obtained the optimal architecture, how do I approach the final evaluation step?
1) do I train the optimal model on the training set alone, followed by evaluation on the test set
2) do I train the optimal model on the training AND validation set combined, followed by evaluation on the test set
Since you tuned your model, you don’t need to (and you shouldn’t) separate the validation set because it means you are throwing your data away. Moreover, consider you do cross-validation, which fold(s) would you choose to ignore and what is your training set really?
Note: How do you optimize and calculate errors in an unsupervised problem?