mistake in Bayes by Backprop from scratch

[ykun91]

I found a mistake in Bayes by Backprop from scratch (NN, classification)

In Model definition - Neural net modeling Section

def net(X, layer_params):
    layer_input = X
    for i in range(len(layer_params) // 2 - 2):
        h_linear = nd.dot(layer_input, layer_params[2*i]) + layer_params[2*i + 1]
        layer_input = relu(h_linear)

it shoud be

def net(X, layer_params):
    layer_input = X
    for i in range(len(layer_params) // 2 - 1):
        h_linear = nd.dot(layer_input, layer_params[2*i]) + layer_params[2*i + 1]
        layer_input = relu(h_linear)

range(len(layer_params) // 2 - 2 will make the second hidden layer to be ignored in forward propagation, but luckily in your case it did not cause a error.

I changed it to range(len(layer_params) // 2 - 1 and rerun the notebook. it wroks fine but seems that there are no improves in result.

Epoch 0. Loss: 2631.05110392, Train_acc 0.94465, Test_acc 0.9443
Epoch 1. Loss: 2607.93411326, Train_acc 0.96125, Test_acc 0.9584
Epoch 2. Loss: 2601.34667942, Train_acc 0.969633, Test_acc 0.9626
Epoch 3. Loss: 2596.63901145, Train_acc 0.974683, Test_acc 0.9679
Epoch 4. Loss: 2593.85184135, Train_acc 0.978433, Test_acc 0.9701
Epoch 5. Loss: 2590.91976531, Train_acc 0.979283, Test_acc 0.9693
Epoch 6. Loss: 2588.03490818, Train_acc 0.983067, Test_acc 0.9734
Epoch 7. Loss: 2586.94554946, Train_acc 0.985667, Test_acc 0.9739
Epoch 8. Loss: 2585.43967896, Train_acc 0.98685, Test_acc 0.9758
Epoch 9. Loss: 2582.76947339, Train_acc 0.98725, Test_acc 0.9755

[Toooodd]

I think you are right!!!