Description
I'm going through the following guide: https://www.kubeflow.org/docs/components/tfserving_new/
I've managed to get the guide running up to this point: "Sending prediction request directly" (see my issue here: #724)
Running kubectl get svc mnist-service will return the external IP to use. But the line that follows will not work:
curl -X POST -d @input.json http://EXTERNAL_IP:8500/v1/models/mnist:predict
Firstly, the file input.json does not exist and there is no explanation in the guide for how to create it.
Secondly (and more importantly), the port 8500 (and also 8501) does not appear to be open. The only open ports are 9000:30347/TCP, 8000:31058/TCP
For example
curl http://aba09de577ae811e9bc1c0a77a60ece6-1422674746.us-east-1.elb.amazonaws.com:8000 returns an empty result
curl http://aba09de577ae811e9bc1c0a77a60ece6-1422674746.us-east-1.elb.amazonaws.com:8500 times out
Lastly, since the example uses supposedly uses an "mnist" model it would be useful to include instructions on where to find such this model and where to place it (for example in S3) in order to use it with this guide.
EDIT
On further inspection, it seems that the pods are not starting either. kubectl describe pods yields the following:
Warning FailedScheduling 40s (x8 over 4m) default-scheduler 0/1 nodes are available: 1 Insufficient nvidia.com/gpu.
I presume this means I need to explicitly add nodes with GPUs to the cluster. Following the steps in https://www.kubeflow.org/docs/aws/deploy/install-kubeflow/ only a m5.2xlarge Nodegroup is created. My expectation working through the guides would be that the one follows the other without any unexplained steps. If indeed it is necessary to create (for example) p3.2xlarge nodegroups as well, I would expect this to be done explicitly in the installation example, or otherwise it should be should here how to add the nodegroups.