Skip to content

Commit 1654842

Browse files
ukclivecoxk8s-ci-robot
authored andcommitted
Update Seldon docs (#1073)
* Update Seldon docs * Update docs based on review
1 parent 1b9e0d6 commit 1654842

File tree

1 file changed

+8
-60
lines changed

1 file changed

+8
-60
lines changed

content/docs/components/serving/seldon.md

+8-60
Original file line numberDiff line numberDiff line change
@@ -6,72 +6,20 @@ weight = 40
66

77
## Serve a model using Seldon
88

9-
_This section has not yet been converted to kustomize, please refer to [kubeflow/manifests/issues/10](https://github.com/kubeflow/manifests/issues/10)._
9+
Seldon comes installed with Kubeflow. Full documentation for running Seldon inference is provided within the [Seldon documentation site](https://docs.seldon.io/projects/seldon-core/en/latest/).
1010

11-
[Seldon-core](https://github.com/SeldonIO/seldon-core) provides deployment for any machine learning runtime that can be [packaged in a Docker container](https://docs.seldon.io/projects/seldon-core/en/latest/wrappers/README.html).
11+
If you have a saved model in a PersistentVolume (PV), Google Cloud Storage bucket or Amazon S3 Storage you can use one of the [prepackaged model servers provided by Seldon](https://docs.seldon.io/projects/seldon-core/en/latest/servers/overview.html).
1212

13-
Install the seldon package:
13+
Seldon also provides [language specific model wrappers](https://docs.seldon.io/projects/seldon-core/en/latest/wrappers/README.html) to wrap your inference code for it to run in Seldon.
1414

15-
```
16-
ks pkg install kubeflow/seldon
17-
```
18-
Generate the core components for v1alpha2 of Seldon's CRD:
15+
### Kubeflow Specifics
1916

20-
```
21-
ks generate seldon seldon
22-
```
17+
* By default Seldon is configured to use the istio Gateway `kubeflow-gateway` and will add Virtual Services for the Seldon resources you create which [expose Seldon paths to the Kubeflow istio gateway](https://docs.seldon.io/projects/seldon-core/en/latest/workflow/serving.html#istio).
2318

24-
If you wish to use Seldon's previous v1alpha1 version of its CRD you need to set the ```seldonVersion``` parameter to one in the 0.1.x range, for example:
19+
### Examples
2520

26-
```
27-
ks param set seldon seldonVersion 0.1.8
28-
ks generate seldon seldon
29-
```
21+
* [Kubeflow Seldon E2E Pipeline](https://docs.seldon.io/projects/seldon-core/en/latest/examples/kubeflow_seldon_e2e_pipeline.html)
3022

31-
Deploy seldon cluster manager:
23+
Seldon provides a [large set of example notebooks](https://docs.seldon.io/projects/seldon-core/en/latest/examples/notebooks.html) showing how to run inference code for a wide range of machine learning toolkits.
3224

33-
```
34-
export KF_ENV=default
35-
ks apply ${KF_ENV} -c seldon
36-
```
3725

38-
The `KF_ENV` environment variable represents a conceptual deployment environment
39-
such as development, test, staging, or production, as defined by
40-
ksonnet. For this example, we use the `default` environment.
41-
You can read more about Kubeflow's use of ksonnet in the Kubeflow
42-
[ksonnet component guide](/docs/components/ksonnet/).
43-
44-
### Seldon Deployment Graphs
45-
46-
Seldon allows complex runtime graphs for model inference to be deployed. Some example prototypes have been provided to help you get started. Follow the [Seldon docs](https://docs.seldon.io/projects/seldon-core/en/latest/wrappers/README.html) to wrap your model code into an image that can be managed by Seldon. In the examples below we will use a model image ```seldonio/mock_classifier``` ; replace this with your actual model image. You will also need to choose between the v1alpha2 and v1alpha1 prototype examples depending on which version of Seldon you generated above. The following prototypes are available:
47-
48-
* **A single model to serve**.
49-
* ```ks generate seldon-serve-simple-<seldonVersion> mymodel --image <image>```
50-
* Example: ```ks generate seldon-serve-simple-v1alpha2 mymodel --image seldonio/mock_classifier:1.0```
51-
* **An A-B test between two models**.
52-
* ```ks generate seldon-abtest-<seldonVersion> myabtest --imageA <imageA> --imageB <imageB>```
53-
* Example: ```ks generate seldon-abtest-v1alpha2 myabtest --imageA seldonio/mock_classifier:1.0 --imageB seldonio/mock_classifier:1.0```
54-
* **A multi-armed bandit between two models**. Allowing you to dynamically push traffic to the best model in real time. For more details see an [e-greedy algorithm example](https://github.com/SeldonIO/seldon-core/blob/master/notebooks/epsilon_greedy_gcp.ipynb).
55-
* ```ks generate seldon-mab-<seldonVersion> mymab --imageA <imageA> --imageB <imageB>```
56-
* Example: ```ks generate seldon-mab-v1alpha2 mymab --imageA seldonio/mock_classifier:1.0 --imageB seldonio/mock_classifier:1.0```
57-
* **An outlier detector for a single model**. See more details on the [default Mahalanobis outlier detection algorithm](https://github.com/SeldonIO/seldon-core/blob/master/components/outlier-detection/mahalanobis/outlier_mahalanobis.ipynb).
58-
* ```ks generate seldon-outlier-detector-<seldonVersion> myout --image <image>```
59-
* Example: ```ks generate seldon-outlier-detector-v1alpha2 myout --image seldonio/mock_classifier:1.0```
60-
61-
### Endpoints
62-
63-
Seldon exposes your prediction graph via REST and gRPC endpoints. Within Kubeflow these will be available via the Ambassador reverse proxy or via Seldon's OAuth API gateway if you installed it (set the ```withApife``` parameter to 'true' in the seldon component).
64-
65-
Assuming Ambassador is exposed at ```<ambassadorEndpoint>``` and with a Seldon deployment name ```<deploymentName>```:
66-
67-
* A REST endpoint will be exposed at : ```http://<ambassadorEndpoint>/seldon/<deploymentName>/api/v0.1/predictions```
68-
* A gRPC endpoint will be exposed at ```<ambassadorEndpoint>``` and you should send metadata in your request with key ```seldon``` and value ```<deploymentName>```.
69-
70-
[Example Jupyter notebooks](https://github.com/SeldonIO/seldon-core#quick-start) are provided in Seldon's docs that show code that can be used as a basis to test the endpoints.
71-
72-
### Next Steps with Seldon
73-
74-
* Seldon provides a set of generic building blocks so users can create their own components to place in their runtime inference graphs. Aside from models, there are Routers (e.g., A-B tests, Multi-Armed Bandits), Combiners (e.g., Ensemblers) and Transformers (e.g., feature normalisation, outlier detection). To understand more [consult the Seldon docs](https://github.com/SeldonIO/seldon-core/blob/master/docs/reference/internal-api.md).
75-
* To understand how to upgrade your Seldon manifest's from v1alpha1 to v1alpha2 follow [this guide](https://github.com/SeldonIO/seldon-core/blob/master/docs/v1alpha2_update.md).
76-
* For an example end-to-end integration see the [kubeflow-seldon example](https://github.com/kubeflow/example-seldon).
77-
* For more details and example on the above see the [Seldon documentation](https://github.com/SeldonIO/seldon-core).

0 commit comments

Comments
 (0)