Skip to content

Commit a591244

Browse files
author
Jiaxiao Zheng
authored
Clean up 'out of date' banner for few more KFP pages (#2174)
* Squashed commit of the following: commit a601bc0 Author: Oleg Butuzov <[email protected]> Date: Wed Aug 21 02:25:17 2019 +0300 markdown formatting fix. (#1088) this fix changes markdown formating for link to `/docs/started/k8s/overview` page, that currently broken on the website. see - https://www.kubeflow.org/docs/other-guides/virtual-dev/getting-started-minikube/ commit 48955d4 Author: Yuan (Bob) Gong <[email protected]> Date: Tue Aug 20 09:49:32 2019 +0800 Update kubeflow pipelines upgrade guide for kubeflow 0.6 (#1055) * Fix kfp upgrade/reinstall preparation guide. * Update upgrade instructions for kubeflow deployment. * Remove kfp upgrade related instructions because they don't work * Rephrase PV to PD to be more accurate commit 384368d Author: Sarah Maddox <[email protected]> Date: Tue Aug 20 10:23:33 2019 +1000 Clarified the usage reporting guide (#1087) * Clarified usage reporting guide. * Clarified and added section formatting. * Changed word component to application * Changed another "component" to "application" commit 564e460 Author: Yuval Yaari <[email protected]> Date: Sun Aug 18 03:46:04 2019 +0000 Fixing command to retrieve the existing Dex configmap (#1086) commit 1a863f4 Author: Leilei Hu <[email protected]> Date: Sat Aug 17 15:58:06 2019 -0400 Modify the out of date guide docs. (#1085) commit 5248a15 Author: Yannis Zarkadas <[email protected]> Date: Sat Aug 17 06:16:08 2019 +0300 pinconfig links for v0.6 to v0.6-branch (#1071) Signed-off-by: Yannis Zarkadas <[email protected]> commit 0f480d3 Author: Adhita Selvaraj <[email protected]> Date: Fri Aug 16 14:14:09 2019 -0700 Update kfctl-k8s-istio.md (#1044) * Update kfctl-k8s-istio.md Update script for quickly curling the latest release. Similar to kustomize: https://github.com/kubernetes-sigs/kustomize/blob/master/docs/INSTALL.md#quickly-curl-the-latest * Keep both versions for install instructions commit 09e44c2 Author: Sarah Maddox <[email protected]> Date: Sat Aug 17 06:38:09 2019 +1000 Clarified port forwarding and auth for GCP (#1083) * Clarifies port forwarding and auth for GCP. * Fixed formatting. * Addressed review comments. commit a8d9887 Author: Yuan (Bob) Gong <[email protected]> Date: Fri Aug 16 08:12:34 2019 +0800 Update pipeline quickstart for kf 0.6 (#1080) commit 1654842 Author: cliveseldon <[email protected]> Date: Fri Aug 16 00:54:34 2019 +0100 Update Seldon docs (#1073) * Update Seldon docs * Update docs based on review commit 1b9e0d6 Author: Jiaxin Shan <[email protected]> Date: Wed Aug 14 19:44:33 2019 -0700 Merge installation guidance for new and existing cluster (#1072) * Merge installation guidance for new and existing cluster * Address code review feedbacks commit 2b5c19a Author: Sarah Maddox <[email protected]> Date: Thu Aug 15 12:30:33 2019 +1000 Change will to may in metadata warning. (#1078) commit 8177767 Author: Sarah Maddox <[email protected]> Date: Tue Aug 13 17:03:39 2019 +1000 Expanded docs for the Metadata component (#1061) * WIP Expanded docs for Metadata component. * Added more about metadata. * Addressed review comments. commit 5b6caa1 Author: Jeremy Lewi <[email protected]> Date: Mon Aug 12 18:33:37 2019 -0700 Fix port-forarding instructions; port-forward to istio gateway (#1067) * Fix port-forarding instructions; port-forward to istio gateway See #1041 and kubeflow/kubeflow#3618 * With 0.6 if no auth is setup and you want to connect via port-forwarding then you need to port-forward to the ISTIO gateway; not ambassador * I suspect the original suggestion in #1041 to port-forward to Ambassador only worked in the basic auth setting in which case we are still setting up Ambassador. Fix: #1041 * Cleanup the text. commit 2d9564c Author: Zhenghui Wang <[email protected]> Date: Mon Aug 12 17:41:38 2019 -0700 Add metadata API reference doc (#1058) * add metadata API spec * add link to metadata doc page * address comments * remove metadata.md commit d01f2cb Author: Christos S. Pavlou <[email protected]> Date: Tue Aug 13 00:25:38 2019 +0300 Amend the kfctl_existing_arrikto doc (#1056) Add instructions for installing MetalLB Add `Troubleshooting` section Add architecture figure Change the kfctl_existing_arrikto description in the Overview page Signed-off-by: Christos S. Pavlou <[email protected]> commit 85ba5bb Author: Sarah Maddox <[email protected]> Date: Tue Aug 13 02:15:13 2019 +1000 Refactored 'Other Guides' and applied redirects. (#1064) commit cf7b9f3 Author: Jiaxiao Zheng <[email protected]> Date: Sun Aug 11 18:21:10 2019 -0700 Fix broken links. (#1060) commit 40a3f49 Author: Carmine Rimi <[email protected]> Date: Sun Aug 11 19:43:10 2019 -0400 Refactor getting started - streamlined hierarchy (#1051) * Refactor getting started - streamlined hierarchy A number of changes / benefits: 1) eliminated some duplicate information 2) added three main top level areas for installation: - cloud - workstation / server - existing kubernetes 3) Adding workstation operating systems - there are some slight variances in experience. For instance, for MacOS, we can emphasize brew to install some of the assets. For Windows, we'll want to add WSL2, and potentially chocolatey. * Updated PR #1051 with recommended changes This commit holds a set of minor changes that were requested in #1051 * Move VM based local workstation to reference Moving the VM instructions out of getting started directory tree so that the navigation bar remains clean / less busy. * Add deleted assets files, reword k8s _index A previous commit deleted the resources/_gen/assets/*/search*.content file. This commit adds it back in. Additionally, fixed the grammar in k8s/_index.md commit 05d3d92 Author: Ning <[email protected]> Date: Fri Aug 9 21:51:13 2019 -0700 update sample link after kfp sample restructure (#1057) commit 9c7fcd8 Author: IronPan <[email protected]> Date: Fri Aug 9 20:33:13 2019 -0700 Update pipeline API version (#1054) * update pipeline deployment doc * update pipeline deployment doc * update pipeline deployment doc * update pipeline deployment doc * Update deploy-pipelines-service.md * update pipeline deployment doc * update pipeline deployment doc * update pipeline deployment doc * update pipeline deployment doc * comment * format * update kf version * update deployment wording * update deployment wording * update deployment wording * update commit 61de916 Author: Lun-Kai Hsu <[email protected]> Date: Fri Aug 9 16:35:14 2019 -0700 Fix tfserving doc (#1025) * fix tfserving doc * fix * fix commit 0823f78 Author: Sarah Maddox <[email protected]> Date: Sat Aug 10 06:51:13 2019 +1000 Updated screenshots and UI info for Kubeflow v0.6 (#1029) * WIP Updating screenshots and UI info for v0.6 * Fixed UI text and URLs in pipelines quickstart. * Fixed UI text and clarified Cloud Shell in Pipelines tutorial. * Removed screenshot of Kubeflow central UI as it doesn't show TFJob. * Started updates to notebooks. * Updated port-forwarding instructions for UI. * Updated UI for notebooks. * Finished UI updates for notebooks. * Fixed caps in URL variable. commit cc50aca Author: Ed Henry <[email protected]> Date: Thu Aug 8 21:41:24 2019 -0700 xgboost url update (#1052) * xgboost url update Pipeline overview doc reflected a stale URL. This commit updates to the correct URL of the xgboost-spark directory of the Kubeflow repo. * Remove docsy submodule Removing the docsy submodule commit ca09f1f Author: Jiaxin Shan <[email protected]> Date: Thu Aug 8 20:33:25 2019 -0700 Update aws docs for 0.6 release (#1030) * Update aws docs for 0.6 release * Address code review feedbacks * Update indents to render markdown * Remove shell in code blocks * Fix indent issue * Make existing cluster solution more clear commit d09cecc Author: Krishna Durai <[email protected]> Date: Fri Aug 9 08:55:25 2019 +0530 Instructions on setting up dynamic volume provisioner (#1040) * Instructions on setting up dynamic volume provisioner Resolves kubeflow/kubeflow#3774 * Clarifies what to delete * Rearranges and clarifies descriptions * Re-arranges and rephrases content * Fixes link Corrects description * Addresses review comments commit f557a0e Author: Hougang Liu <[email protected]> Date: Thu Aug 8 12:27:16 2019 +0800 Update katib Dockerfile locations link (#1049) commit 0b03ae0 Author: Abhishek Gupta <[email protected]> Date: Wed Aug 7 21:19:17 2019 -0700 Update to include limitations and integrations (#1047) commit beefae9 Author: Michael Benavidez <[email protected]> Date: Wed Aug 7 09:02:51 2019 -0700 PR for Issue 795 (outdated Pipelines SDK guide) (#971) * Update pipelines-tutorial.md * Add files via upload * Update kubeflow-current-version.html * Delete kubeflow-current-version.html Unnecessary, kf-stable-tag already serves this purpose. * Update pipelines-tutorial.md * Update pipelines-tutorial.md * Update pipelines-tutorial.md * Update build-component.md * Update build-component.md * Update build-component.md * Update build-component.md * Update build-component.md * Update build-component.md commit 2f5356c Author: Yuan (Bob) Gong <[email protected]> Date: Wed Aug 7 10:32:53 2019 +0800 Fix other js features broken by heading anchor link. (#1039) commit c36ace3 Author: Jin Chi He <[email protected]> Date: Wed Aug 7 09:24:53 2019 +0800 fix mini problem in fairing doc (#1031) commit 670f8b4 Author: Leilei Hu <[email protected]> Date: Tue Aug 6 21:18:51 2019 -0400 Fix docs link 404 errors. (#1032) commit ef6374a Author: Sarah Maddox <[email protected]> Date: Wed Aug 7 10:28:52 2019 +1000 Added redirect and fixed links to getting-started guide for existing clusters (#1035) * Added redirect and fixed links to existing-custer getting-started guide. * Added empty line at end of file. commit dda2f0c Author: Sarah Maddox <[email protected]> Date: Sun Aug 4 06:23:28 2019 +1000 Added a warning. commit d27e54b Author: Sarah Maddox <[email protected]> Date: Fri Aug 2 10:53:22 2019 +1000 Fixed a typo. commit 1a5bd1b Author: Sarah Maddox <[email protected]> Date: Fri Aug 2 10:48:50 2019 +1000 Fixed a couple more links. commit 065a133 Author: Sarah Maddox <[email protected]> Date: Fri Aug 2 10:35:45 2019 +1000 Added page for shared components and resources commit 70bda37 Author: Johnu George <[email protected]> Date: Tue Aug 6 22:57:56 2019 +0530 Fixing doc for KF installation on existing clusters (#977) * Fixing doc for existing clusters * Update authentication.md * Update getting-started-k8s.md * Update getting-started-k8s.md * Address review comments * Minor changes * Address review comments * Address review comments * Minor title change * Fix path commit 63b34cb Author: Sarah Maddox <[email protected]> Date: Tue Aug 6 13:55:53 2019 +1000 Fixed up the GCP deployment guide for CLI v0.6 (#1021) * WIP Fixing up the GCP deployment guide for CLI. * Further updates during CLI deployment testing. * More tweaks to wording for clarity. commit 39bf96e Author: Scott Lee <[email protected]> Date: Mon Aug 5 19:39:53 2019 -0700 Add references to kustomize issues (#1015) commit f1f3a59 Author: Eric Lilienstein <[email protected]> Date: Mon Aug 5 18:53:54 2019 -0700 Deployment UI requires project id, not name (#1028) Following the instructions above: - go to https://deploy.kubeflow.cloud - enter previously created GCP project name and other required information - click 'Create Deployment' Result: An error message indicates that the project can't be located. The required information was the project ID, not the project name commit f3d9c3a Author: Amy <[email protected]> Date: Mon Aug 5 17:43:54 2019 -0700 typo fix (#1027) commit b3fb238 Author: Amy <[email protected]> Date: Mon Aug 5 15:19:49 2019 -0700 added a few clarifications to the GKE CLI installation instructions (#1026) commit ecc9c5b Author: Yuan (Bob) Gong <[email protected]> Date: Mon Aug 5 12:55:48 2019 +0800 Fix anchor icon color + formatting (#1022) commit db5ad7e Author: Matt Callanan <[email protected]> Date: Mon Aug 5 06:33:48 2019 +1000 Fixed link to xgboost-training-cm.py (#1018) commit 89fd3f8 Author: Yuan (Bob) Gong <[email protected]> Date: Mon Aug 5 04:29:49 2019 +0800 Add anchor links to markdown headings. (#1009) * Add anchor links automatically. * Improve anchor link UX * improve corner-case layout stability * improve a11y * Change link icon to material icon commit 1634ea3 Author: Abhishek Gupta <[email protected]> Date: Fri Aug 2 15:43:54 2019 -0400 add owner role requirement in GCP project setup (#1016) commit 1dd668e Author: Iman Tabrizian <[email protected]> Date: Fri Aug 2 01:15:52 2019 -0400 Add troubleshooting guide for notebooks (#1008) * feat: add template of troubleshooting guide for notebooks * feat: improve the notebooks troubleshooting guide * fix: improve the note for GCP users * fix: minor bug in the link * fix: improve the troubleshooting guide * fix: fix troubleshooting guides commit 3bc674e Author: Sarah Maddox <[email protected]> Date: Thu Aug 1 23:26:15 2019 +1000 Moved GCP Pipelines tutorial to GCP section of docs (#1004) * Moved GCP Pipelines tutorial to GCP section of docs. * Addressed review comments. commit 38ab9cc Author: Ce Gao <[email protected]> Date: Thu Aug 1 09:42:14 2019 +0800 fix: Fix the repo (#1005) * fix: Fix the repo Signed-off-by: Ce Gao <[email protected]> * fix: Fix the statement Signed-off-by: Ce Gao <[email protected]> commit 070a0ea Author: Ning <[email protected]> Date: Wed Jul 31 16:54:13 2019 -0700 minor fix (#1007) commit be10697 Author: Sarah Maddox <[email protected]> Date: Thu Aug 1 03:18:14 2019 +1000 Added link from notebooks setup to multi-user isolation doc. (#1001) commit 7b086d7 Author: Sarah Maddox <[email protected]> Date: Wed Jul 31 15:22:20 2019 +1000 Fixed links to tutorials and codelabs. (#1003) commit e6eb291 Author: Jiaxiao Zheng <[email protected]> Date: Tue Jul 30 16:45:10 2019 -0700 Update customizing-gke.md (#987) * Update customizing-gke.md Fix stale links. * Update customizing-gke.md Fix the links to cluster.jinja also commit 67f6edb Author: Jiaxiao Zheng <[email protected]> Date: Tue Jul 30 16:31:09 2019 -0700 Update customizing-gke.md (#999) Correct a typo commit 395c848 Author: Zhenghui Wang <[email protected]> Date: Mon Jul 29 17:21:38 2019 -0700 Update metadata.md change the minimum requirements for using Metadata components. commit 7f4beb7 Author: Sarah Maddox <[email protected]> Date: Tue Jul 30 09:18:49 2019 +1000 Clarified versions available in deployment UI and CLI (#981) * Clarified versions available in deployment UI and CLI. * Removed empty line at EOF because it causes a space in rendering on page. * Review comment Co-Authored-By: Dwight J Lyle <[email protected]> commit fae0bf9 Author: Karen Bradshaw <[email protected]> Date: Mon Jul 29 17:58:49 2019 -0400 fix a few typos (#985) - See #936 - See #982 * minor typo and update a screenshot. * Update central-ui.png and correct wording * Correct stale naming * Update pipeline ui screenshot * Update pipeline upload screenshot * Update three pictures. * SDK install page * pipeline param * lint and remove banner for type check in DSL * remove banner for real * gcp * recover image * fix typo * Apply PR comments
1 parent 2956504 commit a591244

File tree

4 files changed

+86
-59
lines changed

4 files changed

+86
-59
lines changed

content/en/docs/pipelines/sdk/gcp.md

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,6 @@ description = "SDK features that are available on Google Cloud Platform (GCP) on
44
weight = 130
55

66
+++
7-
{{% alert title="Out of date" color="warning" %}}
8-
This guide contains outdated information pertaining to Kubeflow 1.0. This guide
9-
needs to be updated for Kubeflow 1.1.
10-
{{% /alert %}}
117

128
For pipeline features that are specific to GCP, including SDK features, see the
139
[GCP section of the docs](/docs/gke/pipelines/).

content/en/docs/pipelines/sdk/install-sdk.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,6 @@ description = "Setting up your Kubeflow Pipelines development environment"
44
weight = 20
55

66
+++
7-
{{% alert title="Out of date" color="warning" %}}
8-
This guide contains outdated information pertaining to Kubeflow 1.0. This guide
9-
needs to be updated for Kubeflow 1.1.
10-
{{% /alert %}}
117

128
This guide tells you how to install the
139
[Kubeflow Pipelines SDK](https://github.com/kubeflow/pipelines/tree/master/sdk)
@@ -80,16 +76,20 @@ Run the following command to install the Kubeflow Pipelines SDK:
8076
```bash
8177
pip3 install kfp --upgrade
8278
```
79+
8380
**Note:** If you are not using a virtual environment, such as `conda`, when installing the Kubeflow Pipelines SDK, you may receive the following error:
81+
8482
```bash
85-
ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python3.5/dist-packages/kfp-0.2.0.dist-info'
83+
ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python3.5/dist-packages/kfp-<version>.dist-info'
8684
Consider using the `--user` option or check the permissions.
8785
```
8886

8987
If you get this error, install `kfp` with the `--user` option:
88+
9089
```bash
9190
pip3 install kfp --upgrade --user
9291
```
92+
9393
This command installs the `dsl-compile` and `kfp` binaries under `~/.local/bin`, which is not part of the PATH in some Linux distributions, such as Ubuntu. You can add `~/.local/bin` to your PATH by appending the following to a new line at the end of your `.bashrc` file:
9494

9595
```bash

content/en/docs/pipelines/sdk/parameters.md

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,6 @@ description = "Passing data between pipeline components"
44
weight = 70
55

66
+++
7-
{{% alert title="Out of date" color="warning" %}}
8-
This guide contains outdated information pertaining to Kubeflow 1.0. This guide
9-
needs to be updated for Kubeflow 1.1.
10-
{{% /alert %}}
117

128
The [`kfp.dsl.PipelineParam`
139
class](https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.dsl.html#kfp.dsl.PipelineParam)
@@ -23,7 +19,7 @@ The task output references can again be passed to other components as arguments.
2319

2420
In most cases you do not need to construct `PipelineParam` objects manually.
2521

26-
The following code sample shows how to define pipeline with parameters:
22+
The following code sample shows how to define a pipeline with parameters:
2723

2824
```python
2925
@kfp.dsl.pipeline(
@@ -36,7 +32,9 @@ def my_pipeline(
3632
my_url: str = 'http://example.com'
3733
):
3834
...
35+
# In the pipeline function body you can use the `my_num`, `my_name`,
36+
# `my_url` arguments as PipelineParam objects.
3937
```
4038

41-
See more in the guide to [building a
42-
component](/docs/pipelines/sdk/build-component/#create-a-python-class-for-your-component).
39+
For more information, you can refer to the guide on
40+
[building components and pipelines](/docs/pipelines/sdk/build-component/#create-a-python-class-for-your-component).

content/en/docs/pipelines/sdk/static-type-checking.md

Lines changed: 76 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,6 @@ description = "Statically check the component I/O types"
44
weight = 100
55

66
+++
7-
{{% alert title="Out of date" color="warning" %}}
8-
This guide contains outdated information pertaining to Kubeflow 1.0. This guide
9-
needs to be updated for Kubeflow 1.1.
10-
{{% /alert %}}
117

128
This page describes how to integrate the type information in the pipeline and utilize the
139
static type checking for fast development iterations.
@@ -37,6 +33,7 @@ In the component YAML, types are specified as a string or a dictionary with the
3733
"*component a*" expects an input with Integer type and emits three outputs with the type GCSPath, customized_type and GCRPath.
3834
Among these types, Integer, GCSPath, and GCRPath are core types that are predefined in the SDK while customized_type is a user-defined
3935
type.
36+
4037
```yaml
4138
name: component a
4239
description: component desc
@@ -58,55 +55,80 @@ implementation:
5855
field_n: /feature.txt
5956
field_o: /output.txt
6057
```
58+
6159
Similarly, when you write a component with the decorator, you can annotate I/O with types in the function signature, as shown below.
6260
6361
```python
6462
from kfp.dsl import component
6563
from kfp.dsl.types import Integer, GCRPath
64+
65+
6666
@component
67-
def task_factory_a(field_l: Integer()) -> {'field_m': {'GCSPath': {'openapi_schema_validator': '{"type": "string", "pattern": "^gs://.*$"}'}},
68-
'field_n': 'customized_type',
69-
'field_o': GCRPath()
70-
}:
71-
return ContainerOp(
72-
name = 'operator a',
73-
image = 'gcr.io/ml-pipeline/component-a',
74-
command = [
75-
'python3', '/pipelines/component/src/train.py'
76-
],
77-
arguments = [
78-
'--field-l', field_l,
79-
],
80-
file_outputs = {
81-
'field_m': '/schema.txt',
82-
'field_n': '/feature.txt',
83-
'field_o': '/output.txt'
67+
def task_factory_a(field_l: Integer()) -> {
68+
'field_m': {
69+
'GCSPath': {
70+
'openapi_schema_validator':
71+
'{"type": "string", "pattern": "^gs://.*$"}'
8472
}
85-
)
73+
},
74+
'field_n': 'customized_type',
75+
'field_o': GCRPath()
76+
}:
77+
return ContainerOp(
78+
name='operator a',
79+
image='gcr.io/ml-pipeline/component-a',
80+
command=['python3', '/pipelines/component/src/train.py'],
81+
arguments=[
82+
'--field-l',
83+
field_l,
84+
],
85+
file_outputs={
86+
'field_m': '/schema.txt',
87+
'field_n': '/feature.txt',
88+
'field_o': '/output.txt'
89+
})
8690
```
91+
8792
You can also annotate pipeline inputs with types and the input are checked against the component I/O types as well. For example,
93+
8894
```python
8995
@component
90-
def task_factory_a(field_m: {'GCSPath': {'openapi_schema_validator': '{"type": "string", "pattern": "^gs://.*$"}'}}, field_o: 'Integer'):
91-
return ContainerOp(
92-
name = 'operator a',
93-
image = 'gcr.io/ml-pipeline/component-a',
94-
arguments = [
95-
'--field-l', field_m,
96-
'--field-o', field_o,
97-
],
98-
)
96+
def task_factory_a(
97+
field_m: {
98+
'GCSPath': {
99+
'openapi_schema_validator':
100+
'{"type": "string", "pattern": "^gs://.*$"}'
101+
}
102+
}, field_o: 'Integer'):
103+
return ContainerOp(
104+
name='operator a',
105+
image='gcr.io/ml-pipeline/component-a',
106+
arguments=[
107+
'--field-l',
108+
field_m,
109+
'--field-o',
110+
field_o,
111+
],
112+
)
113+
99114

100115
# Pipeline input types are also checked against the component I/O types.
101-
@dsl.pipeline(name='type_check',
102-
description='')
103-
def pipeline(a: {'GCSPath': {'openapi_schema_validator': '{"type": "string", "pattern": "^gs://.*$"}'}}='good', b: Integer()=12):
104-
task_factory_a(field_m=a, field_o=b)
116+
@dsl.pipeline(name='type_check', description='')
117+
def pipeline(
118+
a: {
119+
'GCSPath': {
120+
'openapi_schema_validator':
121+
'{"type": "string", "pattern": "^gs://.*$"}'
122+
}
123+
} = 'good',
124+
b: Integer() = 12):
125+
task_factory_a(field_m=a, field_o=b)
126+
105127

106128
try:
107-
compiler.Compiler().compile(pipeline, 'pipeline.tar.gz', type_check=True)
129+
compiler.Compiler().compile(pipeline, 'pipeline.tar.gz', type_check=True)
108130
except InconsistentTypeException as e:
109-
print(e)
131+
print(e)
110132
```
111133

112134
## How does the type checking work?
@@ -127,26 +149,35 @@ If inconsistent types are detected, it throws an [InconsistentTypeException](htt
127149
Type checking is enabled by default and it can be disabled in two ways:
128150

129151
If you compile the pipeline programmably:
152+
130153
```python
131154
compiler.Compiler().compile(pipeline_a, 'pipeline_a.tar.gz', type_check=False)
132155
```
156+
133157
If you compile the pipeline using the dsl-compiler tool:
158+
134159
```bash
135160
dsl-compiler --py pipeline.py --output pipeline.zip --disable-type-check
136161
```
162+
137163
### Fine-grained configuration
138164

139165
Sometimes, you might want to enable the type checking but disable certain arguments. For example,
140166
when the upstream component generates an output with type "*Float*" and the downstream can ingest either
141167
"*Float*" or "*Integer*", it might fail if you define the type as "*Float_or_Integer*".
142168
Disabling the type checking per-argument is also supported as shown below.
169+
143170
```python
144-
@dsl.pipeline(name='type_check_a',
145-
description='')
171+
@dsl.pipeline(name='type_check_a', description='')
146172
def pipeline():
147-
a = task_factory_a(field_l=12)
148-
# For each of the arguments, you can also ignore the types by calling ignore_type function.
149-
b = task_factory_b(field_x=a.outputs['field_n'], field_y=a.outputs['field_o'], field_z=a.outputs['field_m'].ignore_type())
173+
a = task_factory_a(field_l=12)
174+
# For each of the arguments, you can also ignore the types by calling
175+
# ignore_type function.
176+
b = task_factory_b(
177+
field_x=a.outputs['field_n'],
178+
field_y=a.outputs['field_o'],
179+
field_z=a.outputs['field_m'].ignore_type())
180+
150181
compiler.Compiler().compile(pipeline, 'pipeline.tar.gz', type_check=True)
151182
```
152183

@@ -158,4 +189,6 @@ type checking would still fail if some I/Os lack the type information and some I
158189

159190
## Next steps
160191

161-
* See [type checking sample](https://github.com/kubeflow/pipelines/blob/master/samples/core/dsl_static_type_checking/dsl_static_type_checking.ipynb).
192+
Learn how to define a KubeFlow pipeline with Python DSL and compile the
193+
pipeline with type checking: a
194+
[Jupyter notebook demo](https://github.com/kubeflow/pipelines/blob/master/samples/core/dsl_static_type_checking/dsl_static_type_checking.ipynb).

0 commit comments

Comments
 (0)