Skip to content

Add bash script that extracts images from manifests repо #2419

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

Add bash script that extracts images from manifests repо #2419

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
995ce7a
Add a script that extracts all images used by main KF components
difince Mar 23, 2023
4d7507f
Add "../contrib/bentoml" dir to extract files from
difince Mar 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
117 changes: 117 additions & 0 deletions docs/kf_images.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
busybox:1.28
busybox:latest
docker.io/aipipeline/api-server:1.5.1
docker.io/aipipeline/cache-server:1.5.1
docker.io/aipipeline/frontend:1.5.1
docker.io/aipipeline/metadata-writer:1.5.1
docker.io/aipipeline/persistenceagent:1.5.1
docker.io/aipipeline/pipelineloop-controller:1.5.1
docker.io/aipipeline/pipelineloop-controller:nightly
docker.io/aipipeline/pipelineloop-webhook:1.5.1
docker.io/aipipeline/pipelineloop-webhook:nightly
docker.io/aipipeline/scheduledworkflow:1.5.1
docker.io/istio/pilot:1.16.0
docker.io/istio/proxyv2:1.16.0
docker.io/kubeflowkatib/cert-generator
docker.io/kubeflowkatib/cert-generator:v0.15.0-rc.0
docker.io/kubeflowkatib/enas-cnn-cifar10-cpu:v0.15.0-rc.0
docker.io/kubeflowkatib/katib-controller
docker.io/kubeflowkatib/katib-controller:v0.15.0-rc.0
docker.io/kubeflowkatib/katib-db-manager
docker.io/kubeflowkatib/katib-db-manager:v0.15.0-rc.0
docker.io/kubeflowkatib/katib-ui
docker.io/kubeflowkatib/katib-ui:v0.15.0-rc.0
docker.io/kubeflowkatib/mxnet-mnist:v0.15.0-rc.0
docker.io/kubeflowkatib/pytorch-mnist-cpu:v0.15.0-rc.0
docker.io/kubeflownotebookswg/centraldashboard:v1.7.0-rc.0
docker.io/kubeflownotebookswg/jupyter-web-app:v1.7.0-rc.0
docker.io/kubeflownotebookswg/kfam:v1.7.0-rc.0
docker.io/kubeflownotebookswg/notebook-controller
docker.io/kubeflownotebookswg/notebook-controller:v1.7.0-rc.0
docker.io/kubeflownotebookswg/poddefaults-webhook:v1.7.0-rc.0
docker.io/kubeflownotebookswg/profile-controller
docker.io/kubeflownotebookswg/profile-controller:v1.7.0-rc.0
docker.io/kubeflownotebookswg/tensorboard-controller
docker.io/kubeflownotebookswg/tensorboard-controller:v1.7.0-rc.0
docker.io/kubeflownotebookswg/tensorboards-web-app:v1.7.0-rc.0
docker.io/kubeflownotebookswg/volumes-web-app:v1.7.0-rc.0
docker.io/metacontrollerio/metacontroller:v2.0.4
docker.io/seldonio/mlserver:1.0.0
docker.io/seldonio/seldon-core-operator:1.15.0
gcr.io/arrikto/kubeflow/oidc-authservice:e236439
gcr.io/cloudsql-docker/gce-proxy:1.25.0
gcr.io/knative-releases/knative.dev/eventing/cmd/controller@sha256:33d78536e9b38dbb2ec2952207b48ff8e05acb48e7d28c2305bd0a0f7156198f
gcr.io/knative-releases/knative.dev/eventing/cmd/mtping@sha256:282b5265e1ef26309b3343038c9b4f172654e06cbee46f6ddffd23ea9ad9a3be
gcr.io/knative-releases/knative.dev/eventing/cmd/webhook@sha256:d217ab7e3452a87f8cbb3b45df65c98b18b8be39551e3e960cd49ea44bb415ba
gcr.io/knative-releases/knative.dev/net-istio/cmd/controller@sha256:2b484d982ef1a5d6ff93c46d3e45f51c2605c2e3ed766e20247d1727eb5ce918
gcr.io/knative-releases/knative.dev/net-istio/cmd/webhook@sha256:59b6a46d3b55a03507c76a3afe8a4ee5f1a38f1130fd3d65c9fe57fff583fa8d
gcr.io/knative-releases/knative.dev/pkg/apiextensions/storageversion/cmd/migrate@sha256:b6a9bb1c500669410d1ec2fef88af0de011375e619c335973a2894f109135858
gcr.io/knative-releases/knative.dev/pkg/apiextensions/storageversion/cmd/migrate@sha256:bb95337552b07e1a52d6205a95342317b1b081b6b57ef7764c0d59c466978b6f
gcr.io/knative-releases/knative.dev/serving/cmd/activator@sha256:c3bbf3a96920048869dcab8e133e00f59855670b8a0bbca3d72ced2f512eb5e1
gcr.io/knative-releases/knative.dev/serving/cmd/autoscaler@sha256:caae5e34b4cb311ed8551f2778cfca566a77a924a59b775bd516fa8b5e3c1d7f
gcr.io/knative-releases/knative.dev/serving/cmd/controller@sha256:38f9557f4d61ec79cc2cdbe76da8df6c6ae5f978a50a2847c22cc61aa240da95
gcr.io/knative-releases/knative.dev/serving/cmd/domain-mapping@sha256:763d648bf1edee2b4471b0e211dbc53ba2d28f92e4dae28ccd39af7185ef2c96
gcr.io/knative-releases/knative.dev/serving/cmd/domain-mapping-webhook@sha256:a4ba0076df2efaca2eed561339e21b3a4ca9d90167befd31de882bff69639470
gcr.io/knative-releases/knative.dev/serving/cmd/queue@sha256:505179c0c4892ea4a70e78bc52ac21b03cd7f1a763d2ecc78e7bbaa1ae59c86c
gcr.io/knative-releases/knative.dev/serving/cmd/webhook@sha256:bc13765ba4895c0fa318a065392d05d0adc0e20415c739e0aacb3f56140bf9ae
gcr.io/kubebuilder/kube-rbac-proxy:v0.13.1
gcr.io/kubebuilder/kube-rbac-proxy:v0.8.0
gcr.io/kubeflow-images-public/kubebench/kubebench-operator-v1alpha2:bc682c1
gcr.io/kubeflow-images-public/kubebench/workflow-agent:bc682c1
gcr.io/ml-pipeline/api-server:2.0.0-alpha.7
gcr.io/ml-pipeline/application-crd-controller:1.0-beta-non-cluster-role
gcr.io/ml-pipeline/cache-deployer:1.8.4
gcr.io/ml-pipeline/cache-deployer:2.0.0-alpha.7
gcr.io/ml-pipeline/cache-server:2.0.0-alpha.7
gcr.io/ml-pipeline/frontend:2.0.0-alpha.7
gcr.io/ml-pipeline/inverse-proxy-agent:1.8.4
gcr.io/ml-pipeline/inverse-proxy-agent:2.0.0-alpha.7
gcr.io/ml-pipeline/metadata-envoy:1.8.4
gcr.io/ml-pipeline/metadata-envoy:2.0.0-alpha.7
gcr.io/ml-pipeline/metadata-writer:1.8.4
gcr.io/ml-pipeline/metadata-writer:2.0.0-alpha.7
gcr.io/ml-pipeline/minio:RELEASE.2019-08-14T20-37-41Z-license-compliance
gcr.io/ml-pipeline/mysql:5.7-debian
gcr.io/ml-pipeline/mysql:8.0.26
gcr.io/ml-pipeline/persistenceagent:2.0.0-alpha.7
gcr.io/ml-pipeline/scheduledworkflow:2.0.0-alpha.7
gcr.io/ml-pipeline/viewer-crd-controller:1.8.4
gcr.io/ml-pipeline/viewer-crd-controller:2.0.0-alpha.7
gcr.io/ml-pipeline/visualization-server:1.8.4
gcr.io/ml-pipeline/visualization-server:2.0.0-alpha.7
gcr.io/ml-pipeline/workflow-controller:v3.3.8-license-compliance
gcr.io/tekton-releases/github.com/tektoncd/dashboard/cmd/dashboard:v0.30.0@sha256:85f7d38086fadb07556052ce873d44861c29ef690f47735f32d7e6a153ca8a92
gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/controller:v0.41.0@sha256:556953d6367b28504b7ad32f58a50b3e3609f60aaddfca3aad217e93465551e7
gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/resolvers:v0.41.0@sha256:de08fa01e521144d9852dd14fe64f75da0b471c7379b0f721043f69fc86a8647
gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/webhook:v0.41.0@sha256:f4e766d21b0ea2735f487888c0155c9d8287f04ac77a4948a616250d24175475
gcr.io/tfx-oss-public/ml_metadata_store_server:1.5.0
ghcr.io/dexidp/dex:v2.31.2
grafana/grafana:5.3.4
kserve/kserve-controller:v0.10.0
kserve/lgbserver:v0.10.0
kserve/models-web-app:v0.10.0
kserve/paddleserver:v0.10.0
kserve/pmmlserver:v0.10.0
kserve/sklearnserver:v0.10.0
kserve/xgbserver:v0.10.0
kubeflow/training-operator
kubeflow/training-operator:v1-d0fb5c0
minio/minio
mysql:8
mysql:8.0.29
mysql:8.0.3
nvcr.io/nvidia/tritonserver:21.09-py3
postgres:12-alpine
postgres:14.5-alpine
prom/prometheus
python:3.7
pytorch/torchserve-kfs:0.7.0
quay.io/argoproj/argocli:latest
quay.io/argoproj/workflow-controller:latest
quay.io/bentoml/yatai-deployment:1.1.4
quay.io/bentoml/yatai-image-builder:1.1.3
quay.io/dexidp/dex:v2.23.0
quay.io/jetstack/cert-manager-cainjector:v1.10.1
quay.io/jetstack/cert-manager-controller:v1.10.1
quay.io/jetstack/cert-manager-webhook:v1.10.1
tensorflow/serving:2.6.2
32 changes: 32 additions & 0 deletions hack/extract_manifests_images.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#!/usr/bin/env bash

# The script extracts all images from specific manifests sub-directories.
# Future release process enhancements may include an automatic image inventory scan.
# The reported image list can later be used for image vulnerability scanning and managing license risks management

#VERSION=1.7.0
output_file="../docs/kf${VERSION}_images.txt"
declare -a dirs=("../apps" "../common" "../example" "../contrib/metacontroller"
"../contrib/seldon" "../contrib/bentoml" )
rm -f .tmp
# Iterate over all files with names: 'kustomization.yaml', 'kustomization.yml', 'Kustomization' found recursively in the provided list of directories
for F in $(find "${dirs[@]}" \( -name kustomization.yaml -o -name kustomization.yml -o -name Kustomization \)); do

dir=$(dirname -- "$F")
# Generate k8s resources specified in 'dir' using the 'kustomize build' command.
# Log the 'dir' name where the 'kustomize build' command fails.
kbuild=$(kustomize build "$dir")
return_code=$?
if [ $return_code -ne 0 ]; then
printf 'ERROR:\t Failed \"kustomize build\" command for directory: %s. See error above\n' "$dir"
continue
fi
# Grep the output of 'kustomize build' command for 'image:' and '- image' lines and return just the image itself
# Redirect the output to '.tmp' file
grep '\-\?\s\image:'<<<"$kbuild" | sed -re 's/\s-?\simage: *//;s/^[ \t]*//g' | sed '/^$/d;/{/d' >> .tmp
done

sort .tmp | uniq > "$output_file"
rm -f .tmp

echo "File ${output_file} successfully created"