Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[v3] Add validator in render v2. #5942

Merged
merged 2 commits into from
Jun 7, 2021

Conversation

yuwenma
Copy link
Contributor

@yuwenma yuwenma commented Jun 4, 2021

Fixes: #nnn
Related: #5673
Merge after: #5940

Description

  • Add the render validator and write the skaffold validation rules to Kptfile pipeline.
  • Fixed corner cases when render.generate is not given in skaffold.yaml
  • More test coverage on Render configs.

To reviewers:
This PR is diff-based from 5940. Please only review the last commit

@yuwenma yuwenma added this to the v1.26.0 milestone Jun 4, 2021
@yuwenma yuwenma requested a review from a team as a code owner June 4, 2021 07:35
@google-cla google-cla bot added the cla: yes label Jun 4, 2021
@codecov
Copy link

codecov bot commented Jun 4, 2021

Codecov Report

Merging #5942 (67fe089) into master (f30fcc0) will increase coverage by 0.02%.
The diff coverage is 68.57%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #5942      +/-   ##
==========================================
+ Coverage   70.57%   70.59%   +0.02%     
==========================================
  Files         457      458       +1     
  Lines       17531    17564      +33     
==========================================
+ Hits        12372    12399      +27     
- Misses       4244     4250       +6     
  Partials      915      915              
Impacted Files Coverage Δ
pkg/skaffold/render/renderer/renderer.go 50.87% <60.71%> (+9.70%) ⬆️
pkg/skaffold/render/validate/validate.go 100.00% <100.00%> (ø)
pkg/skaffold/server/v2/endpoints.go 41.02% <0.00%> (-1.08%) ⬇️
pkg/skaffold/docker/parse.go 86.19% <0.00%> (-0.96%) ⬇️
pkg/skaffold/event/v2/event.go 73.33% <0.00%> (-0.71%) ⬇️
pkg/skaffold/docker/image.go 79.53% <0.00%> (+1.39%) ⬆️
pkg/skaffold/util/tar.go 56.00% <0.00%> (+5.33%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f30fcc0...67fe089. Read the comment docs.

@yuwenma yuwenma force-pushed the render-4-validate branch from 4c48902 to 1347f8c Compare June 4, 2021 19:19
tejal29
tejal29 previously approved these changes Jun 4, 2021
@tejal29 tejal29 dismissed their stale review June 4, 2021 21:22

wrong button click

@pull-request-size pull-request-size bot added size/L and removed size/XL labels Jun 4, 2021
Copy link
Contributor

@tejal29 tejal29 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

general LGTM.
Few qeustions

cmd := exec.CommandContext(ctx, "kpt", "pkg", "init", outputPath)
kptFilePath := filepath.Join(r.hydrationDir, kptfile.KptFileName)
if _, err := os.Stat(kptFilePath); os.IsNotExist(err) {
cmd := exec.CommandContext(ctx, "kpt", "pkg", "init", r.hydrationDir)
if _, err := util.RunCmdOut(cmd); err != nil {
return err
Copy link
Contributor

@tejal29 tejal29 Jun 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a kpt user error ? Can we define function and error codes like

func userErr(err error) error {
and user kpt user error elsewhere.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't guarantee it is a user error or not. I updated the err message with user actions.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to identify all possible if this was a kpt pkg init error or user config error?
Does kpt return unique exit codes based on different errors?
We are trying to attach an error code for all cases. You can also define KPT_PKG_INIT as an error code and return here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's a good point. I'm not sure if kpt returns the different exit codes or not. We may need more user data to know how likely a user config error may happen in kpt pkg init or kpt fn

defer file.Close()
kfConfig := &kptfile.KptFile{}
if err := yaml.NewDecoder(file).Decode(&kfConfig); err != nil {
return fmt.Errorf("unable to parse %v\n, please check if the kptfile is updated to new apiVersion > v1alpha2",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for creating an actionable error. Can you use sErrors.New() and add error code as well as suggestion code to this error.
Docs here
https://github.com/GoogleContainerTools/skaffold/blob/master/DEVELOPMENT.md#adding-actionable-error-messages-to-code

The advantage of this this is, IDEs can consume the suggestion code and decide to show an actionable error message to the user as is or relevant one.

Copy link
Contributor Author

@yuwenma yuwenma Jun 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, if the Decode fails, it basically means the yaml is invalid, it's not a skaffold error. the suggestion I added is just one possibility but not suggesting the error must be due to the API version difference. I'm afraid if using the suggestion code, it may mislead users. FYI, I attach the Decode error to the return value.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we still need an error code with an actionable error message KPT_INVALID_YAML or KPT_USER_ERROR so we can track these in skaffold and get metrics on it.
Right, if a skaffold dev fails, the error code logged via metrics pipeline will be DEPLOY_UNKNOWN. We are trying to reduce these overall.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. I added some TODO comments (here and other errors in ./render). Will add the Suggestion and error code in the following CL.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tejal29 Here's the proto PR #5953

@yuwenma yuwenma force-pushed the render-4-validate branch from 0835455 to d602906 Compare June 4, 2021 23:14
@pull-request-size pull-request-size bot added size/XL and removed size/L labels Jun 4, 2021
@yuwenma
Copy link
Contributor Author

yuwenma commented Jun 4, 2021

Updated. Please take another look @tejal29

There are two approaches to read Kptfile. One is via templating which we can add comments to help users understand the Kptfile. The other is via go struct which is more accurate and less error-prone. Considering the reliability of the Kpt deprecation policies (the api schema is not expected to change and if it is changed it will be backward compatible. If not backward compatible, the deprecation has a year-long period)
@yuwenma yuwenma force-pushed the render-4-validate branch from d602906 to e6360ce Compare June 4, 2021 23:15
@pull-request-size pull-request-size bot added size/L and removed size/XL labels Jun 4, 2021
@yuwenma yuwenma force-pushed the render-4-validate branch from e6360ce to e7bc1ee Compare June 4, 2021 23:32
@tejal29 tejal29 added the kokoro:force-run forces a kokoro re-run on a PR label Jun 7, 2021
@kokoro-team kokoro-team removed the kokoro:force-run forces a kokoro re-run on a PR label Jun 7, 2021
@tejal29 tejal29 merged commit 903acf3 into GoogleContainerTools:master Jun 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants