Skip to content

Feat/snyk timeout secs #4942

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Nov 27, 2023
Merged

Feat/snyk timeout secs #4942

merged 2 commits into from
Nov 27, 2023

Conversation

cmars
Copy link
Contributor

@cmars cmars commented Nov 22, 2023

Pull Request Submission

Please check the boxes once done.

The pull request must:

  • Reviewer Documentation
    • follow CONTRIBUTING rules
    • be accompanied by a detailed description of the changes
    • contain a risk assessment of the change (Low | Medium | High) with regards to breaking existing functionality. A change e.g. of an underlying language plugin can completely break the functionality for that language, but appearing as only a version change in the dependencies.
    • highlight breaking API if applicable
    • contain a link to the automatic tests that cover the updated functionality.
    • contain testing instructions in case that the reviewer wants to manual verify as well, to add to the manual testing done by the author.
    • link to the link to the PR for the User-facing documentation
  • User facing Documentation
    • update any relevant documentation in gitbook by submitting a gitbook PR, and including the PR link here
    • ensure that the message of the final single commit is descriptive and prefixed with either feat: or fix: , others might be used in rare occasions as well, if there is no need to document the changes in the release notes. The changes or fixes should be described in detail in the commit message for the changelog & release notes.
  • Testing
    • Changes, removals and additions to functionality must be covered by acceptance / integration tests or smoke tests - either already existing ones, or new ones, created by the author of the PR.

Pull Request Review

All pull requests must undergo a thorough review process before being merged.
The review process of the code PR should include code review, testing, and any necessary feedback or revisions.
Pull request reviews of functionality developed in other teams only review the given documentation and test reports.

Manual testing will not be performed by the reviewing team, and is the responsibility of the author of the PR.

For Node projects: It’s important to make sure changes in package.json are also affecting package-lock.json correctly.

If a dependency is not necessary, don’t add it.

When adding a new package as a dependency, make sure that the change is absolutely necessary. We would like to refrain from adding new dependencies when possible.
Documentation PRs in gitbook are reviewed by Snyk's content team. They will also advise on the best phrasing and structuring if needed.

Pull Request Approval

Once a pull request has been reviewed and all necessary revisions have been made, it is approved for merging into
the main codebase. The merging of the code PR is performed by the code owners, the merging of the documentation PR
by our content writers.

What does this PR do?

Terminate CLI with exit code EX_UNAVAILABLE after a timeout, when set
using SNYK_TIMEOUT_SECS=N environment variable, where N is the number of
seconds the user is willing to wait for the command to complete.

Where should the reviewer start?

setTimeout function added to main.go

How should this be manually tested?

Build the CLI with make clean build. Test it with:

SNYK_TIMEOUT_SECS=1 ./binary-releases/snyk-macos-arm64 test

1 second is usually not long enough for snyk test, so you should see the CLI terminate with exit code 69 (EX_UNAVAILABLE).

Testing CI mode:

  • Test normal operation, should exit 0:
    • SNYK_CI=1 snyk test
    • SNYK_CI=1 snyk container test alpine
    • SNYK_CI=1 snyk code test
    • SNYK_CI=1 snyk iac test
  • Test network partitions by setting SNYK_API=http://localhost. With or without SNYK_CI=1, you should get exit code 2.
  • Test service unavailable by running a local HTTP server which always responds 500 on localhost:9000. SNYK_API=http://localhost:9000 should exit code 2, with or without SNYK_CI=1.
  • Test "unauthorized" by setting SNYK_TOKEN=nope in the above commands. With CI mode, you should get exit code 77 (EX_NOPERM), not 2. You can also use a token that doesn't have necessary entitlement (like code). You should get a 69 (EX_UNAVAILABLE) or 77 (EX_NOPERM), not 2.
  • Test client errors by running a local HTTP server which always responds 400, 404, 422, etc. on localhost:9000. SNYK_API=http://localhost:9000 should exit code 69 (EX_UNAVAILABLE) with SNYK_CI=1, not 2.

Note that without SNYK_CI=1, the above test cases should all exit code 2.

Any background context you want to provide?

Some CI/CD use cases run Snyk test with "best effort". Aborting the command after a maximum duration provides assurance that Snyk does not hang the CI/CD pipeline.

This PR is stacked on #4941 so it may be helpful to review the last commit in isolation.

Still a draft because:

What are the relevant tickets?

Screenshots

Additional questions

Copy link
Contributor

github-actions bot commented Nov 22, 2023

Warnings
⚠️

Since the CLI is unifying on a standard and improved tooling, we're starting to migrate old-style imports and exports to ES6 ones.
A file you've modified is using either module.exports or require(). If you can, please update them to ES6 import syntax and export syntax.
Files found:

  • src/cli/main.ts
⚠️

You've modified files in src/ directory, but haven't updated anything in test folder. Is there something that could be tested?

Generated by 🚫 dangerJS against c8c9a45

@cmars cmars force-pushed the feat/snyk_timeout_secs branch 2 times, most recently from 1728822 to 2b0677c Compare November 22, 2023 00:44
Copy link
Contributor

@bastiandoetsch bastiandoetsch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fips import (for fedramp) and go 1.21 (for the build pipeline) must be resolved before merging

@cmars cmars force-pushed the feat/snyk_timeout_secs branch 2 times, most recently from 2363e2d to 55816cf Compare November 27, 2023 14:56
@cmars cmars marked this pull request as ready for review November 27, 2023 14:58
@cmars cmars requested review from a team as code owners November 27, 2023 14:58
@cmars cmars requested review from robh-snyk and gitphill November 27, 2023 14:58
Temporary failures might be treated differently from permanent failures
in a CI context.

A temporary network partition might be retried with an exponential
backoff delay in a CI script.

Such a temporary failure might even be skipped in the CI workflow, with
assurance that security testing will be covered by Snyk monitors and
reporting.

This option gives more flexibility and self-determination in choosing
security posture. In some cases landing a critical fix is more important
than waiting for a network partition or temporary outage to resolve.

This CI mode may be opted into by setting an environment variable SNYK_CI=1
when running the CLI. CI mode causes the following errors to terminate
with a different exit code:

- Authorization errors, including entitlement errors (HTTP 401 and 403),
  will exit code 77 (EX_NOPERM)
- Other non-recoverable errors will exit code 69 (EX_UNAVAILABLE). This
  includes all other HTTP 4xx errors from Snyk APIs.

Recoverable errors (network connection errors, HTTP 5xx) will exit code 2.

The difference between CI and non-CI mode, is that the above errors will
normally exit code 2.
@cmars cmars force-pushed the feat/snyk_timeout_secs branch from 55816cf to b638a59 Compare November 27, 2023 15:12
Terminate CLI with exit code EX_UNAVAILABLE after a timeout, when set
using SNYK_TIMEOUT_SECS=N environment variable, where N is the number of
seconds the user is willing to wait for the command to complete.
@cmars cmars force-pushed the feat/snyk_timeout_secs branch from b638a59 to c8c9a45 Compare November 27, 2023 16:11
@cmars cmars merged commit a0308f1 into master Nov 27, 2023
@cmars cmars deleted the feat/snyk_timeout_secs branch November 27, 2023 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants