Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: slsa-framework/[email protected]: add testdata #758

Merged
merged 3 commits into from
Apr 23, 2024
Merged

chore: slsa-framework/[email protected]: add testdata #758

merged 3 commits into from
Apr 23, 2024

Conversation

ramonpetgrave64
Copy link
Contributor

@ramonpetgrave64 ramonpetgrave64 commented Apr 23, 2024 •
edited
Loading

slsa-framework/slsa-github-generator#3576

Next step in
https://github.com/slsa-framework/slsa-github-generator/blob/main/RELEASE.md#update-verifier

Creating new test data for [email protected]

Instructions:

diff to download-artifacts.sh

diff --git a/download-artifacts.sh b/download-artifacts.sh
old mode 100644
new mode 100755
index e5e218e8..49257ea6
--- a/download-artifacts.sh
+++ b/download-artifacts.sh
@@ -88,6 +88,10 @@ unzip_files() {
         rm -rf "${tmp_dir}"
         ;;
 
+    ./*.zip)
+        unzip -o "${zip_path}" -d "${output_path}"
+        ;;
+
     *)
         echo "unexpected file path: ${zip_path}"
         exit 1
@@ -167,7 +171,7 @@ rename_java_files "test-java-project-" "maven"
 rename_java_files "workflow_dispatch-" "gradle"
 
 # Files downloaded. Now copy them
-repo_path="../.."
+repo_path="/path/to/slsa-verifier"
 
 # Go builder files.
 copy_files "gha_go-binary-linux-amd64-" "${repo_path}/cli/slsa-verifier/testdata/gha_go/${version}"

download the artifacts

../slsa-verifier/download-artifacts.sh 8791212155 v2.0.0
../slsa-verifier/download-artifacts.sh 8791219359 v2.0.0
../slsa-verifier/download-artifacts.sh 8791219514 v2.0.0
../slsa-verifier/download-artifacts.sh 8791219607 v2.0.0

docker github auth

gh auth login --scopes=read:packages
echo `gh auth token` | docker login ghcr.io -u ramonpetgrave64 --password-stdin
cosign save \
    --dir ./cli/slsa-verifier/testdata/gha_generic_container/v2.0.0/container_workflow_dispatch \
    ghcr.io/slsa-framework/example-package.verifier-e2e.all.tag.main.default.slsa3@sha256:55aee984fd6b1d0e0a19a55265d10d40063a2212bdbabd75b202b1728236548d

@ramonpetgrave64
Copy link
Contributor Author

ramonpetgrave64 commented Apr 23, 2024 •
edited
Loading

Test failing
https://github.com/slsa-framework/slsa-verifier/actions/runs/8801653756/job/24155532842?pr=758#step:6:6

2024-04-23T14:07:06.0587137Z Verified build using builder "https://github.com/slsa-framework/slsa-github-generator/.github/workflows/generator_container_slsa3.yml@refs/tags/v1.9.0" at commit 2bcaa7495e1cbd11fbd4f598d857b3a6f18df933
2024-04-23T14:07:06.0588248Z --- FAIL: Test_runVerifyGHAArtifactImage (0.00s)
2024-04-23T14:07:06.0589075Z     --- FAIL: Test_runVerifyGHAArtifactImage/versioned_tag_no_match_empty_tag_workflow_dispatch_>_v1.9.0 (0.11s)
2024-04-23T14:07:06.0590280Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0591552Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0592595Z             + 	e"tag used to generate the binary does not match provenance",
2024-04-23T14:07:06.0593057Z               )
2024-04-23T14:07:06.0593410Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0594628Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0595643Z             + 	e"tag used to generate the binary does not match provenance",
2024-04-23T14:07:06.0596275Z               )
2024-04-23T14:07:06.0596608Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0597812Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0598825Z             + 	e"tag used to generate the binary does not match provenance",
2024-04-23T14:07:06.0599256Z               )
2024-04-23T14:07:06.0599595Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0600780Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0601796Z             + 	e"tag used to generate the binary does not match provenance",
2024-04-23T14:07:06.0602235Z               )
2024-04-23T14:07:06.0602726Z     --- FAIL: Test_runVerifyGHAArtifactImage/valid_main_branch_default (0.43s)
2024-04-23T14:07:06.0603295Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0604502Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0605250Z               )
2024-04-23T14:07:06.0605568Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0606747Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0607480Z               )
2024-04-23T14:07:06.0607799Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0608982Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0609714Z               )
2024-04-23T14:07:06.0610174Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0611367Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0612102Z               )
2024-04-23T14:07:06.0612719Z     --- FAIL: Test_runVerifyGHAArtifactImage/tag_no_match_empty_tag_workflow_dispatch_>_v1.9.0 (0.12s)
2024-04-23T14:07:06.0613367Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0614543Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0615546Z             + 	e"tag used to generate the binary does not match provenance",
2024-04-23T14:07:06.0615975Z               )
2024-04-23T14:07:06.0616326Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0617525Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0618545Z             + 	e"tag used to generate the binary does not match provenance",
2024-04-23T14:07:06.0618980Z               )
2024-04-23T14:07:06.0619307Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0620496Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0621497Z             + 	e"tag used to generate the binary does not match provenance",
2024-04-23T14:07:06.0621926Z               )
2024-04-23T14:07:06.0622252Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0623429Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0624434Z             + 	e"tag used to generate the binary does not match provenance",
2024-04-23T14:07:06.0625117Z               )
2024-04-23T14:07:06.0625598Z     --- FAIL: Test_runVerifyGHAArtifactImage/wrong_branch_master (0.58s)
2024-04-23T14:07:06.0626149Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0627514Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0628542Z             + 	e"branch used to generate the binary does not match provenance",
2024-04-23T14:07:06.0628986Z               )
2024-04-23T14:07:06.0629314Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0630504Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0631527Z             + 	e"branch used to generate the binary does not match provenance",
2024-04-23T14:07:06.0631974Z               )
2024-04-23T14:07:06.0632299Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0633487Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0634516Z             + 	e"branch used to generate the binary does not match provenance",
2024-04-23T14:07:06.0634967Z               )
2024-04-23T14:07:06.0635293Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0636473Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0637498Z             + 	e"branch used to generate the binary does not match provenance",
2024-04-23T14:07:06.0637941Z               )
2024-04-23T14:07:06.0638404Z     --- FAIL: Test_runVerifyGHAArtifactImage/valid_main_branch_set (0.52s)
2024-04-23T14:07:06.0638951Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0640306Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0641055Z               )
2024-04-23T14:07:06.0641381Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0642599Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0643335Z               )
2024-04-23T14:07:06.0643654Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0644836Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0645566Z               )
2024-04-23T14:07:06.0645882Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0647058Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0647794Z               )

@ramonpetgrave64
add testdata
b8f6ecd 
Signed-off-by: Ramon Petgrave <[email protected]>
@ramonpetgrave64
revert from main cli/slsa-verifier/testdata/gha_generic_container/v1.…
a82387e 
…10.0/container_workflow_dispatch.digest

Signed-off-by: Ramon Petgrave <[email protected]>
@ramonpetgrave64
fix the hash
ac533a9 
Signed-off-by: Ramon Petgrave <[email protected]>
@ramonpetgrave64
Copy link
Contributor Author

Now passing. I had recorded the incorrect hash.

@ramonpetgrave64 ramonpetgrave64 marked this pull request as ready for review April 23, 2024 14:49
@ramonpetgrave64
Copy link
Contributor Author

@laurentsimon @kpk47

laurentsimon
laurentsimon approved these changes Apr 23, 2024
View reviewed changes
Copy link
Contributor

@laurentsimon laurentsimon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thanks!

@ramonpetgrave64 ramonpetgrave64 merged commit 637b07f into slsa-framework:main Apr 23, 2024
15 checks passed
ramonpetgrave64 added a commit to ramonpetgrave64/slsa-verifier that referenced this pull request Apr 25, 2024
@ramonpetgrave64
Revert "chore: slsa-framework/[email protected]: add testd…
e6d63c6 
…ata (slsa-framework#758)"

This reverts commit 637b07f.
@ramonpetgrave64 ramonpetgrave64 mentioned this pull request Feb 25, 2025
Merged
ramonpetgrave64 added a commit that referenced this pull request Feb 27, 2025
@ramonpetgrave64
chore: update test files for v2.1.0 (#836)
b53bd94 
Similar to #758, we are updating the test files.

Errors for checking the tag in attestations are slightly different. Unit
tests are adjusted with the new test cases.

---------

Signed-off-by: Ramon Petgrave <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@laurentsimon laurentsimon laurentsimon approved these changes

@kpk47 kpk47 Awaiting requested review from kpk47

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ramonpetgrave64 @laurentsimon