Include shadowJar artifact to release workflow #8

GumpacG · 2023-07-12T21:47:55Z

Description

Included shadowJar artifact to release workflow and bumped to 1.4.0.1

Check List

New functionality includes testing.
- All tests pass, including unit test, integration test and doctest
New functionality has been documented.
- New functionality has javadoc added
- New functionality has user manual doc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Guian Gumpac <[email protected]>

.github/workflows/release-drafter.yml

.github/workflows/sql-jdbc-test-and-build-workflow.yml

Signed-off-by: Guian Gumpac <[email protected]>

Yury-Fridlyand

I'm worrying that shadow jar would be published on maven too...

MaxKsyunz

Do we need to increment the version at this point?

Changes to Jenkins are necessary before we can release 1.4.0.1 so it includes shadow jar.

GumpacG · 2023-07-12T23:20:14Z

@MaxKsyunz what would the process be like? Merge shadowJar workflow changes, then Change Jenkins, then bump version?

@Yury-Fridlyand do we not want shadowJar on Maven?

Yury-Fridlyand · 2023-07-12T23:28:41Z

No idea, it is not harmful though.
Right, we have to update jenkins/release.jenkinsFile too and add there a task to upload shadowjar to artifacts. See how it works on ODBC driver.

GumpacG · 2023-07-13T14:14:25Z

Based on the Jenkins file in https://github.com/opensearch-project/sql-jdbc/blob/6cc8e0a3c7933f13c58e32cbc6da3bdb9fb0d066/jenkins/release.jenkinsFile#L13C13-L13C59,
I think it signs everything that is in build/repository which is where the shadowJar is being moved to with the changes in the PR. Is there a way to test this without actually publishing to Maven?

Yury-Fridlyand · 2023-07-13T16:32:29Z

I think no.

MaxKsyunz · 2023-07-13T22:02:23Z

Having the shadow jar on Maven Central would be confusing -- if you are getting a dependency from Maven Central, you can just as easily get its dependencies and retain control of which versions are used. If there's a CVE in one of the dependencies, without shadow jar a client application can force the build to use another version of the dependency, with shadow jar we would need to release a new version of the shadow jar with the CVE fixed.

Regarding artifacts, the easiest to follow solution would be to save shadow jar as a separate artifact and have a separate stage in Jenkins job to process it.

standardReleasePipelineWithGenericTrigger can accept the name of the artifact to download. See here.

Signed-off-by: Guian Gumpac <[email protected]>

acarbonetto · 2023-07-17T16:03:46Z

jenkins/release.jenkinsFile

+            sigOverwrite: true
+        )
+        publishToArtifactsProdBucket(
+            assumedRoleName: 'sql-jdbc-shadowJar-upload-role',


they have a separate role for shadowJars?

I'm not actually too sure this works. I assumed that it was just a name for the jenkins job.
@param Map[assumedRoleName] <required> - IAM role to be assumed for uploading artifacts
Is from https://github.com/opensearch-project/opensearch-build-libraries/blob/de8ea9024a1a95a72c668562cbdd13065bf1ed46/vars/publishToArtifactsProdBucket.groovy#L12C1-L12C88 where that is being called.

Yury-Fridlyand · 2023-07-17T16:44:56Z

jenkins/release.jenkinsFile

@@ -14,4 +14,18 @@ standardReleasePipelineWithGenericTrigger(
            mavenArtifactsPath: "$WORKSPACE/repository/",
            autoPublish: true
        )
+        publishToArtifactsProdBucket(


why do you have 2 publishToArtifactsProdBucket for one jar?
The second one seems more correct

Not sure how it works if we specify this. As I understand artifacts.tar.gz is uploaded by default. I'm not sure if it ignores this file if publishToArtifactsProdBucket is called.

Yury-Fridlyand · 2023-07-17T16:45:38Z

jenkins/release.jenkinsFile

+        )
+        publishToArtifactsProdBucket(
+            assumedRoleName: 'sql-jdbc-shadowJar-upload-role',
+            source: "$WORKSPACE/build/libs/opensearch-sql-jdbc-shadow-*.jar",


Suggested change

source: "$WORKSPACE/build/libs/opensearch-sql-jdbc-shadow-*.jar",

source: "$WORKSPACE/opensearch-sql-jdbc-shadow-${tag}.jar",

build/libs/opensearch-sql-jdbc-shadow-*.jar would be the location of where the shadowJar is generated when it is built.

Yes, but jenkins gets it from release artifacts.

Ohhh I see. Thanks

Yury-Fridlyand · 2023-07-17T16:45:51Z

jenkins/release.jenkinsFile

+            assumedRoleName: 'sql-jdbc-shadowJar-upload-role',
+            source: "$WORKSPACE/build/libs/opensearch-sql-jdbc-shadow-*.jar",
+            destination: "https://artifacts.opensearch.org/opensearch-clients/jdbc/opensearch-sql-jdbc-shadow-${tag}.jar",
+            signingPlatform: 'windows',


Suggested change

signingPlatform: 'windows',

signingPlatform: 'linux',

Signed-off-by: Guian Gumpac <[email protected]>

…ct#106) * Include shadowJar artifact to release workflow (#8) * Test packaging tar with shadowJar Signed-off-by: Guian Gumpac <[email protected]> * Setting version as a variable Signed-off-by: Guian Gumpac <[email protected]> * Changed release-drafter to include shadowJar Signed-off-by: Guian Gumpac <[email protected]> * Bumped version to 1.4.0.1 Signed-off-by: Guian Gumpac <[email protected]> * Bumped version to 1.4.0.1 Signed-off-by: Guian Gumpac <[email protected]> * Simplified adding shadowJar to artifacts Signed-off-by: Guian Gumpac <[email protected]> * Reverted whitespace change Signed-off-by: Guian Gumpac <[email protected]> * Reverted whitespace change Signed-off-by: Guian Gumpac <[email protected]> * Create shadowJar as a separate artifact Signed-off-by: Guian Gumpac <[email protected]> * Fixed wrong version Signed-off-by: Guian Gumpac <[email protected]> * Added jenkins job to publish shadowJar to artifacts bucket Signed-off-by: Guian Gumpac <[email protected]> * Changed source and signing platform Signed-off-by: Guian Gumpac <[email protected]> --------- Signed-off-by: Guian Gumpac <[email protected]> * Separated shadowJar from repository directory Signed-off-by: Guian Gumpac <[email protected]> * Fixed tar issue with * Signed-off-by: Guian Gumpac <[email protected]> * Addressed PR comments Signed-off-by: Guian Gumpac <[email protected]> * Update jenkins version and release Signed-off-by: acarbonetto <[email protected]> --------- Signed-off-by: Guian Gumpac <[email protected]> Signed-off-by: acarbonetto <[email protected]> Co-authored-by: acarbonetto <[email protected]>

GumpacG added 30 commits July 11, 2023 10:59

Test packaging tar with shadowJar

ac1a325

Signed-off-by: Guian Gumpac <[email protected]>

Test packaging tar with shadowJar

209d23c

Signed-off-by: Guian Gumpac <[email protected]>

Test packaging tar with shadowJar

3cba794

Signed-off-by: Guian Gumpac <[email protected]>

Test packaging tar with shadowJar

7bd7b1a

Signed-off-by: Guian Gumpac <[email protected]>

Test packaging tar with shadowJar

396bd0e

Signed-off-by: Guian Gumpac <[email protected]>

Test packaging tar with shadowJar

1ccf819

Signed-off-by: Guian Gumpac <[email protected]>

Test packaging tar with shadowJar

ca7b8cc

Signed-off-by: Guian Gumpac <[email protected]>

Test packaging tar with shadowJar

450422f

Signed-off-by: Guian Gumpac <[email protected]>

Test packaging tar with shadowJar

58069fa

Signed-off-by: Guian Gumpac <[email protected]>

Test packaging tar with shadowJar

9dca6f7

Signed-off-by: Guian Gumpac <[email protected]>

Test packaging tar with shadowJar

085e221

Signed-off-by: Guian Gumpac <[email protected]>

Test packaging tar with shadowJar

89b1525

Signed-off-by: Guian Gumpac <[email protected]>

Test packaging tar with shadowJar

ca0e56f

Signed-off-by: Guian Gumpac <[email protected]>

Test packaging tar with shadowJar

1b91d55

Signed-off-by: Guian Gumpac <[email protected]>

Test packaging tar with shadowJar

796c993

Signed-off-by: Guian Gumpac <[email protected]>

Test packaging tar with shadowJar

2f891a3

Signed-off-by: Guian Gumpac <[email protected]>

Test packaging tar with shadowJar

96aa1e9

Signed-off-by: Guian Gumpac <[email protected]>

Setting version as a variable

f88ad17

Signed-off-by: Guian Gumpac <[email protected]>

Setting version as a variable

78078d0

Signed-off-by: Guian Gumpac <[email protected]>

Setting version as a variable

59a58ae

Signed-off-by: Guian Gumpac <[email protected]>

Setting version as a variable

61da296

Signed-off-by: Guian Gumpac <[email protected]>

Setting version as a variable

74b4f83

Signed-off-by: Guian Gumpac <[email protected]>

Setting version as a variable

e5f75d7

Signed-off-by: Guian Gumpac <[email protected]>

Setting version as a variable

6bb3dcb

Signed-off-by: Guian Gumpac <[email protected]>

Setting version as a variable

d9153c0

Signed-off-by: Guian Gumpac <[email protected]>

Setting version as a variable

2e30e14

Signed-off-by: Guian Gumpac <[email protected]>

Setting version as a variable

d4036d1

Signed-off-by: Guian Gumpac <[email protected]>

Setting version as a variable

e5e1637

Signed-off-by: Guian Gumpac <[email protected]>

Setting version as a variable

3a42837

Signed-off-by: Guian Gumpac <[email protected]>

Setting version as a variable

3d2059f

Signed-off-by: Guian Gumpac <[email protected]>

GumpacG added 5 commits July 12, 2023 13:29

Setting version as a variable

8e13534

Signed-off-by: Guian Gumpac <[email protected]>

Setting version as a variable

d6eea68

Signed-off-by: Guian Gumpac <[email protected]>

Changed release-drafter to include shadowJar

6e9c634

Signed-off-by: Guian Gumpac <[email protected]>

Bumped version to 1.4.0.1

02f198e

Signed-off-by: Guian Gumpac <[email protected]>

Bumped version to 1.4.0.1

03e6993

Signed-off-by: Guian Gumpac <[email protected]>

GumpacG requested review from MaxKsyunz, Yury-Fridlyand, forestmvey and acarbonetto as code owners July 12, 2023 21:47

Yury-Fridlyand reviewed Jul 12, 2023

View reviewed changes

.github/workflows/release-drafter.yml Outdated Show resolved Hide resolved

.github/workflows/sql-jdbc-test-and-build-workflow.yml Outdated Show resolved Hide resolved

GumpacG added 3 commits July 12, 2023 14:58

Simplified adding shadowJar to artifacts

f28d5ad

Signed-off-by: Guian Gumpac <[email protected]>

Reverted whitespace change

5816750

Signed-off-by: Guian Gumpac <[email protected]>

Reverted whitespace change

17124e2

Signed-off-by: Guian Gumpac <[email protected]>

Yury-Fridlyand approved these changes Jul 12, 2023

View reviewed changes

MaxKsyunz reviewed Jul 12, 2023

View reviewed changes

GumpacG added 3 commits July 16, 2023 07:32

Create shadowJar as a separate artifact

dc4ad86

Signed-off-by: Guian Gumpac <[email protected]>

Fixed wrong version

4af84df

Signed-off-by: Guian Gumpac <[email protected]>

Added jenkins job to publish shadowJar to artifacts bucket

4bcbe80

Signed-off-by: Guian Gumpac <[email protected]>

acarbonetto approved these changes Jul 17, 2023

View reviewed changes

Yury-Fridlyand reviewed Jul 17, 2023

View reviewed changes

Changed source and signing platform

db0095e

Signed-off-by: Guian Gumpac <[email protected]>

MaxKsyunz approved these changes Jul 17, 2023

View reviewed changes

GumpacG merged commit b7f9188 into integ-shadow-release Jul 17, 2023

GumpacG mentioned this pull request Jul 24, 2023

Include shadowJar artifact to release workflow (#8) opensearch-project/sql-jdbc#106

Merged

6 tasks

	source: "$WORKSPACE/build/libs/opensearch-sql-jdbc-shadow-*.jar",
	source: "$WORKSPACE/opensearch-sql-jdbc-shadow-${tag}.jar",

Include shadowJar artifact to release workflow #8

Include shadowJar artifact to release workflow #8

Uh oh!

Conversation

GumpacG commented Jul 12, 2023

Description

Check List

Uh oh!

Uh oh!

Uh oh!

Yury-Fridlyand left a comment

Choose a reason for hiding this comment

Uh oh!

MaxKsyunz left a comment

Choose a reason for hiding this comment

Uh oh!

GumpacG commented Jul 12, 2023

Uh oh!

Yury-Fridlyand commented Jul 12, 2023

Uh oh!

GumpacG commented Jul 13, 2023

Uh oh!

Yury-Fridlyand commented Jul 13, 2023

Uh oh!

MaxKsyunz commented Jul 13, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!