Upload min targ.gz snapshots to publicly accessible bucket

[gaiksaya]

The buildManifest just builds the snapshot but does not upload it anywhere. Its the publish-snapshot that uploads the artifacts to maven.

https://github.com/opensearch-project/opensearch-build/blob/main/jenkins/opensearch/distribution-build.jenkinsfile#L45-L47

Should we directly upload from here (jenkins local) to snapshots (prod) s3? Also what about arm64? While I executed the jenkinfile to see the generated artifacts, I did not see anything generated for arm64 here.
OR Can we separate this problem from the release-candidates one?
Also talked to @mch2 seems like we have 2 possible solutions:

1. Rename the above received artifact to snapshots as it guarantees that user is running tests against an artifact that will get released
2. Build x64 (already doing) and arm64 snapshots and then upload to s3 (either just to prod or both staging and prod).

Originally posted by @gaiksaya in #1378 (comment)

Acceptance Criteria:

Min snapshot artifacts should be accessible via https://artifacts.opensearch.org/
Example: https://artifacts.opensearch.org/snapshots/core/opensearch/1.2.2-SNAPSHOT/opensearch-min-1.2.2-SNAPSHOT-linux-x64.tar.gz
https://artifacts.opensearch.org/snapshots/core/opensearch/1.2.2-SNAPSHOT/opensearch-min-1.2.2-SNAPSHOT-linux-arm64.tar.gz

[dblock]

The -SNAPSHOT build is fundamentally a different build from non-SNAPSHOT. It will never be released. For daily builds that are used for testing I think we want to keep it this way.

I noticed that with the requirements above the distribution build is looking increasingly similar to what we want the snapshot build to be (both x64 and arm), except that the latter is built with snapshot: true and uploads its artifacts to a different location in S3 + publishes snapshot JARs to Maven. We might consider having 2 jobs.

[mch2]

The -SNAPSHOT build is fundamentally a different build from non-SNAPSHOT. It will never be released. For daily builds that are used for testing I think we want to keep it this way.

I noticed that with the requirements above the distribution build is looking increasingly similar to what we want the snapshot build to be (both x64 and arm), except that the latter is built with snapshot: true and uploads its artifacts to a different location in S3 + publishes snapshot JARs to Maven. We might consider having 2 jobs.

Agree - I'd suggest we repurpose the existing arm/x64 builds to include snapshot: true and create a new script to build a release artifact off of a locked down manifest.

The intent of making those original builds 'release' builds was so any daily build used for testing could be promoted. I'm not sure this is providing us much value if we can perform a release build against a locked down manifest.

[dblock]

The -SNAPSHOT build is fundamentally a different build from non-SNAPSHOT. It will never be released. For daily builds that are used for testing I think we want to keep it this way.
I noticed that with the requirements above the distribution build is looking increasingly similar to what we want the snapshot build to be (both x64 and arm), except that the latter is built with snapshot: true and uploads its artifacts to a different location in S3 + publishes snapshot JARs to Maven. We might consider having 2 jobs.

Agree - I'd suggest we repurpose the existing arm/x64 builds to include snapshot: true and create a new script to build a release artifact off of a locked down manifest.

The intent of making those original builds 'release' builds was so any daily build used for testing could be promoted. I'm not sure this is providing us much value if we can perform a release build against a locked down manifest.

I think that's not quite what was I was saying, we still want those dailies for people to download and try that aren't snapshot, but maybe I'm overthinking it.

[mch2]

Will they be used in addition to the 2 snapshot artifacts? If they have their separate use then that makes sense. If not, I think having 4 parallel builds is a bit redundant. The snapshots "should" be identical to the release tarball outside of maven tagging its publications as snapshots.

[gaiksaya]

After talking to @peternied looks like this issue is urgent. Working on implementing a fix. This is what it looks like currently:
https://github.com/gaiksaya/opensearch-build/blob/snapshots/jenkins/opensearch/distribution-build.jenkinsfile#L57-L91

[gaiksaya]

PR #1472 resolved this issue. Thanks!