Skip to content

[1.x backport] Fix opensearch-env always sources the environment from hardcoded file #875 #2907

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

peterzhuamazon
Copy link
Member

Description

[1.x backport] Fix opensearch-env always sources the environment from hardcoded file #875

Issues Resolved

opensearch-project/opensearch-build#1545 (comment)

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

mch2 and others added 30 commits July 2, 2021 15:02
…search-project#927)

Found when attempting to build on an `arm64` machine when I recieved an error message below.  Root cause is that string equality in java cannot be done with the `==` sign.

```
unknown architecture [arm64] for jdk [provisioned_runtime], must be one of [aarch64, x64]
```

Signed-off-by: Peter Nied <[email protected]>

Co-authored-by: Peter Nied <[email protected]>
…pensearch-project#928) (opensearch-project#935)

Fixes the cat.health yaml failures when running in a bwc mixed cluster with
legacy (pre 1.0.0) nodes.

Signed-off-by: Nicholas Walter Knize <[email protected]>
* backport PR opensearch-project#829 to 1.x branch, so that we can publish snapshots to our maven repository.
Signed-off-by: Marc Handalian <[email protected]>
…r.java (opensearch-project#947) (opensearch-project#998)

* Add addRoutesPrefix() method to RestHandler.java

Signed-off-by: Azar Fazel <[email protected]>
Signed-off-by: cliu123 <[email protected]>

Co-authored-by: afazel <[email protected]>
…pensearch-project#1001)

Replace the docs links In scripts bin/opensearch-env and config/jvm.options, with OpenSearch docs links.

Signed-off-by: Poojita-Raj <[email protected]>
(cherry picked from commit 6bc4ce0)
Signed-off-by: Poojita-Raj <[email protected]>
…nsearch-project#1060) (opensearch-project#1070)

* Avoid crashing on using the index.lifecycle.name in the API body

Signed-off-by: frotsch <[email protected]>
… msearch request (opensearch-project#986) (opensearch-project#1085)

* Part 1: Support for cancel_after_timeinterval parameter in search and msearch request

This commit introduces the new request level parameter to configure the timeout interval after which
a search request will be cancelled. For msearch request the parameter is supported both at parent
request and at sub child search requests. If it is provided at parent level and child search request
doesn't have it then the parent level value is set at such child request. The parent level msearch
is not used to cancel the parent request as it may be tricky to come up with correct value in cases
when child search request can have different runtimes

TEST: Added test for ser/de with new parameter

Signed-off-by: Sorabh Hamirwasia <[email protected]>

* Part 2: Support for cancel_after_timeinterval parameter in search and msearch request

This commit adds the handling of the new request level parameter and schedule cancellation task. It
also adds a cluster setting to set a global cancellation timeout for search request which will be
used in absence of request level timeout.

TEST: Added new tests in SearchCancellationIT
Signed-off-by: Sorabh Hamirwasia <[email protected]>

* Address Review feedback for Part 1

Signed-off-by: Sorabh Hamirwasia <[email protected]>

* Address review feedback for Part 2

Signed-off-by: Sorabh Hamirwasia <[email protected]>

* Update CancellableTask to remove the cancelOnTimeout boolean flag

Signed-off-by: Sorabh Hamirwasia <[email protected]>

* Replace search.cancellation.timeout cluster setting with search.enforce_server.timeout.cancellation to control if cluster level cancel_after_time_interval should take precedence over request level cancel_after_time_interval value

Signed-off-by: Sorabh Hamirwasia <[email protected]>

* Removing the search.enforce_server.timeout.cancellation cluster setting and just keeping search.cancel_after_time_interval setting with request level parameter taking the precedence.

Signed-off-by: Sorabh Hamirwasia <[email protected]>

Co-authored-by: Sorabh Hamirwasia <[email protected]>

Co-authored-by: Sorabh Hamirwasia <[email protected]>
…opensearch-project#1079) (opensearch-project#1087)

This change refactors the circular reference check in the Grok processor class
to use a formal depth-first traversal. It also includes a logic update to
prevent a stack overflow in one scenario and a check for malformed patterns.
This bugfix addresses CVE-2021-22144.

Signed-off-by: Kartik Ganesh <[email protected]>
…ch-project#1090)

* Support for bwc tests for plugins

Signed-off-by: Vacha <[email protected]>

* Adding support for restart upgrades for plugins bwc

Signed-off-by: Vacha <[email protected]>
…thHybridTimeout (opensearch-project#1105)

In some cases as one shared with issue opensearch-project#1099, the maxConcurrentSearchRequests was chosen as 0 which
will compute the final value during execution of the request based on processor counts. When this
computed value is less than number of search request in msearch request, it will execute all the
requests in multiple iterations causing the failure since test will only wait for one such
iteration. Hence setting the maxConcurrentSearchRequests explicitly to number of search requests
being added in the test to ensure correct behavior

Signed-off-by: Sorabh Hamirwasia <[email protected]>
…nsearch-project#1114)

Lucene 9 removes support for SimpleFS File System format. This commit deprecates
the SimpleFS format in favor of NIOFS.

Signed-off-by: Nicholas Walter Knize <[email protected]>
)

This commit upgrades to the official lucene 8.9 release

Signed-off-by: Nicholas Walter Knize <[email protected]>
…ository (opensearch-project#1049) (opensearch-project#1139)

This change fixes the issue where the sources and javadoc artifacts were not built and included with the publish.

Signed-off-by: Rabi Panda <[email protected]>
PluginInfo should use .onOrAfter(Version.V_1_1_0) instead of
.after(Version.V_1_0_0) for the new custom folder name for plugin feature.

Signed-off-by: Nicholas Walter Knize <[email protected]>
This commit stages the branch to the next 1.0.1 patch release. BWC testing needs
this even if the next revision is never actually released.

Signed-off-by: Nicholas Walter Knize <[email protected]>
…t (no needed) (opensearch-project#1174) (opensearch-project#1187)

* Drop mocksocket & securemock dependencies from sniffer and rest client (not needed)

Signed-off-by: Andriy Redko <[email protected]>

* Removing .gitignore

Signed-off-by: Andriy Redko <[email protected]>
…#1191) (opensearch-project#1195)

The version framework only added support for OpenSearch 1.x bwc with legacy
clusters. This commit adds support for v2.0 which will be the last version with
bwc support for legacy clusters (v7.10)

Signed-off-by: Nicholas Walter Knize <[email protected]>
opensearch-trigger-bot bot and others added 22 commits March 7, 2022 11:37
opensearch-project#2276) (opensearch-project#2394)

* Bump bc-fips from 1.0.2.1 to 1.0.2.3 in /distribution/tools/plugin-cli

Bumps bc-fips from 1.0.2.1 to 1.0.2.3.

---
updated-dependencies:
- dependency-name: org.bouncycastle:bc-fips
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
(cherry picked from commit 9780fc6)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ject#2264)

* edited opensearch-plugin install help output to include plugin URL

Signed-off-by: Joshua Palis <[email protected]>

* fixed unit test for plugin install help output by correctly identifying the beginning og the non-option argument list

Signed-off-by: Joshua Palis <[email protected]>

* added comments to install plugins help non option argument ouput unit test

Signed-off-by: Joshua Palis <[email protected]>

* fixed format violation

Signed-off-by: Joshua Palis <[email protected]>

* added additional details on valid plugin ids and how to use plugin URLs

Signed-off-by: Joshua Palis <[email protected]>

* added additional information to plugin install help output

Signed-off-by: Joshua Palis <[email protected]>

Co-authored-by: Joshua Palis <[email protected]>
(cherry picked from commit b251d2b)
Signed-off-by: Joshua Palis <[email protected]>
…ct#2282) (opensearch-project#2398)

Bumps commons-math3 from 3.2 to 3.6.1.

---
updated-dependencies:
- dependency-name: org.apache.commons:commons-math3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit c8a7606)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
opensearch-project#2365)

* Replace the exclusionary word whitelist with allowlist, and blacklist with denylist, in code commet and internal variable/method/class/package name.

Signed-off-by: Andreas <[email protected]>
…pensearch-project#2281) (opensearch-project#2402)

Bumps log4j-core from 2.17.1 to 2.17.2.

---
updated-dependencies:
- dependency-name: org.apache.logging.log4j:log4j-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit cb57b92)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…-project#2279) (opensearch-project#2401)

* Bump gson from 2.8.9 to 2.9.0 in /plugins/repository-hdfs

Bumps [gson](https://github.com/google/gson) from 2.8.9 to 2.9.0.
- [Release notes](https://github.com/google/gson/releases)
- [Changelog](https://github.com/google/gson/blob/master/CHANGELOG.md)
- [Commits](google/gson@gson-parent-2.8.9...gson-parent-2.9.0)

---
updated-dependencies:
- dependency-name: com.google.code.gson:gson
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
(cherry picked from commit 1b8181c)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…pensearch-project#2382) (opensearch-project#2403)

* Bump guava from 30.1.1-jre to 31.1-jre in /plugins/repository-azure

Bumps [guava](https://github.com/google/guava) from 30.1.1-jre to 31.1-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

---
updated-dependencies:
- dependency-name: com.google.guava:guava
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
(cherry picked from commit 09e16e3)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
… (opensearch-project#2409)

Bumps [guava](https://github.com/google/guava) from 31.0.1-jre to 31.1-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

---
updated-dependencies:
- dependency-name: com.google.guava:guava
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit 4395ed5)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ice constructor (opensearch-project#2177) (opensearch-project#2413)

* MapperService has to be passed in as null for EnginePlugins CodecService constructor

Signed-off-by: Andriy Redko <[email protected]>

* Addressing code review comments

Signed-off-by: Andriy Redko <[email protected]>

* Delayed CodecService instantiation up to the shard initialization

Signed-off-by: Andriy Redko <[email protected]>

* Added logger (associated with shard) to CodecServiceConfig

Signed-off-by: Andriy Redko <[email protected]>

* Refactored the EngineConfigFactory / IndexShard instantiation of the CodecService

Signed-off-by: Andriy Redko <[email protected]>
(cherry picked from commit 9c679cb)

Co-authored-by: Andriy Redko <[email protected]>
…earch-project#2385) (opensearch-project#2408)

* Bump asm-commons from 5.0.4 to 9.2 in /modules/lang-expression

Bumps asm-commons from 5.0.4 to 9.2.

---
updated-dependencies:
- dependency-name: org.ow2.asm:asm-commons
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
(cherry picked from commit 1f0361a)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…pensearch-project#2456) (opensearch-project#2465)

* Bump woodstox-core from 6.1.1 to 6.2.8 in /plugins/repository-azure

Bumps [woodstox-core](https://github.com/FasterXML/woodstox) from 6.1.1 to 6.2.8.
- [Release notes](https://github.com/FasterXML/woodstox/releases)
- [Commits](FasterXML/woodstox@woodstox-core-6.1.1...woodstox-core-6.2.8)

---
updated-dependencies:
- dependency-name: com.fasterxml.woodstox:woodstox-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
(cherry picked from commit d190813)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…pensearch-project#2455) (opensearch-project#2457)

* Bump commons-lang3 from 3.4 to 3.12.0 in /plugins/repository-azure

Bumps commons-lang3 from 3.4 to 3.12.0.

---
updated-dependencies:
- dependency-name: org.apache.commons:commons-lang3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
(cherry picked from commit 189563b)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ity (opensearch-project#2416) (opensearch-project#2466)

* Discrepancy in result from _validate/query API and actual query validity

Signed-off-by: Andriy Redko <[email protected]>

* Moved the validate() check later into the flow to allow range validation to trigger first

Signed-off-by: Andriy Redko <[email protected]>
Trademark added to readme file

Signed-off-by: Kyle J. Davis <[email protected]>
…tream solution is available (opensearch-project#2446) (opensearch-project#2475)

Signed-off-by: Andriy Redko <[email protected]>
(cherry picked from commit 12dd5d7)

Co-authored-by: Andriy Redko <[email protected]>
…-project#2562)

Resolves CVE-2020-36518

Signed-off-by: Peter Nied <[email protected]>
(cherry picked from commit d43235c)

Co-authored-by: Peter Nied <[email protected]>
…nlyOnes when CGroups are not available (opensearch-project#2101) (opensearch-project#2102)

Signed-off-by: Andriy Redko <[email protected]>
(cherry picked from commit a942b27)

Co-authored-by: Andriy Redko <[email protected]>
…opensearch-project#875)

distribution/bin/opensearch-env always sources the environment from the default environment file /etc/default/opensearch. This is an issue if we want to run multiple instances of OpenSearch on the same host. This change lets users override the default behavior by not sourcing the default environment file in case OPENSEARCH_PATH_CONF  is set.

Signed-off-by: Xue Zhou <[email protected]>
@peterzhuamazon peterzhuamazon requested a review from dblock April 14, 2022 22:00
@peterzhuamazon peterzhuamazon self-assigned this Apr 14, 2022
@peterzhuamazon peterzhuamazon requested a review from a team as a code owner April 14, 2022 22:00
@peterzhuamazon peterzhuamazon deleted the opensearch-1.x-backport-e3d86baa5a97a813072cb3d5fe9b466d31a44f31 branch April 14, 2022 22:02
@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure b2b402f
Log 4496

Reports 4496

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.