Skip to content

opensearch 3.0.0 #222665

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 7, 2025
Merged

opensearch 3.0.0 #222665

merged 2 commits into from
May 7, 2025

Conversation

BrewTestBot
Copy link
Member

Created by brew bump


Created with brew bump-formula-pr.

release notes
## 2025-04-28 Version 3.0.0 Release Notes

[3.0.0]

Breaking Changes

  • Upgrade to Lucene 10.1.0 - PR (#16366)
  • JDK21 as minimum supported Java runtime (#10745)
  • Remove deprecated terms from Java API (#5214)
  • JPMS Support (Eliminate top level split packages ) Phase-0 only (#8110)
  • Add ThreadContextPermission for stashAndMergeHeaders and stashWithOrigin (#15039)
  • Add ThreadContextPermission for markAsSystemContext and allow core to perform the method (#15016)
  • Migrate client transports to Apache HttpClient / Core 5.x (#4459)
  • Validation changes on the Bulk Index API like enforcing 512 byte _id size limit (#6595)
  • Ability to use the node as coordinating node by passing node.roles as empty array (#3412)
  • Treat Setting value with empty array string as empty array (#10625)
  • Ensure Jackson default maximums introduced in 2.16.0 do not conflict with OpenSearch settings (#11811)
  • Setting a maximum depth for nested queries (#11670)
  • Fix interchanged formats of total_indexing_buffer_in_bytes and total_indexing_buffer (#17070)
  • Cleanup deprecated thread pool settings (#2595)
  • Replace "blacklist/whitelist" terminology in Java APIs (#1683)
  • Remove deprecated methods from JodaCompatibleZonedDateTime which are called by scripts (#3346)
  • List of deprecated code removal in 3.0- partially done (#2773)
  • Remove mmap.extensions setting (#9392)
  • Remove COMPAT locale provider (#13988)
  • Remove transport-nio plugin (#16887)
  • Deprecate CamelCase PathHierarchy tokenizer name (#10894)
  • Rename Class ending with Plugin to Module under modules dir (#4042)
  • Remove deprecated batch_size parameter from _bulk (#14283)
  • Unset discovery nodes for every transport node actions request (#17682)

Added

  • Support for HTTP/2 (server-side) (#3847)
  • Allow mmap to use new JDK-19 preview APIs in Apache Lucene 9.4+ (#5151)
  • Add events correlation engine plugin (#6854)
  • Implement on behalf of token passing for extensions (#8679, #10664)
  • Provide service accounts tokens to extensions (#9618)
  • GHA to verify checklist items completion in PR descriptions (#10800)
  • [WLM] Add WLM support for search scroll API (#16981)
  • Views, simplify data access and manipulation by providing a virtual layer over one or more indices (#11957)
  • Add systemd configurations to strengthen OS core security (#17107)
  • Added pull-based Ingestion (APIs, for ingestion source, a Kafka plugin, and IngestionEngine that pulls data from the ingestion source) (#16958)
  • Added ConfigurationUtils to core for the ease of configuration parsing #17223
  • Add cluster and index level settings to limit the total primary shards per node and per index #17295
  • Add execution_hint to cardinality aggregator request (#17312)
  • Arrow Flight RPC plugin with Flight server bootstrap logic and client for internode communication (#16962)
  • Added offset management for the pull-based Ingestion (#17354)
  • Added integ tests for systemd configs (#17410)
  • Add filter function for AbstractQueryBuilder, BoolQueryBuilder, ConstantScoreQueryBuilder(#17409)
  • [Star Tree] [Search] Resolving keyword & numeric bucket aggregation with metric aggregation using star-tree (#17165)
  • Added error handling support for the pull-based ingestion (#17427)
  • Added Warm index setting and Updated nomenclature to differentiate between hot and warm tiering implementation (#17490)
  • Latency and Memory allocation improvements to Multi Term Aggregation queries (#14993)
  • Add logic in master service to optimize performance and retain detailed logging for critical cluster operations. (#14795)
  • Add Setting to adjust the primary constraint weights (#16471)
  • Introduce a setting to disable download of full cluster state from remote on term mismatch(#16798)
  • Added ability to retrieve value from DocValues in a flat_object filed(#16802)
  • Improve performace of NumericTermAggregation by avoiding unnecessary sorting(#17252)
  • Fix Bug - Handle unsigned long in sorting order assertion of LongHashSet (#17207)
  • Implemented computation of segment replication stats at shard level (#17055)
  • [Rule Based Auto-tagging] Add in-memory attribute value store (#17342)
  • Change priority for scheduling reroute during timeout(#16445)
  • [Rule Based Auto-tagging] Add rule schema for auto tagging (#17238)
  • Renaming the node role search to warm (#17573)
  • Introduce a new search node role to hold search only shards (#17620)
  • Fix systemd integTest on deb regarding path ownership check (#17641)
  • Add dfs transformation function in XContentMapValues (#17612)
  • Add tracking for long-running SearchTask post cancellation (#17726)
  • Added Kinesis support as a plugin for the pull-based ingestion (#17615)
  • Add FilterFieldType for developers who want to wrap MappedFieldType (#17627)
  • [Rule Based Auto-tagging] Add in-memory rule processing service (#17365)
  • [Security Manager Replacement] Create initial Java Agent to intercept Socket::connect calls (#17724)
  • Faster terms_query with already sorted terms (#17714)
  • Add ingestion management APIs for pause, resume and get ingestion state (#17631)
  • [Security Manager Replacement] Enhance Java Agent to intercept System::exit (#17746)
  • [Security Manager Replacement] Add a policy parser for Java agent security policies (#17753)
  • [Security Manager Replacement] Implement File Interceptor and add integration tests (#17760)
  • [Security Manager Replacement] Enhance Java Agent to intercept Runtime::halt (#17757)
  • [Security Manager Replacement] Phase off SecurityManager usage in favor of Java Agent (#17861)
  • Support AutoExpand for SearchReplica (#17741)
  • Add TLS enabled SecureNetty4GrpcServerTransport (#17796)
  • Implement fixed interval refresh task scheduling (#17777)
  • [Tiered caching] Create a single cache manager for all the disk caches. (#17513)
  • Add GRPC DocumentService and Bulk endpoint (#17727)
  • Added scale to zero (search_only mode) support for OpenSearch reader writer separation (#17299
  • [Star Tree] [Search] Resolving numeric range aggregation with metric aggregation using star-tree (#17273)
  • Added Search Only strict routing setting (#17803)
  • Disable the index API for ingestion engine (#17768)
  • Add SearchService and Search GRPC endpoint (#17830)
  • Add update and delete support in pull-based ingestion (#17822)
  • Allow maxPollSize and pollTimeout in IngestionSource to be configurable (#17863)
  • [Star Tree] [Search] Add query changes to support unsigned-long in star tree (#17275)
  • Add ApproximateMatchAllQuery that targets match_all queries and approximates sorts (#17772)
  • Add TermsQuery support to Search GRPC endpoint (#17888)
  • Support sub agg in filter rewrite optimization (#17447
  • Disable scoring of keyword term search by default, fallback logic with new use_similarity:true parameter (#17889)
  • Add versioning support in pull-based ingestion (#17918)
  • Introducing MergedSegmentWarmerFactory to support the extension of IndexWriter.IndexReaderWarmer (#17881)
  • Introduce mapping transformer to allow transform mappings during index create/update or index template create/update (#17635)
  • Introduce new dynamic cluster setting to tune maxMergeAtOnce at cluster level (#17774)
  • Add multi-threaded writer support in pull-based ingestion (#17912)
  • [Star Tree] Support of Boolean Queries in Aggregations (#17941)
  • Enabled default throttling for all tasks submitted to cluster manager (#17711)

Dependencies

  • Bump Apache HttpCore5/HttpClient5 dependencies from 5.2.5/5.3.1 to 5.3.1/5.4.1 to support ExtendedSocketOption in HttpAsyncClient (#16757)
  • Bumps jetty version from 9.4.55.v20240627 to 9.4.57.v20241219 (#17395)
  • Switch main/3.x to use JDK21 LTS version (#17515)
  • Bump org.awaitility:awaitility from 4.2.0 to 4.3.0 (#17230, #17439)
  • Bump dnsjava:dnsjava from 3.6.2 to 3.6.3 (#17231)
  • Bump com.google.code.gson:gson from 2.11.0 to 2.13.0 (#17229, #17668, #17921), #17926)
  • Bump org.jruby.joni:joni from 2.2.1 to 2.2.6 (#17136, #17608, #17732)
  • Bump org.apache.ant:ant from 1.10.14 to 1.10.15 (#17288)
  • Bump reactor_netty from 1.1.26 to 1.2.3 (#17322, #17322, #17377)
  • Bump me.champeau.gradle.japicmp from 0.4.5 to 0.4.6 (#17375)
  • Bump com.google.api.grpc:proto-google-common-protos from 2.37.1 to 2.54.1 (#17379, #17733)
  • Bump net.minidev:json-smart from 2.5.1 to 2.5.2 (#17378)
  • Bump com.netflix.nebula.ospackage-base from 11.10.1 to 11.11.2 (#17374, #17734)
  • Bump ch.qos.logback:logback-classic from 1.5.16 to 1.5.18 (#17497, #17730)
  • Bump software.amazon.awssdk from 2.20.86 to 2.30.31 (#17396)
  • Bump org.jruby.jcodings:jcodings from 1.0.61 to 1.0.63 (#17560)
  • Bump com.azure:azure-storage-blob from 12.28.1 to 12.30.0 (#17562, #17667)
  • Bump com.nimbusds:nimbus-jose-jwt from 9.41.1 to 10.0.2 (#17607, #17669)
  • Bump com.google.api:api-common from 1.8.1 to 2.46.1 (#17604)
  • Bump ch.qos.logback:logback-core from 1.5.16 to 1.5.18 (#17609, #17809)
  • Bump dangoslen/dependabot-changelog-helper from 3 to 4 (#17498)
  • Bump com.google.api:gax from 2.35.0 to 2.63.1 (#17465)
  • Bump tj-actions/changed-files from 46.0.1 to 46.0.5 (#17666, #17813, #17920)
  • Bump com.github.luben:zstd-jni from 1.5.5-1 to 1.5.6-1 (#17674)
  • Bump lycheeverse/lychee-action from 2.3.0 to 2.4.0 (#17731)
  • Bump com.nimbusds:oauth2-oidc-sdk from 11.21 to 11.23.1 (#17729)
  • Bump com.google.api.grpc:proto-google-iam-v1 from 1.33.0 to 1.49.1 (#17811)
  • Bump com.azure:azure-core from 1.54.1 to 1.55.3 (#17810)
  • Bump org.apache.poi version from 5.2.5 to 5.4.1 in /plugins/ingest-attachment (#17887)
  • Bump org.opensearch:protobufs from 0.2.0 to 0.3.0 (#17888)

Changed

  • Changed locale provider from COMPAT to CLDR (#14345)
  • Change http code on create index API with bad input raising NotXContentException from 500 to 400 (#4773)
  • Improve summary error message for invalid setting updates (#4792)
  • Return 409 Conflict HTTP status instead of 503 on failure to concurrently execute snapshots (#8986)
  • Add task completion count in search backpressure stats API (#10028)
  • Breaking change: Do not request "search_pipelines" metrics by default in NodesInfoRequest (#12497)
  • Use simpler matching logic for source fields when explicit field names (no wildcards or dot-paths) are specified (#17160)
  • Refactor :libs module bootstrap package to eliminate top level split packages for JPMS support (#17117)
  • Refactor the codebase to eliminate top level split packages for JPMS support (#17153
  • Refactor :server module org.apacge.lucene package to eliminate top level split packages for JPMS support (#17241)
  • Stop minimizing automata used for case-insensitive matches (#17268)
  • Refactor the :server module org.opensearch.client to org.opensearch.transport.client to eliminate top level split packages for JPMS support (#17272)
  • Use Lucene BM25Similarity as default since the LegacyBM25Similarity is marked as deprecated (#17306)
  • Wildcard field index only 3gram of the input data #17349
  • Use BC libraries to parse PEM files, increase key length, allow general use of known cryptographic binary extensions, remove unused BC dependencies (#3420)
  • Add optional enum set read / write functionality to stream input / output (#17556)
  • Convert transport-reactor-netty4 to use gradle version catalog #17233
  • Increase force merge threads to 1/8th of cores #17255
  • TieredSpilloverCache took-time threshold now guards heap tier as well as disk tier #17190
  • Migrate BC libs to their FIPS counterparts (#14912)
  • Increase the floor segment size to 16MB (#17699)
  • Unwrap singleton DocValues in global ordinal value source of composite histogram aggregation (#17740)
  • Unwrap singleton DocValues in date histogram aggregation. (#17643)
  • Introduce 512 byte limit to search and ingest pipeline IDs (#17786)
  • Avoid skewed segment replication lag metric (#17831)
  • Increase the default segment counter step size when replica promoting (#17568)
  • [WLM] Rename QueryGroup to WorkloadGroup (#17901)
  • Relaxes jarHell check for optionally extended plugins(#17893))
  • Add cluster setting for retry timeout of publish checkpoint tx action (#17749)
  • Increase the default maxMergesAtOnce to 30 (#17774)
  • Change the default max header size from 8KB to 16KB. (#18024)
  • Enable concurrent_segment_search auto mode by default#17978
  • Skip approximation when track_total_hits is set to true #18017

Deprecated

Removed

  • Remove deprecated code to add node name into log pattern of log4j property file (#4568)
  • Unused object and import within TransportClusterAllocationExplainAction (#4639)
  • Remove LegacyESVersion.V_7_0_* and V_7_1_* Constants (#2768)
  • Remove LegacyESVersion.V_7_2_ and V_7_3_ Constants (#4702)
  • Always auto release the flood stage block (#4703)
  • Remove LegacyESVersion.V_7_4_ and V_7_5_ Constants (#4704)
  • Remove Legacy Version support from Snapshot/Restore Service (#4728)
  • Remove deprecated serialization logic from pipeline aggs (#4847)
  • Remove unused private methods (#4926)
  • Remove LegacyESVersion.V_7_8_ and V_7_9_ Constants (#4855)
  • Remove LegacyESVersion.V_7_6_ and V_7_7_ Constants (#4837)
  • Remove LegacyESVersion.V_7_10_ Constants (#5018)
  • Remove Version.V_1_ Constants (#5021)
  • Remove custom Map, List and Set collection classes (#6871)
  • Remove package org.opensearch.action.support.master (#4856)
  • Remove deprecated 'gateway' settings used to defer cluster recovery (#3117)
  • Remove FeatureFlags.PLUGGABLE_CACHE as the feature is no longer experimental (#17344)
  • Remove FeatureFlags.APPROXIMATE_POINT_RANGE_QUERY_SETTING since range query approximation is no longer experimental (#17769)

Fixed

  • Fix 'org.apache.hc.core5.http.ParseException: Invalid protocol version' under JDK 16+ (#4827)
  • Fix compression support for h2c protocol (#4944)
  • Don't over-allocate in HeapBufferedAsyncEntityConsumer in order to consume the response (#9993)
  • Add HTTP/2 protocol support to HttpRequest.HttpVersion (#17248)
  • Fix missing bucket in terms aggregation with missing value (#17418)
  • Fix explain action on query rewrite (#17286)
  • Fix visit of inner query for FunctionScoreQueryBuilder (#16776)
  • Fix case insensitive and escaped query on wildcard (#16827)
  • Fix exists queries on nested flat_object fields throws exception (#16803)
  • Add highlighting for wildcard search on match_only_text field (#17101)
  • Fix illegal argument exception when creating a PIT (#16781)
  • Fix NPE in node stats due to QueryGroupTasks (#17576)
  • Fix bytes parameter on _cat/recovery (#17598)
  • Fix slow performance of FeatureFlag checks (#17611)
  • Fix shard recovery in pull-based ingestion to avoid skipping messages (#17868))
  • Fix ingest pipeline cannot be executed when retry the failed index requests for update_by_query API and reindex API (#18003)
  • With creation of FilterFieldType, we need unwrap all the MappedFieldType before using the instanceof check. (#17951)

@github-actions github-actions bot added java Java use is a significant feature of the PR or issue bump-formula-pr PR was created using `brew bump-formula-pr` labels May 6, 2025
@branchvincent branchvincent force-pushed the bump-opensearch-3.0.0 branch from db9dd58 to 01abee3 Compare May 7, 2025 02:36
@daeho-ro daeho-ro force-pushed the bump-opensearch-3.0.0 branch from 01abee3 to cc09dec Compare May 7, 2025 10:36
Copy link
Contributor

github-actions bot commented May 7, 2025

🤖 An automated task has requested bottles to be published to this PR.

Please do not push to this PR branch before the bottle commits have been pushed, as this results in a state that is difficult to recover from. If you need to resolve a merge conflict, please use a merge commit. Do not force-push to this PR branch.

@github-actions github-actions bot added the CI-published-bottle-commits The commits for the built bottles have been pushed to the PR branch. label May 7, 2025
@BrewTestBot BrewTestBot enabled auto-merge May 7, 2025 12:08
@BrewTestBot BrewTestBot added this pull request to the merge queue May 7, 2025
Merged via the queue into master with commit 45e1a15 May 7, 2025
17 checks passed
@BrewTestBot BrewTestBot deleted the bump-opensearch-3.0.0 branch May 7, 2025 12:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bump-formula-pr PR was created using `brew bump-formula-pr` CI-published-bottle-commits The commits for the built bottles have been pushed to the PR branch. java Java use is a significant feature of the PR or issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants