Releases 3.0.0
2025-04-28 Version 3.0.0 Release Notes
[3.0.0]
Breaking Changes
Upgrade to Lucene 10.1.0 - PR (#16366 )
JDK21 as minimum supported Java runtime (#10745 )
Remove deprecated terms from Java API (#5214 )
JPMS Support (Eliminate top level split packages ) Phase-0 only (#8110 )
Add ThreadContextPermission for stashAndMergeHeaders and stashWithOrigin (#15039 )
Add ThreadContextPermission for markAsSystemContext and allow core to perform the method (#15016 )
Migrate client transports to Apache HttpClient / Core 5.x (#4459 )
Validation changes on the Bulk Index API like enforcing 512 byte _id size limit (#6595 )
Ability to use the node as coordinating node by passing node.roles as empty array (#3412 )
Treat Setting value with empty array string as empty array (#10625 )
Ensure Jackson default maximums introduced in 2.16.0 do not conflict with OpenSearch settings (#11811 )
Setting a maximum depth for nested queries (#11670 )
Fix interchanged formats of total_indexing_buffer_in_bytes and total_indexing_buffer (#17070 )
Cleanup deprecated thread pool settings (#2595 )
Replace "blacklist/whitelist" terminology in Java APIs (#1683 )
Remove deprecated methods from JodaCompatibleZonedDateTime which are called by scripts (#3346 )
List of deprecated code removal in 3.0- partially done (#2773 )
Remove mmap.extensions setting (#9392 )
Remove COMPAT locale provider (#13988 )
Remove transport-nio plugin (#16887 )
Deprecate CamelCase PathHierarchy tokenizer name (#10894 )
Rename Class ending with Plugin to Module under modules dir (#4042 )
Remove deprecated batch_size parameter from _bulk (#14283 )
Unset discovery nodes for every transport node actions request (#17682 )
Added
Support for HTTP/2 (server-side) (#3847 )
Allow mmap to use new JDK-19 preview APIs in Apache Lucene 9.4+ (#5151 )
Add events correlation engine plugin (#6854 )
Implement on behalf of token passing for extensions (#8679 , #10664 )
Provide service accounts tokens to extensions (#9618 )
GHA to verify checklist items completion in PR descriptions (#10800 )
[WLM] Add WLM support for search scroll API (#16981 )
Views, simplify data access and manipulation by providing a virtual layer over one or more indices (#11957 )
Add systemd configurations to strengthen OS core security (#17107 )
Added pull-based Ingestion (APIs, for ingestion source, a Kafka plugin, and IngestionEngine that pulls data from the ingestion source) (#16958 )
Added ConfigurationUtils to core for the ease of configuration parsing #17223
Add cluster and index level settings to limit the total primary shards per node and per index #17295
Add execution_hint to cardinality aggregator request (#17312 )
Arrow Flight RPC plugin with Flight server bootstrap logic and client for internode communication (#16962 )
Added offset management for the pull-based Ingestion (#17354 )
Added integ tests for systemd configs (#17410 )
Add filter function for AbstractQueryBuilder, BoolQueryBuilder, ConstantScoreQueryBuilder(#17409 )
[Star Tree] [Search] Resolving keyword & numeric bucket aggregation with metric aggregation using star-tree (#17165 )
Added error handling support for the pull-based ingestion (#17427 )
Added Warm index setting and Updated nomenclature to differentiate between hot and warm tiering implementation (#17490 )
Latency and Memory allocation improvements to Multi Term Aggregation queries (#14993 )
Add logic in master service to optimize performance and retain detailed logging for critical cluster operations. (#14795 )
Add Setting to adjust the primary constraint weights (#16471 )
Introduce a setting to disable download of full cluster state from remote on term mismatch(#16798 )
Added ability to retrieve value from DocValues in a flat_object filed(#16802 )
Improve performace of NumericTermAggregation by avoiding unnecessary sorting(#17252 )
Fix Bug - Handle unsigned long in sorting order assertion of LongHashSet (#17207 )
Implemented computation of segment replication stats at shard level (#17055 )
[Rule Based Auto-tagging] Add in-memory attribute value store (#17342 )
Change priority for scheduling reroute during timeout(#16445 )
[Rule Based Auto-tagging] Add rule schema for auto tagging (#17238 )
Renaming the node role search to warm (#17573 )
Introduce a new search node role to hold search only shards (#17620 )
Fix systemd integTest on deb regarding path ownership check (#17641 )
Add dfs transformation function in XContentMapValues (#17612 )
Add tracking for long-running SearchTask post cancellation (#17726 )
Added Kinesis support as a plugin for the pull-based ingestion (#17615 )
Add FilterFieldType for developers who want to wrap MappedFieldType (#17627 )
[Rule Based Auto-tagging] Add in-memory rule processing service (#17365 )
[Security Manager Replacement] Create initial Java Agent to intercept Socket::connect calls (#17724 )
Faster terms_query with already sorted terms (#17714 )
Add ingestion management APIs for pause, resume and get ingestion state (#17631 )
[Security Manager Replacement] Enhance Java Agent to intercept System::exit (#17746 )
[Security Manager Replacement] Add a policy parser for Java agent security policies (#17753 )
[Security Manager Replacement] Implement File Interceptor and add integration tests (#17760 )
[Security Manager Replacement] Enhance Java Agent to intercept Runtime::halt (#17757 )
[Security Manager Replacement] Phase off SecurityManager usage in favor of Java Agent (#17861 )
Support AutoExpand for SearchReplica (#17741 )
Add TLS enabled SecureNetty4GrpcServerTransport (#17796 )
Implement fixed interval refresh task scheduling (#17777 )
[Tiered caching] Create a single cache manager for all the disk caches. (#17513 )
Add GRPC DocumentService and Bulk endpoint (#17727 )
Added scale to zero (search_only mode) support for OpenSearch reader writer separation (#17299
[Star Tree] [Search] Resolving numeric range aggregation with metric aggregation using star-tree (#17273 )
Added Search Only strict routing setting (#17803 )
Disable the index API for ingestion engine (#17768 )
Add SearchService and Search GRPC endpoint (#17830 )
Add update and delete support in pull-based ingestion (#17822 )
Allow maxPollSize and pollTimeout in IngestionSource to be configurable (#17863 )
[Star Tree] [Search] Add query changes to support unsigned-long in star tree (#17275 )
Add ApproximateMatchAllQuery that targets match_all queries and approximates sorts (#17772 )
Add TermsQuery support to Search GRPC endpoint (#17888 )
Support sub agg in filter rewrite optimization (#17447
Disable scoring of keyword term search by default, fallback logic with new use_similarity:true parameter (#17889 )
Add versioning support in pull-based ingestion (#17918 )
Introducing MergedSegmentWarmerFactory to support the extension of IndexWriter.IndexReaderWarmer (#17881 )
Introduce mapping transformer to allow transform mappings during index create/update or index template create/update (#17635 )
Introduce new dynamic cluster setting to tune maxMergeAtOnce at cluster level (#17774 )
Add multi-threaded writer support in pull-based ingestion (#17912 )
[Star Tree] Support of Boolean Queries in Aggregations (#17941 )
Enabled default throttling for all tasks submitted to cluster manager (#17711 )
Dependencies
Bump Apache HttpCore5/HttpClient5 dependencies from 5.2.5/5.3.1 to 5.3.1/5.4.1 to support ExtendedSocketOption in HttpAsyncClient (#16757 )
Bumps jetty version from 9.4.55.v20240627 to 9.4.57.v20241219 (#17395 )
Switch main/3.x to use JDK21 LTS version (#17515 )
Bump org.awaitility:awaitility from 4.2.0 to 4.3.0 (#17230 , #17439 )
Bump dnsjava:dnsjava from 3.6.2 to 3.6.3 (#17231 )
Bump com.google.code.gson:gson from 2.11.0 to 2.13.0 (#17229 , #17668 , #17921 ), #17926 )
Bump org.jruby.joni:joni from 2.2.1 to 2.2.6 (#17136 , #17608 , #17732 )
Bump org.apache.ant:ant from 1.10.14 to 1.10.15 (#17288 )
Bump reactor_netty from 1.1.26 to 1.2.3 (#17322 , #17322 , #17377 )
Bump me.champeau.gradle.japicmp from 0.4.5 to 0.4.6 (#17375 )
Bump com.google.api.grpc:proto-google-common-protos from 2.37.1 to 2.54.1 (#17379 , #17733 )
Bump net.minidev:json-smart from 2.5.1 to 2.5.2 (#17378 )
Bump com.netflix.nebula.ospackage-base from 11.10.1 to 11.11.2 (#17374 , #17734 )
Bump ch.qos.logback:logback-classic from 1.5.16 to 1.5.18 (#17497 , #17730 )
Bump software.amazon.awssdk from 2.20.86 to 2.30.31 (#17396 )
Bump org.jruby.jcodings:jcodings from 1.0.61 to 1.0.63 (#17560 )
Bump com.azure:azure-storage-blob from 12.28.1 to 12.30.0 (#17562 , #17667 )
Bump com.nimbusds:nimbus-jose-jwt from 9.41.1 to 10.0.2 (#17607 , #17669 )
Bump com.google.api:api-common from 1.8.1 to 2.46.1 (#17604 )
Bump ch.qos.logback:logback-core from 1.5.16 to 1.5.18 (#17609 , #17809 )
Bump dangoslen/dependabot-changelog-helper from 3 to 4 (#17498 )
Bump com.google.api:gax from 2.35.0 to 2.63.1 (#17465 )
Bump tj-actions/changed-files from 46.0.1 to 46.0.5 (#17666 , #17813 , #17920 )
Bump com.github.luben:zstd-jni from 1.5.5-1 to 1.5.6-1 (#17674 )
Bump lycheeverse/lychee-action from 2.3.0 to 2.4.0 (#17731 )
Bump com.nimbusds:oauth2-oidc-sdk from 11.21 to 11.23.1 (#17729 )
Bump com.google.api.grpc:proto-google-iam-v1 from 1.33.0 to 1.49.1 (#17811 )
Bump com.azure:azure-core from 1.54.1 to 1.55.3 (#17810 )
Bump org.apache.poi version from 5.2.5 to 5.4.1 in /plugins/ingest-attachment (#17887 )
Bump org.opensearch:protobufs from 0.2.0 to 0.3.0 (#17888 )
Changed
Changed locale provider from COMPAT to CLDR (#14345 )
Change http code on create index API with bad input raising NotXContentException from 500 to 400 (#4773 )
Improve summary error message for invalid setting updates (#4792 )
Return 409 Conflict HTTP status instead of 503 on failure to concurrently execute snapshots (#8986 )
Add task completion count in search backpressure stats API (#10028 )
Breaking change: Do not request "search_pipelines" metrics by default in NodesInfoRequest (#12497 )
Use simpler matching logic for source fields when explicit field names (no wildcards or dot-paths) are specified (#17160 )
Refactor :libs module bootstrap package to eliminate top level split packages for JPMS support (#17117 )
Refactor the codebase to eliminate top level split packages for JPMS support (#17153
Refactor :server module org.apacge.lucene package to eliminate top level split packages for JPMS support (#17241 )
Stop minimizing automata used for case-insensitive matches (#17268 )
Refactor the :server module org.opensearch.client to org.opensearch.transport.client to eliminate top level split packages for JPMS support (#17272 )
Use Lucene BM25Similarity as default since the LegacyBM25Similarity is marked as deprecated (#17306 )
Wildcard field index only 3gram of the input data #17349
Use BC libraries to parse PEM files, increase key length, allow general use of known cryptographic binary extensions, remove unused BC dependencies (#3420 )
Add optional enum set read / write functionality to stream input / output (#17556 )
Convert transport-reactor-netty4 to use gradle version catalog #17233
Increase force merge threads to 1/8th of cores #17255
TieredSpilloverCache took-time threshold now guards heap tier as well as disk tier #17190
Migrate BC libs to their FIPS counterparts (#14912 )
Increase the floor segment size to 16MB (#17699 )
Unwrap singleton DocValues in global ordinal value source of composite histogram aggregation (#17740 )
Unwrap singleton DocValues in date histogram aggregation. (#17643 )
Introduce 512 byte limit to search and ingest pipeline IDs (#17786 )
Avoid skewed segment replication lag metric (#17831 )
Increase the default segment counter step size when replica promoting (#17568 )
[WLM] Rename QueryGroup to WorkloadGroup (#17901 )
Relaxes jarHell check for optionally extended plugins(#17893 ))
Add cluster setting for retry timeout of publish checkpoint tx action (#17749 )
Increase the default maxMergesAtOnce to 30 (#17774 )
Change the default max header size from 8KB to 16KB. (#18024 )
Enable concurrent_segment_search auto mode by default#17978
Skip approximation when track_total_hits is set to true #18017
Deprecated
Removed
Remove deprecated code to add node name into log pattern of log4j property file (#4568 )
Unused object and import within TransportClusterAllocationExplainAction (#4639 )
Remove LegacyESVersion.V_7_0_* and V_7_1_* Constants (#2768 )
Remove LegacyESVersion.V_7_2_ and V_7_3_ Constants (#4702 )
Always auto release the flood stage block (#4703 )
Remove LegacyESVersion.V_7_4_ and V_7_5_ Constants (#4704 )
Remove Legacy Version support from Snapshot/Restore Service (#4728 )
Remove deprecated serialization logic from pipeline aggs (#4847 )
Remove unused private methods (#4926 )
Remove LegacyESVersion.V_7_8_ and V_7_9_ Constants (#4855 )
Remove LegacyESVersion.V_7_6_ and V_7_7_ Constants (#4837 )
Remove LegacyESVersion.V_7_10_ Constants (#5018 )
Remove Version.V_1_ Constants (#5021 )
Remove custom Map, List and Set collection classes (#6871 )
Remove package org.opensearch.action.support.master (#4856 )
Remove deprecated 'gateway' settings used to defer cluster recovery (#3117 )
Remove FeatureFlags.PLUGGABLE_CACHE as the feature is no longer experimental (#17344 )
Remove FeatureFlags.APPROXIMATE_POINT_RANGE_QUERY_SETTING since range query approximation is no longer experimental (#17769 )
Fixed
Fix 'org.apache.hc.core5.http.ParseException: Invalid protocol version' under JDK 16+ (#4827 )
Fix compression support for h2c protocol (#4944 )
Don't over-allocate in HeapBufferedAsyncEntityConsumer in order to consume the response (#9993 )
Add HTTP/2 protocol support to HttpRequest.HttpVersion (#17248 )
Fix missing bucket in terms aggregation with missing value (#17418 )
Fix explain action on query rewrite (#17286 )
Fix visit of inner query for FunctionScoreQueryBuilder (#16776 )
Fix case insensitive and escaped query on wildcard (#16827 )
Fix exists queries on nested flat_object fields throws exception (#16803 )
Add highlighting for wildcard search on match_only_text field (#17101 )
Fix illegal argument exception when creating a PIT (#16781 )
Fix NPE in node stats due to QueryGroupTasks (#17576 )
Fix bytes parameter on _cat/recovery (#17598 )
Fix slow performance of FeatureFlag checks (#17611 )
Fix shard recovery in pull-based ingestion to avoid skipping messages (#17868 ))
Fix ingest pipeline cannot be executed when retry the failed index requests for update_by_query API and reindex API (#18003 )
With creation of FilterFieldType, we need unwrap all the MappedFieldType before using the instanceof check. (#17951 )
You can’t perform that action at this time.
