[BUG] Getting a document with a semantic field leads to server stop AND persistent server break

[reuschling]

I declared a semantic field as a mapping and put a document with an according field entry into the index.

The mapping looks good so far, but when I tried to read the document with Url/indexname/_doc/id the server shows an error and exited himself:

[2025-06-30T16:26:07,480][ERROR][o.o.b.OpenSearchUncaughtExceptionHandler] [pc-4156] fatal error in thread [opensearch[pc-4156][generic][T#4]], exiting
java.lang.UnsatisfiedLinkError: no opensearchknn_faiss_avx2 in java.library.path: /usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2458) ~[?:?]
	at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:916) ~[?:?]
	at java.base/java.lang.System.loadLibrary(System.java:2063) ~[?:?]
	at org.opensearch.knn.jni.FaissService.lambda$static$0(FaissService.java:52) ~[?:?]
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:319) ~[?:?]
	at org.opensearch.knn.jni.FaissService.<clinit>(FaissService.java:42) ~[?:?]
	at org.opensearch.knn.jni.JNIService.initIndex(JNIService.java:50) ~[?:?]
	at org.opensearch.knn.index.codec.nativeindex.MemOptimizedNativeIndexBuildStrategy.lambda$buildAndWriteIndex$0(MemOptimizedNativeIndexBuildStrategy.java:63) ~[?:?]
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:319) ~[?:?]
	at org.opensearch.knn.index.codec.nativeindex.MemOptimizedNativeIndexBuildStrategy.buildAndWriteIndex(MemOptimizedNativeIndexBuildStrategy.java:62) ~[?:?]
	at org.opensearch.knn.index.codec.nativeindex.NativeIndexWriter.buildAndWriteIndex(NativeIndexWriter.java:159) ~[?:?]
	at org.opensearch.knn.index.codec.nativeindex.NativeIndexWriter.flushIndex(NativeIndexWriter.java:105) ~[?:?]
	at org.opensearch.knn.index.codec.KNN990Codec.NativeEngines990KnnVectorsWriter.flush(NativeEngines990KnnVectorsWriter.java:128) ~[?:?]
	at org.apache.lucene.codecs.perfield.PerFieldKnnVectorsFormat$FieldsWriter.flush(PerFieldKnnVectorsFormat.java:120) ~[lucene-core-10.2.1.jar:10.2.1 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2 - 2025-04-25 16:50:53]
	at org.apache.lucene.index.VectorValuesConsumer.flush(VectorValuesConsumer.java:76) ~[lucene-core-10.2.1.jar:10.2.1 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2 - 2025-04-25 16:50:53]
	at org.apache.lucene.index.IndexingChain.flush(IndexingChain.java:305) ~[lucene-core-10.2.1.jar:10.2.1 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2 - 2025-04-25 16:50:53]
	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:456) ~[lucene-core-10.2.1.jar:10.2.1 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2 - 2025-04-25 16:50:53]
	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:502) ~[lucene-core-10.2.1.jar:10.2.1 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2 - 2025-04-25 16:50:53]
	at org.apache.lucene.index.DocumentsWriter.maybeFlush(DocumentsWriter.java:456) ~[lucene-core-10.2.1.jar:10.2.1 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2 - 2025-04-25 16:50:53]
	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:649) ~[lucene-core-10.2.1.jar:10.2.1 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2 - 2025-04-25 16:50:53]
	at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3677) ~[lucene-core-10.2.1.jar:10.2.1 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2 - 2025-04-25 16:50:53]
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:4125) ~[lucene-core-10.2.1.jar:10.2.1 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2 - 2025-04-25 16:50:53]
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:4087) ~[lucene-core-10.2.1.jar:10.2.1 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2 - 2025-04-25 16:50:53]
	at org.opensearch.index.engine.InternalEngine.commitIndexWriter(InternalEngine.java:2575) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.index.engine.InternalEngine.flush(InternalEngine.java:1909) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.index.engine.InternalEngine$1.onAfterTranslogRecovery(InternalEngine.java:269) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.index.translog.listener.CompositeTranslogEventListener.onAfterTranslogRecovery(CompositeTranslogEventListener.java:63) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.index.translog.InternalTranslogManager.recoverFromTranslogInternal(InternalTranslogManager.java:167) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.index.translog.InternalTranslogManager.recoverFromTranslog(InternalTranslogManager.java:131) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.index.shard.IndexShard.openEngineAndRecoverFromTranslog(IndexShard.java:2597) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.index.shard.IndexShard.openEngineAndRecoverFromTranslog(IndexShard.java:2560) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.index.shard.StoreRecovery.internalRecoverFromStore(StoreRecovery.java:754) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.index.shard.StoreRecovery.lambda$recoverFromStore$0(StoreRecovery.java:128) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.core.action.ActionListener.completeWith(ActionListener.java:344) ~[opensearch-core-3.1.0.jar:3.1.0]
	at org.opensearch.index.shard.StoreRecovery.recoverFromStore(StoreRecovery.java:123) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.index.shard.IndexShard.recoverFromStore(IndexShard.java:2980) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.action.ActionRunnable$2.doRun(ActionRunnable.java:89) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:975) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) ~[opensearch-3.1.0.jar:3.1.0]
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
	at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
fatal error in thread [opensearch[pc-4156][generic][T#4]], exiting
java.lang.UnsatisfiedLinkError: no opensearchknn_faiss_avx2 in java.library.path: /usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2458)
	at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:916)
	at java.base/java.lang.System.loadLibrary(System.java:2063)
	at org.opensearch.knn.jni.FaissService.lambda$static$0(FaissService.java:52)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:319)
	at org.opensearch.knn.jni.FaissService.<clinit>(FaissService.java:42)
	at org.opensearch.knn.jni.JNIService.initIndex(JNIService.java:50)
	at org.opensearch.knn.index.codec.nativeindex.MemOptimizedNativeIndexBuildStrategy.lambda$buildAndWriteIndex$0(MemOptimizedNativeIndexBuildStrategy.java:63)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:319)
	at org.opensearch.knn.index.codec.nativeindex.MemOptimizedNativeIndexBuildStrategy.buildAndWriteIndex(MemOptimizedNativeIndexBuildStrategy.java:62)
	at org.opensearch.knn.index.codec.nativeindex.NativeIndexWriter.buildAndWriteIndex(NativeIndexWriter.java:159)
	at org.opensearch.knn.index.codec.nativeindex.NativeIndexWriter.flushIndex(NativeIndexWriter.java:105)
	at org.opensearch.knn.index.codec.KNN990Codec.NativeEngines990KnnVectorsWriter.flush(NativeEngines990KnnVectorsWriter.java:128)
	at org.apache.lucene.codecs.perfield.PerFieldKnnVectorsFormat$FieldsWriter.flush(PerFieldKnnVectorsFormat.java:120)
	at org.apache.lucene.index.VectorValuesConsumer.flush(VectorValuesConsumer.java:76)
	at org.apache.lucene.index.IndexingChain.flush(IndexingChain.java:305)
	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:456)
	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:502)
	at org.apache.lucene.index.DocumentsWriter.maybeFlush(DocumentsWriter.java:456)
	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:649)
	at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3677)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:4125)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:4087)
	at org.opensearch.index.engine.InternalEngine.commitIndexWriter(InternalEngine.java:2575)
	at org.opensearch.index.engine.InternalEngine.flush(InternalEngine.java:1909)
	at org.opensearch.index.engine.InternalEngine$1.onAfterTranslogRecovery(InternalEngine.java:269)
	at org.opensearch.index.translog.listener.CompositeTranslogEventListener.onAfterTranslogRecovery(CompositeTranslogEventListener.java:63)
	at org.opensearch.index.translog.InternalTranslogManager.recoverFromTranslogInternal(InternalTranslogManager.java:167)
	at org.opensearch.index.translog.InternalTranslogManager.recoverFromTranslog(InternalTranslogManager.java:131)
	at org.opensearch.index.shard.IndexShard.openEngineAndRecoverFromTranslog(IndexShard.java:2597)
	at org.opensearch.index.shard.IndexShard.openEngineAndRecoverFromTranslog(IndexShard.java:2560)
	at org.opensearch.index.shard.StoreRecovery.internalRecoverFromStore(StoreRecovery.java:754)
	at org.opensearch.index.shard.StoreRecovery.lambda$recoverFromStore$0(StoreRecovery.java:128)
	at org.opensearch.core.action.ActionListener.completeWith(ActionListener.java:344)
	at org.opensearch.index.shard.StoreRecovery.recoverFromStore(StoreRecovery.java:123)
	at org.opensearch.index.shard.IndexShard.recoverFromStore(IndexShard.java:2980)
	at org.opensearch.action.ActionRunnable$2.doRun(ActionRunnable.java:89)
	at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:975)
	at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)

Last but not least, when I want to start the server again, it fails again with

[2025-06-30T16:30:35,950][INFO ][o.o.s.i.DetectorIndexManagementService] [pc-4156] info deleteOldIndices
[2025-06-30T16:30:35,950][INFO ][o.o.s.i.DetectorIndexManagementService] [pc-4156] No Old IOC Findings Indices to delete
[2025-06-30T16:30:35,958][ERROR][o.o.b.OpenSearchUncaughtExceptionHandler] [pc-4156] fatal error in thread [opensearch[pc-4156][generic][T#4]], exiting
java.lang.UnsatisfiedLinkError: no opensearchknn_faiss_avx2 in java.library.path: /usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2458) ~[?:?]
	at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:916) ~[?:?]
	at java.base/java.lang.System.loadLibrary(System.java:2063) ~[?:?]
	at org.opensearch.knn.jni.FaissService.lambda$static$0(FaissService.java:52) ~[?:?]
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:319) ~[?:?]
	at org.opensearch.knn.jni.FaissService.<clinit>(FaissService.java:42) ~[?:?]
	at org.opensearch.knn.jni.JNIService.initIndex(JNIService.java:50) ~[?:?]
	at org.opensearch.knn.index.codec.nativeindex.MemOptimizedNativeIndexBuildStrategy.lambda$buildAndWriteIndex$0(MemOptimizedNativeIndexBuildStrategy.java:63) ~[?:?]
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:319) ~[?:?]
	at org.opensearch.knn.index.codec.nativeindex.MemOptimizedNativeIndexBuildStrategy.buildAndWriteIndex(MemOptimizedNativeIndexBuildStrategy.java:62) ~[?:?]
	at org.opensearch.knn.index.codec.nativeindex.NativeIndexWriter.buildAndWriteIndex(NativeIndexWriter.java:159) ~[?:?]
	at org.opensearch.knn.index.codec.nativeindex.NativeIndexWriter.flushIndex(NativeIndexWriter.java:105) ~[?:?]
	at org.opensearch.knn.index.codec.KNN990Codec.NativeEngines990KnnVectorsWriter.flush(NativeEngines990KnnVectorsWriter.java:128) ~[?:?]
	at org.apache.lucene.codecs.perfield.PerFieldKnnVectorsFormat$FieldsWriter.flush(PerFieldKnnVectorsFormat.java:120) ~[lucene-core-10.2.1.jar:10.2.1 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2 - 2025-04-25 16:50:53]
	at org.apache.lucene.index.VectorValuesConsumer.flush(VectorValuesConsumer.java:76) ~[lucene-core-10.2.1.jar:10.2.1 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2 - 2025-04-25 16:50:53]
	at org.apache.lucene.index.IndexingChain.flush(IndexingChain.java:305) ~[lucene-core-10.2.1.jar:10.2.1 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2 - 2025-04-25 16:50:53]
	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:456) ~[lucene-core-10.2.1.jar:10.2.1 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2 - 2025-04-25 16:50:53]
	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:502) ~[lucene-core-10.2.1.jar:10.2.1 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2 - 2025-04-25 16:50:53]
	at org.apache.lucene.index.DocumentsWriter.maybeFlush(DocumentsWriter.java:456) ~[lucene-core-10.2.1.jar:10.2.1 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2 - 2025-04-25 16:50:53]
	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:649) ~[lucene-core-10.2.1.jar:10.2.1 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2 - 2025-04-25 16:50:53]
	at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3677) ~[lucene-core-10.2.1.jar:10.2.1 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2 - 2025-04-25 16:50:53]
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:4125) ~[lucene-core-10.2.1.jar:10.2.1 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2 - 2025-04-25 16:50:53]
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:4087) ~[lucene-core-10.2.1.jar:10.2.1 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2 - 2025-04-25 16:50:53]
	at org.opensearch.index.engine.InternalEngine.commitIndexWriter(InternalEngine.java:2575) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.index.engine.InternalEngine.flush(InternalEngine.java:1909) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.index.engine.InternalEngine$1.onAfterTranslogRecovery(InternalEngine.java:269) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.index.translog.listener.CompositeTranslogEventListener.onAfterTranslogRecovery(CompositeTranslogEventListener.java:63) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.index.translog.InternalTranslogManager.recoverFromTranslogInternal(InternalTranslogManager.java:167) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.index.translog.InternalTranslogManager.recoverFromTranslog(InternalTranslogManager.java:131) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.index.shard.IndexShard.openEngineAndRecoverFromTranslog(IndexShard.java:2597) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.index.shard.IndexShard.openEngineAndRecoverFromTranslog(IndexShard.java:2560) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.index.shard.StoreRecovery.internalRecoverFromStore(StoreRecovery.java:754) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.index.shard.StoreRecovery.lambda$recoverFromStore$0(StoreRecovery.java:128) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.core.action.ActionListener.completeWith(ActionListener.java:344) ~[opensearch-core-3.1.0.jar:3.1.0]
	at org.opensearch.index.shard.StoreRecovery.recoverFromStore(StoreRecovery.java:123) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.index.shard.IndexShard.recoverFromStore(IndexShard.java:2980) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.action.ActionRunnable$2.doRun(ActionRunnable.java:89) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:975) ~[opensearch-3.1.0.jar:3.1.0]
	at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) ~[opensearch-3.1.0.jar:3.1.0]
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
	at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
fatal error in thread [opensearch[pc-4156][generic][T#4]], exiting
java.lang.UnsatisfiedLinkError: no opensearchknn_faiss_avx2 in java.library.path: /usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2458)
	at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:916)
	at java.base/java.lang.System.loadLibrary(System.java:2063)
	at org.opensearch.knn.jni.FaissService.lambda$static$0(FaissService.java:52)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:319)
	at org.opensearch.knn.jni.FaissService.<clinit>(FaissService.java:42)
	at org.opensearch.knn.jni.JNIService.initIndex(JNIService.java:50)
	at org.opensearch.knn.index.codec.nativeindex.MemOptimizedNativeIndexBuildStrategy.lambda$buildAndWriteIndex$0(MemOptimizedNativeIndexBuildStrategy.java:63)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:319)
	at org.opensearch.knn.index.codec.nativeindex.MemOptimizedNativeIndexBuildStrategy.buildAndWriteIndex(MemOptimizedNativeIndexBuildStrategy.java:62)
	at org.opensearch.knn.index.codec.nativeindex.NativeIndexWriter.buildAndWriteIndex(NativeIndexWriter.java:159)
	at org.opensearch.knn.index.codec.nativeindex.NativeIndexWriter.flushIndex(NativeIndexWriter.java:105)
	at org.opensearch.knn.index.codec.KNN990Codec.NativeEngines990KnnVectorsWriter.flush(NativeEngines990KnnVectorsWriter.java:128)
	at org.apache.lucene.codecs.perfield.PerFieldKnnVectorsFormat$FieldsWriter.flush(PerFieldKnnVectorsFormat.java:120)
	at org.apache.lucene.index.VectorValuesConsumer.flush(VectorValuesConsumer.java:76)
	at org.apache.lucene.index.IndexingChain.flush(IndexingChain.java:305)
	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:456)
	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:502)
	at org.apache.lucene.index.DocumentsWriter.maybeFlush(DocumentsWriter.java:456)
	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:649)
	at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3677)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:4125)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:4087)
	at org.opensearch.index.engine.InternalEngine.commitIndexWriter(InternalEngine.java:2575)
	at org.opensearch.index.engine.InternalEngine.flush(InternalEngine.java:1909)
	at org.opensearch.index.engine.InternalEngine$1.onAfterTranslogRecovery(InternalEngine.java:269)
	at org.opensearch.index.translog.listener.CompositeTranslogEventListener.onAfterTranslogRecovery(CompositeTranslogEventListener.java:63)
	at org.opensearch.index.translog.InternalTranslogManager.recoverFromTranslogInternal(InternalTranslogManager.java:167)
	at org.opensearch.index.translog.InternalTranslogManager.recoverFromTranslog(InternalTranslogManager.java:131)
	at org.opensearch.index.shard.IndexShard.openEngineAndRecoverFromTranslog(IndexShard.java:2597)
	at org.opensearch.index.shard.IndexShard.openEngineAndRecoverFromTranslog(IndexShard.java:2560)
	at org.opensearch.index.shard.StoreRecovery.internalRecoverFromStore(StoreRecovery.java:754)
	at org.opensearch.index.shard.StoreRecovery.lambda$recoverFromStore$0(StoreRecovery.java:128)
	at org.opensearch.core.action.ActionListener.completeWith(ActionListener.java:344)
	at org.opensearch.index.shard.StoreRecovery.recoverFromStore(StoreRecovery.java:123)
	at org.opensearch.index.shard.IndexShard.recoverFromStore(IndexShard.java:2980)
	at org.opensearch.action.ActionRunnable$2.doRun(ActionRunnable.java:89)
	at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:975)
	at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)

...this breaked my server permanently, and I have also no chance to delete the bad document. The only what I can do is to erase my data directory.

OpenSearch version: 3.1, linux tar.gz

[brianf-aws]

Hi @bzhangam & @q-andy, can you take a look at this issue?

[martin-gaievski]

@reuschling what kind of cluster/server setup you're using? The error in the log is typically appears if you using one of the native platform engines (that's the case because error says faiss) and artifacts for that engine are not available. This should be the case if you use any of the official releases.

[bzhangam]

@reuschling How did you set up your cluster and use the neural search plugin? This may be related to this issue opensearch-project/k-NN#2777

The semantic field will use the faiss engine for the underlying knn_vector field as recommended by the knn plugin. But seems like there is some issue to make the faiss engine available.

Unfortunately for now we don't allow configuring the engine through the semantic field so you can not change the engine. We are developing a feature to support it.

For now I think you have to start your cluster through this docker image which will have the faiss engine compiled properly.

[dhrubo-os]

@reuschling could you please move this issue to the K-NN repo as this is not related to ml-commons?

[reuschling]

@reuschling what kind of cluster/server setup you're using? The error in the log is typically appears if you using one of the native platform engines (that's the case because error says faiss) and artifacts for that engine are not available. This should be the case if you use any of the official releases.

OpenSearch version: 3.1, the Linux tar.gz. I use Manjaro Linux. This is the info log when I start the server:

[2025-07-02T12:27:45,400][INFO ][o.o.n.Node               ] [pc-4156] version[3.1.0], pid[72113], build[tar/8ff7c6ee924a49f0f59f80a6e1c73073c8904214/2025-06-21T08:05:43.345081313Z], OS[Linux/6.1.141-1-MANJARO/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/21.0.2/21.0.2+13-58]

[reuschling]

@reuschling could you please move this issue to the K-NN repo as this is not related to ml-commons?

Sure, but how can I move issues? I see no button, maybe I haven't the permissions for this.