Semantic fields not possible as dynamic mapping and/or as internal mapping field

[reuschling]

I tried to declare a semantic field inside my dynamic mappings and failed:

"dynamic_templates": [
{
	"semantic": {
		"match": "semantic_*",
		"mapping": {
			"type": "semantic",
			"model_id": "4j4iwZcB7dKCiuIqGvlV"
		}
	}
},
{
	"textwithNgramSemantic": {
		"match": "tnsem_*",
		"mapping": {
			"type": "text",
			"analyzer": "standardWithStopwords",
			"fields": {
				"ngram": {
					"type": "text",
					"analyzer": "4gram"
				},
				"semantic": {
					"type": "semantic",
					"model_id": "4j4iwZcB7dKCiuIqGvlV"
				}
			}
		}
	}
}
]

No one of these get a _semantic_info mapping entry, also inserting documents and searching them with neural search is not possible.

I then tried it without the dynamic pattern and declared two fields directly:

"properties": {
	"semanticBody": {
		"type": "semantic",
		"model_id": "4j4iwZcB7dKCiuIqGvlV"
	},
	"semanticNgramBody": {
		"type": "text",
		"analyzer": "standardWithStopwords",
		"fields": {
			"ngram": {
				"type": "text",
				"analyzer": "4gram"
			},
			"semantic": {
				"type": "semantic",
				"model_id": "4j4iwZcB7dKCiuIqGvlV"
			}
		}
	}
}

Only the semanticBody that directly corresponds to the semantic field documentation gets a _semantic_info mapping entry.

So there are 3 regular OpenSearch field declarations where semantic fields are not possible currently:

1. Declaration as additional field inside a regular "mappings" => "properties" setting
2. Declaration without internal/additional field as dynamic mapping under "mappings" => "dynamic_templates"
3. Declaration as additional field inside the dynamic templates (the combination of 1. and 2.)

[brianf-aws]

Hi @bzhangam & @q-andy, can you take a look at this issue?

[bzhangam]

@reuschling Thanks for your report. The semantic field currently does not support dynamic mapping—you need to define it explicitly in your index mapping. It also cannot be used inside the fields section at this time.

If you believe these features would be useful for your use case, we encourage you to open a feature request on GitHub. Your feedback helps us prioritize future improvements.

[dhrubo-os]

@reuschling also I think you can move this issue to the neural search repo: https://github.com/opensearch-project/neural-search

[reuschling]

@reuschling Thanks for your report. The semantic field currently does not support dynamic mapping—you need to define it explicitly in your index mapping. It also cannot be used inside the fields section at this time.

If you believe these features would be useful for your use case, we encourage you to open a feature request on GitHub. Your feedback helps us prioritize future improvements.

I know, this is the reason for my issue, and we are on GitHub, isn't it? Just the label 'feature request' is missing.

And yes, all of our mappings are specified dynamically, this gives the huge benefit to pre-configure these fields only once, then just dealing with field name prefixes afterwards. Thus, we can use the same mapping in all our projects, and for each field you can see at the name which type it is, inspired by hungarian notation.

[reuschling]

@reuschling also I think you can move this issue to the neural search repo: https://github.com/opensearch-project/neural-search

Sure, but as in my other issue, I don't know how I can move issues. I have no according interface possibility, maybe I haven't the permissions for this.

[bzhangam]

@reuschling Thanks for your report. The semantic field currently does not support dynamic mapping—you need to define it explicitly in your index mapping. It also cannot be used inside the fields section at this time.
If you believe these features would be useful for your use case, we encourage you to open a feature request on GitHub. Your feedback helps us prioritize future improvements.

I know, this is the reason for my issue, and we are on GitHub, isn't it? Just the label 'feature request' is missing.

And yes, all of our mappings are specified dynamically, this gives the huge benefit to pre-configure these fields only once, then just dealing with field name prefixes afterwards. Thus, we can use the same mapping in all our projects, and for each field you can see at the name which type it is, inspired by hungarian notation.

I've investigated this, and supporting the semantic field within dynamic mapping or as multi-fields presents some challenges. So we may not be able to support them in near future.

Why dynamic mapping doesn't work out of the box
Currently, we rely on a system-generated ingest pipeline to detect whether an index has a semantic field, in order to trigger embedding generation. However, dynamic mapping is applied only after the ingest pipeline runs, which means the pipeline can't see the new field in time to generate its corresponding embedding. This makes native support for semantic fields in dynamic mappings non-trivial.

Why multi-fields are problematic
Similarly, using a semantic field as a multi-field of a text field isn't straightforward. Our ingest pipeline logic doesn't support embedding generation when the semantic field is nested inside another field via fields, since it can't automatically add an embedding field in that context.

Workaround for dynamic mapping
If you still want to use a semantic field with dynamic mapping, here's a workaround:

1. Index a document to trigger dynamic mapping and create the field.

2. Perform a no-op mapping update (e.g., re-submit the same mapping) to explicitly trigger the embedding field generation logic.

3. Reindex your documents to generate the actual embeddings.

Using semantic as a standalone field with sub-fields
In most cases, you don’t need to use semantic as a multi-field of a text field, because the semantic field already stores the original content as a text field internally.

That means you can define sub-fields (like ngrams) directly on a semantic field, for example:

"semanticNgramBody": {
		"type": "semantic",
		"analyzer": "standardWithStopwords",
		"fields": {
			"ngram": {
				"type": "text",
				"analyzer": "4gram"
			}
		}
	}