feat: Add Secret handling in OpenSearchDocumentStore

[vblagoje]

Why:

Enhances the authentication mechanism within the OpenSearch integration to allow for sensitive credentials to be managed securely using environment variables.

• fixes Add support for secret management and from_env_var for OpenSearch integration haystack#8659

What:

• Introduced new capability to read OpenSearch credentials (username and password) from environment variables using the Secret class.
• Updated the initialization process to handle missing environment variables gracefully.
• Extended serialization and deserialization methods to include the handling of Secret objects for secure storage and retrieval of credentials.

How can it be used:

Environment variable-based authentication can be leveraged by setting:

export OPENSEARCH_USERNAME="your_username"
export OPENSEARCH_PASSWORD="your_password"

Then, initialize the OpenSearchDocumentStore as follows:

document_store = OpenSearchDocumentStore(hosts="your_host")

How did you test it:

Testing involved mocking the OpenSearch client to verify the correct behavior of the authentication logic, including the execution of environment variable-based initialization and validation of serialized objects to check the correct capture of credentials as Secret dictionaries.

Notes for the reviewer:

Focus on the sections handling the http_auth in dict serialization and the test cases for environment-based authentication to ensure consistent and secure handling of credentials. Verify that no credentials are logged or exposed improperly.

[vblagoje]

This one is not ready yet @tstadel 👍

[vblagoje]

@tstadel should be ready now, please be really careful with this one 🙏

[tstadel]

@vblagoje in general looks good. I'm very confident we don't break anything. But let's please support List[Secret] besides/instead of Tuple[Secret] here. Reason:
YAML does not support tuples, so once you serialize the to_dict output containing tuples to yaml, and deserialize it again you will end up with List[Secret].

import yaml

serialized = OpenSearchDocumentStore().to_dict()
yaml_serialized = yaml.safe_dump(serialized)
yaml_deserialized = yaml.safe_load(yaml_serialized)
# http_auth will be [...] and hence secrets not deserialized
deserialzed = OpenSearchDocumentStore.from_dict(yaml_deserialized)

[vblagoje]

@tstadel although the init parameters for secrets use tuple they are always serialized as a list as you can see in the tests. I wanted to remain 100% backward compatible here so all changes are focused only on secrets addition and making sure everything else stays the same.

[tstadel]

That works! Thanks for applying the changes! 🚀