Python: restructed data folder and multiple improvements to vector stores

[eavanvalkenburg]

Motivation and Context

This PR:

• restructures the data folder, flattening the hierarchy
• it adds overloads to the Get, Upsert, Delete methods, to allow batch as well, marks the batch versions as deprecated, closes Port dotnet feature: .Net: [MEVD] Rename UpsertBatch/DeleteBatch/GetBatch to remove the Batch suffix #11301
• restructured the inheritance of VectorStoreRecordCollection, which now has a base that does record handling, which is shared with the VectorSearchBase for the serialization, and the search method mixins now subclass the VectorSearchBase so that there is one less parent direct parent, a full Collection will now inherit from VectorStoreRecordCollection and zero or more VectorSearch mixins, this is done to clean things up and in preparation of adding HybridSearch.
• Also adds a convenience method to each search mixin to create a VectorStoreTextSearch directly, allowing folks to import one less concept, closes Python: Add convenience function for creating VectorStoreTextSearch from a VectorSearchBase/Mixin #11111

Description

Most of the code has been moved out of many files and folders, into a few files which make purpose and collaboration between the classes easier to understand.

This is a breaking change:
If you have implemented your own VectorStoreRecordCollection, make sure to:

• Add VectorStoreRecordCollection[TKey, TModel] as superclass to the implementation
• Remove VectorSearchBase as superclass from implementation
• Set the SearchMixins to [TKey, TModel] instead of just [TModel]

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄

[markwallace-microsoft]

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
semantic_kernel/connectors/memory
faiss.py	117	7	94%	88, 96, 100, 128–129, 156, 214
semantic_kernel/connectors/memory/azure_ai_search
azure_ai_search_collection.py	132	36	73%	171, 173, 246–287, 297–307, 311, 315, 320–323
azure_ai_search_store.py	42	2	95%	129–130
utils.py	72	7	90%	125, 127, 129, 150–151, 154–155
semantic_kernel/connectors/memory/azure_cosmos_db
azure_cosmos_db_mongodb_collection.py	93	63	32%	77–110, 138–139, 142–185, 194–218, 229–249
azure_cosmos_db_mongodb_store.py	36	16	56%	56–85, 107–116
azure_cosmos_db_no_sql_collection.py	163	67	59%	97–98, 139–146, 157–171, 177–185, 191–195, 218–242, 246, 250, 274, 302–303, 307–312
azure_cosmos_db_no_sql_store.py	32	6	81%	90–95
utils.py	54	7	87%	43, 66, 91, 186, 200–203
semantic_kernel/connectors/memory/chroma
chroma.py	198	43	78%	67–71, 84–85, 93–94, 110, 118, 126, 133–136, 143, 206–208, 226–229, 241, 257, 260–281, 304, 353–354, 356
semantic_kernel/connectors/memory/in_memory
in_memory_collection.py	130	15	88%	66, 118, 120, 140, 179, 194, 206, 224–227, 231, 233, 235–236
semantic_kernel/connectors/memory/mongodb_atlas
mongodb_atlas_collection.py	130	42	68%	143, 150, 183–185, 191–193, 200, 204, 216–217, 225–232, 242, 253–255, 263–288, 295–301, 305, 309, 319–320
mongodb_atlas_store.py	52	7	87%	89–90, 92, 139–140, 144–145
utils.py	30	10	67%	38–40, 62, 86–88, 106–113
semantic_kernel/connectors/memory/pinecone
_pinecone.py	280	43	85%	116–119, 124, 129, 154, 160, 167, 200, 204–205, 213, 218–219, 238, 244–248, 274, 302–305, 322, 364, 366, 371, 378, 387, 389, 392, 401, 403, 407, 422, 424, 429, 435, 440, 451, 499, 518, 583
semantic_kernel/connectors/memory/postgres
postgres_collection.py	227	44	81%	130, 138–140, 145–149, 157, 162, 182, 235, 252, 264, 314, 323, 373, 393, 413, 422, 430, 465, 471–474, 493, 520, 522–524, 547, 605–625, 629, 633
utils.py	58	12	79%	72, 116, 119–126, 158, 173–174
semantic_kernel/connectors/memory/redis
redis_collection.py	237	47	80%	179, 184–185, 196–206, 214–234, 237–244, 259–264, 268, 344, 381–388, 466, 488–495
redis_store.py	42	2	95%	108–109
utils.py	69	32	54%	152–153, 171, 173, 180–195, 202–232
semantic_kernel/connectors/memory/weaviate
utils.py	78	22	72%	91–96, 193, 242, 261, 268–271, 280–291
weaviate_collection.py	170	59	65%	171–174, 178–184, 188–189, 200–217, 224–231, 240–260, 269–285, 289, 292–296, 340, 344–351, 366, 385, 401, 410–411, 417, 422
weaviate_store.py	58	17	71%	109–117, 121–126, 131–136, 141–142
semantic_kernel/connectors/search/bing
bing_search.py	109	12	89%	113, 119, 129, 152–153, 193, 199–200, 204–205, 211–212
semantic_kernel/connectors/search/google
google_search.py	98	11	89%	114, 124, 150–151, 166–167, 191–195
semantic_kernel/data
text_search.py	149	4	97%	188–189, 468–470
vector_search.py	115	17	85%	204–205, 244–247, 269–271, 311–314, 334–336, 369–372, 392–394
vector_storage.py	252	7	97%	205–206, 325–326, 535, 593, 652
semantic_kernel/functions
kernel_plugin.py	180	6	97%	393, 396, 409, 434, 455, 480
TOTAL	21002	2475	88%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
3336	5 💤	0 ❌	0 🔥	1m 37s ⏱️