[BUG] In-Memory Vector Store doesn't work in Document Store after Upsert and Test Retrieval

[rprrafa]

Describe the bug
Access Document Store and add some content using In-Memory Vector Store doesn't work

To Reproduce
Steps to reproduce the behavior:

1. Go to 'Document Store'
2. Click on 'Add new'
3. Put "name" and "description' in modal
4. Click on 'Add' to confirm
5. Click on created Document Store
6. Add some Document Loader
7. Click on "Upsert Config"
8. Select some Embedding Model in step 1
9. Select "In-Memory Vector Store" in step 2
10. Save Config
11. Click on "Upsert"
12. Click on "Test Retrieval"
13. Try searching for something
14. See error

Expected behavior
Find chunks related to searching terms inserted in In-Memory Vector Store

Screenshots

Screen.Recording.2024-10-17.at.11.49.16.mov

Setup

• Installation [npx flowise start]
• Flowise Version 2.1.2
• OS: macOS
• Browser edge and safari

[AndersonNascimentoDosSantos]

in my case i'm using the github document loader, togetherAi embedding and groqchat

and the in memory vector store and i'm getting the following message in the logs

Error: SyntaxError: Unexpected token '<', "<!DOCTYPE "... is not valid JSON

    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

    at async buildFlow (/usr/local/lib/node_modules/flowise/dist/utils/index.js:417:37)

    at async upsertVector (/usr/local/lib/node_modules/flowise/dist/utils/upsertVector.js:146:32)

    at async Object.upsertVectorMiddleware (/usr/local/lib/node_modules/flowise/dist/services/vectors/index.js:9:16)

2024-10-18 04:21:20 [ERROR]: [server]: Error: Error: SyntaxError: Unexpected token '<', "<!DOCTYPE "... is not valid JSON

Error: Error: SyntaxError: Unexpected token '<', "<!DOCTYPE "... is not valid JSON

    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

    at async upsertVector (/usr/local/lib/node_modules/flowise/dist/utils/upsertVector.js:146:32)

    at async Object.upsertVectorMiddleware (/usr/local/lib/node_modules/flowise/dist/services/vectors/index.js:9:16)

    at async createInternalUpsert (/usr/local/lib/node_modules/flowise/dist/controllers/vectors/index.js:28:29)

[HenryHengZJ]

thanks for the video! long story short, you have to use other vector store. We missed that one, should've disabled in-memory store for document store. It is an ephemeral vectorstore that only works when you attach to chatflow, and vectors are upserted on the fly. For document store approach, it doesn't work.

Related: #3375