Skip to content

Commit 33ba3bd

Browse files
sid-swirlsid-swirl
committed
More updates
DS-3769
1 parent 48e55a8 commit 33ba3bd

File tree

2 files changed

+262
-171
lines changed

2 files changed

+262
-171
lines changed

‎docs/RAG-Guide.md

Lines changed: 109 additions & 66 deletions
Original file line numberDiff line numberDiff line change
@@ -12,113 +12,156 @@ nav_order: 5
1212
{:toc}
1313
</details>
1414

15-
# Retrieval Augmented Generation (RAG) Configuration - Community Edition
15+
# Retrieval Augmented Generation (RAG) Configuration - Community Edition
1616

17-
{: .warning }
18-
This document applies only to the SWIRL AI Connect, Community Edition. [Switch to the AI Connect, Enterprise Edition guide](AI-Connect.html)
17+
{: .warning }
18+
This document applies only to the **SWIRL AI Connect, Community Edition**.
19+
[Switch to the AI Connect, Enterprise Edition guide](AI-Connect.html)
1920

20-
SWIRL supports Real Time [Retrieval Augmented Generation (RAG)](index.html#what-is-retrieval-augmented-generation-rag-does-swirl-support-it) out of the box, using result snippets and/or the full text of fetched result pages.
21+
SWIRL supports Real-Time **[Retrieval Augmented Generation (RAG)](index.html#what-is-retrieval-augmented-generation-rag-does-swirl-support-it)** out of the box, using result snippets and/or the full text of fetched result pages.
2122

22-
# Setting up RAG
23+
---
2324

24-
* Install SWIRL as noted in the [Quick Start Guide](Quick-Start.html#local-installation), including the latest version of the Galaxy UI.
25+
# Configuring RAG
2526

26-
* Add either your OpenAI API key or your Azure OpenAI API credentials to the `.env` file:
27+
1. **Install SWIRL** as noted in the [Quick Start Guide](Quick-Start.html#local-installation), including the latest version of the **Galaxy UI**.
2728

28-
```
29-
OPENAI_API_KEY='your-key-here'
30-
```
31-
*Check out [OpenAI's YouTube video](https://youtu.be/nafDyRsVnXU?si=YpvyaRvhX65vtBrb) if you don't have an OpenAI API Key.*
29+
2. **Add your OpenAI API key or Azure OpenAI credentials** to the `.env` file:
3230

33-
```
34-
AZURE_OPENAI_KEY=<your-key>
35-
AZURE_OPENAI_ENDPOINT=<your-azure-openai-endpoint-url>
36-
AZURE_MODEL=<your-azure-openai-model>
37-
```
31+
```shell
32+
OPENAI_API_KEY='your-key-here'
33+
```
3834

39-
{: .warning }
40-
SWIRL AI Connect Community Edition supports RAG only against OpenAI and Azure/OpenAI. The [Enterprise Edition](AI-Connect.html#connecting-to-generative-ai-gai-and-large-language-models-llms)supports many more.
35+
*Check out [OpenAI's YouTube video](https://youtu.be/nafDyRsVnXU?si=YpvyaRvhX65vtBrb) if you don't have an OpenAI API Key.*
4136

42-
* When installing for PRODUCTION use, change the following line in `static/api/config/default` from:
37+
```shell
38+
AZURE_OPENAI_KEY=<your-key>
39+
AZURE_OPENAI_ENDPOINT=<your-azure-openai-endpoint-url>
40+
AZURE_MODEL=<your-azure-openai-model>
41+
```
4342

44-
```
45-
"webSocketConfig": {
46-
"url": "ws://<yourhost>:<your-port>/chatgpt-data"
47-
}
48-
```
43+
{: .warning }
44+
**SWIRL AI Connect Community Edition supports RAG only with OpenAI and Azure OpenAI.**
45+
The [Enterprise Edition](AI-Connect.html#connecting-to-generative-ai-gai-and-large-language-models-llms) supports additional providers.
4946

50-
...to...
47+
3. **For PRODUCTION use**, update `static/api/config/default`:
5148

52-
```
53-
"webSocketConfig": {
54-
"url": "wss://<yourhost>:<your-port>/chatgpt-data"
55-
}
56-
```
49+
Change this:
50+
```json
51+
"webSocketConfig": {
52+
"url": "ws://<yourhost>:<your-port>/chatgpt-data"
53+
}
54+
```
55+
...to this:
56+
```json
57+
"webSocketConfig": {
58+
"url": "wss://<yourhost>:<your-port>/chatgpt-data"
59+
}
60+
```
61+
*Using `ws://` locally is fine, but it is **not secure** for production!*
5762

58-
*The default `ws:` prefix can be used locally but should NEVER be used in production as it is not secure!*
63+
4. **Configure SearchProviders for RAG:**
5964

60-
* Add the following configuration to the `page_fetch_config_json` parameter of each SearchProvider you wish to have participate in RAG:
65+
Modify the `page_fetch_config_json` parameter for each **SearchProvider**:
6166

62-
```
63-
"page_fetch_config_json": {
67+
```json
68+
"page_fetch_config_json": {
6469
"cache": "false",
6570
"headers": {
6671
"User-Agent": "Swirlbot/1.0 (+http://swirl.today)"
6772
},
6873
"timeout": 10
69-
},
70-
```
74+
}
75+
```
7176

72-
Adjust the `timeout` value if necessary. Change the `User-Agent` string as needed, and/or authorize it to fetch pages from internal applications.
77+
- Adjust `timeout` if needed.
78+
- Change `User-Agent` if required.
79+
- Authorize SWIRL to fetch pages from internal applications.
7380

74-
You can also override the default timeout value with a URL parameter in the Galaxy UI. For example, to set a timeout of 90 seconds: `http://localhost:8000/galaxy/?q=gig%20economics&rag=true&rag_timeout=90`
81+
To override the timeout **via the Galaxy UI**, use:
82+
```
83+
http://localhost:8000/galaxy/?q=gig%20economics&rag=true&rag_timeout=90
84+
```
7585

76-
* Restart SWIRL:
86+
5. **Restart SWIRL:**
7787

78-
```
79-
python swirl.py restart
80-
```
88+
```shell
89+
python swirl.py restart
90+
```
91+
92+
6. **Run a Search in the Galaxy UI**:
8193

82-
* Go to the Galaxy UI ([http://localhost:8000/galaxy/](http://localhost:8000/galaxy/)). The "Generate AI Response" switch should be "off".
94+
- Open [http://localhost:8000/galaxy/](http://localhost:8000/galaxy/).
95+
- Ensure the **"Generate AI Response"** switch is **off**.
96+
- Search for:
97+
```shell
98+
http://localhost:8000/galaxy/?q=SWIRL+AI+Connect
99+
```
83100

84-
* Run a search. Results appear quickly after you press the "Search" button ([http://localhost:8000/galaxy/?q=epmc:future+of+ai+care](http://localhost:8000/galaxy/?q=SWIRL+AI+Connect)):
85-
![Galaxy with RAG results ready for selection](images/swirl_40_community_rag.png)
101+
Results appear:
102+
![Galaxy with RAG results ready for selection](images/swirl_40_community_rag.png)
86103

87-
* If you wish to manually select the results to RAG with, click the "Select Items" switch to make the shopping cart appear. Results that SWIRL thinks should be used in RAG will be pre-checked. Check or uncheck results, and optionally sort and/or filter them. When ready, click the "Generate AI Response" switch. A spinner will appear. The RAG response will appear several seconds later depending on a variety of factors. :slightly_smiling_face:
88-
![Galaxy with RAG results selected](images/swirl_40_rag_select.png)
104+
7. **Manually Select Results for RAG (Optional)**:
89105

90-
* Verify the RAG insight you received by reviewing the citations below the RAG response.
106+
- Click **"Select Items"** to enable manual selection.
107+
- Pre-selected results indicate **SWIRL's best matches for RAG**.
108+
- Check/uncheck results, sort, or filter.
109+
- Click **"Generate AI Response"**.
110+
- A spinner appears; results follow within seconds.
91111
92-
{: .highlight }
93-
To cancel a RAG process, click the "Generate AI Summary" toggle off.
112+
![Galaxy with RAG results selected](images/swirl_40_rag_select.png)
94113
95-
{: .warning }
96-
By default, SWIRL's RAG processing utilizes the *first 10 results* that are selected either automatically or manually using the "Select Items" option. To change the number, specify `SWIRL_RAG_MAX_TO_CONSIDER` in the .env file as noted in the [AI Connect Guide](./AI-Connect.md#configuration-options).
114+
8. **Verify citations** under the RAG response.
97115
98-
## Notes
116+
{: .highlight }
117+
To **cancel** a RAG process, toggle **"Generate AI Summary"** off.
99118
100-
* The RAG process is as follows:
119+
{: .warning }
120+
By default, **SWIRL's RAG uses the first 10 selected results** (auto or manual).
121+
To adjust this, set `SWIRL_RAG_MAX_TO_CONSIDER` in `.env`, as noted in the [AI Connect Guide](AI-Connect.html#configuration-options-1).
101122

102-
![SWIRL AI Connect Insight Pipeline](images/swirl_rag_pipeline.png)
123+
# SWIRL RAG Process:
103124

104-
* The community edition of SWIRL is intended to RAG with sources you can fetch without authenticating. If you need to perform RAG with content from enterprise services like Microsoft 365, ServiceNow, Salesforce, Atlassian with OAUTH2 and SSO, please [contact us for information about SWIRL Enterprise](mailto:[email protected]) - which supports all of that, and more, out of the box.
125+
![SWIRL AI Connect Insight Pipeline](images/swirl_rag_pipeline.png)
105126

106-
* RAG Page fetch configurations are preloaded for the following SearchProviders:
127+
SWIRL's RAG workflow is shown above, and explained below:
107128
108-
* [European PMC](https://github.com/swirlai/swirl-search/blob/main/SearchProviders/europe_pmc.json)
109-
* [Google Web](https://github.com/swirlai/swirl-search/blob/main/SearchProviders/google.json)
110-
* [Google News](https://github.com/swirlai/swirl-search/blob/main/SearchProviders/google.json)
111-
* [LinkedIn](https://github.com/swirlai/swirl-search/blob/main/SearchProviders/google.json)
112-
* [SWIRL Documentation](https://github.com/swirlai/swirl-search/blob/main/SearchProviders/google.json)
129+
1. **Search** - SWIRL sends the users query to one or more SearchProviders, then aggregates and normalizes the results
130+
2. **Re-Rank** - The last step of the Search workflow re-ranks the aggregated, normalized results to find the most relevant results across all sources
131+
3. **Review** - SWIRL supports presenting re-ranked results for review and optional adjustment, prior to executing the rest of the workflow
132+
4. **Fetch** - This involves following the result link and downloading the full text or data set of the result
133+
5. **Read** - SWIRL extracts text from 1,500 file formats then identifies the most relevant passages while chunking it for the next step
134+
6. **Prompt** - SWIRL binds the chunked text to the appropriate prompt, connects to the configured LLM, sends the prompt and waits patiently for the response
135+
7. **Package** - Finally, SWIRL compiles the results, prompt and response and returns it in a neat JSON package, ready for visualization in SWIRL's Galaxy Search UI
113136

114-
* RAG processing is available through a single API call, e.g. `?qs=metasearch&rag=true`. See the [Developer Guide](https://docs.swirl.today/Developer-Guide.html#get-synchronous-results-with-the-qs-url-parameter) for more details about the `?qs=` parameter.
137+
### Enterprise RAG Support
138+
- The **Community Edition** can fetch **publicly accessible sources**.
139+
- For **RAG with enterprise services** (e.g., Microsoft 365, ServiceNow, Salesforce, Atlassian) with **OAuth2 and SSO**, [contact us for SWIRL Enterprise](mailto:[email protected]).
115140

116-
* The default timeout value (60 seconds) and the text to display when the timeout is exceeded cab be configured in the `static/api/config/default` file.
141+
### Preloaded RAG Configurations
142+
The following **SearchProviders** come pre-configured for RAG:
117143

144+
✅ **[European PMC](https://github.com/swirlai/swirl-search/blob/main/SearchProviders/europe_pmc.json)**
145+
✅ **[Google Web](https://github.com/swirlai/swirl-search/blob/main/SearchProviders/google.json)**
146+
✅ **[Google News](https://github.com/swirlai/swirl-search/blob/main/SearchProviders/google.json)**
147+
✅ **[LinkedIn](https://github.com/swirlai/swirl-search/blob/main/SearchProviders/google.json)**
148+
✅ **[SWIRL Documentation](https://github.com/swirlai/swirl-search/blob/main/SearchProviders/google.json)**
149+
150+
### API-Based RAG Processing
151+
RAG processing is available via **a single API call**:
152+
```
153+
?qs=metasearch&rag=true
118154
```
155+
For details, see the [Developer Guide](https://docs.swirl.today/Developer-Guide.html#get-synchronous-results-with-the-qs-url-parameter).
156+
157+
### Configuring Timeout Behavior
158+
- The **default timeout is 60 seconds**.
159+
- To modify the timeout and error message, update:
160+
161+
```json
119162
"webSocketConfig": {
120163
"url": "ws://localhost:8000/chatgpt-data",
121164
"timeout": 60000,
122165
"timeoutText": "Timeout: No response from Generative AI."
123-
}
124-
```
166+
}
167+
```

0 commit comments

Comments
 (0)