swirlai · erikspears · Feb 19, 2025 · Feb 18, 2025 · Feb 18, 2025 · Feb 18, 2025
diff --git a/docs/AI-Co-Pilot.md b/docs/AI-Co-Pilot.md
@@ -39,7 +39,7 @@ SWIRL AI Connect, included with Co-Pilot, defines four roles for GAI/LLMs, as no
 
 * Edit any AI Provider by adding the `id` value to the end of the `swirl/aiproviders` URL. For example: `http://localhost:8000/swirl/aiproviders/4/`
 
-![SWIRL AIProvider - Azure/OpenAI GPT-4](./images/swirl_aiprovider_4.png)
+![SWIRL AIProvider - Azure/OpenAI GPT-4](images/swirl_aiprovider_4.png)
 
 * Review the provider definition to ensure that the `active` property is `true`.
 
@@ -78,11 +78,11 @@ For example, here is the preloaded OpenAI GPT-4 provider, set for chat as well a
     }
 ```
 
-Assuming the AI provider is correctly configured, you should be able to load the Co-Pilot using a browser. If you are using the default installation, this will be:
+Assuming the AI provider is correctly configured, you should be able to load the Co-Pilot using a browser. If using the default installation, this will be:
 
 [http://localhost:8000/galaxy/chat](http://localhost:8000/galaxy/chat)
 
-![SWIRL Co-Pilot](images/swirl_copilot_chat_rag.png)
+![SWIRL Co-Pilot discussion](images/swirl_40_chat_scop.png)
 
 Please refer to the AI Connect, Enterprise Edition, section on [Connecting to Enteprise AI](AI-Connect.html#connecting-to-enterprise-ai) for more information on configuring AI Providers.
 

diff --git a/docs/AI-Connect.md b/docs/AI-Connect.md
@@ -192,7 +192,7 @@ The following links present the models available through LiteLLM:
 
 Edit any AI Provider by adding the `id` value to the end of the `swirl/aiproviders` URL. For example: `http://localhost:8000/swirl/aiproviders/4/`
 
-![SWIRL AIProvider - Azure/OpenAI GPT-4](./images/swirl_aiprovider_4.png)
+![SWIRL AIProvider - Azure/OpenAI GPT-4](images/swirl_aiprovider_4.png)
 
 From here, use the form at the bottom of the page to:
 
@@ -343,21 +343,7 @@ This will ensure you see messages from the `celery-beats` log. However, most of
 
 ## Default Prompt
 
-By default, the SWIRL system prompt is as follows:
-
-```
-{
-    "name": "default",
-    "shared": false,
-    "prompt": "Answer this query '{query}' given the following recent search results as background information. Do not mention that you are using the provided background information. Please list the sources at the end of your response. Ignore information that is off-topic or obviously conflicting, without warning about it. The results may include information for different entities with identical names, try to disambiguate them in your response. If you discover possible familiar relationships in the content, mention it as a possibility.",
-    "data": [],
-    "note": "Important: Text between {RAG_IMPORTANT_START_TAG} and {RAG_IMPORTANT_END_TAG} is most pertinent to the query.\n",
-    "footer": "\n\n\n\n--- Final Instructions ---\nIn your response do not assume people with vastly different work histories are the same person. If the query appears to be a proper name, focus on answering the question, 'Who is?' or 'What is?', as appropriate. If the query appears to be a question, then try to answer it. For the list of sources, use the HTML tags and format in the example below:\n\n<p>\n<br><b>Sources:</b>\n<br><i>example description 1</i> &nbsp;&nbsp;&nbsp; <b>example website or source name 1</b>\n<br><i>example description 2</i> &nbsp;&nbsp;&nbsp; <b>example website or source name 2</b>\n</p>\n\nEnclose your response in HTML tags <p></p> and insert a <br> HTML tag every two sentences.",
-    "tags": []
-}
-```
-
-This is designed to operate on search queries by providing summarization of the provided data and/or answering simple questions from it. 
+You can view the default prompts from the SWIRL AI Connect page:
 
 ## Creating New Saved Prompts
 
@@ -370,16 +356,15 @@ For example, to modify the default prompt so that the response is in pirate-spea
 ```
     {
         "name": "pirate",
-        "prompt": "Answer this query '{query}' given the following recent search results as background information. Do not mention that you are using the provided background information. Please list the sources at the end of your response. Ignore information that is off-topic or obviously conflicting, without warning about it. The results may include information for different entities with identical names, try to disambiguate them in your response. If you discover possible familiar relationships in the content, mention it as a possibility.",
-        "note": "Important: Text between {RAG_IMPORTANT_START_TAG} and {RAG_IMPORTANT_END_TAG} is most pertinent to the query.",
+        ... etc ...
         "footer": "--- Final Instructions ---\nIn your response, pretend you are a pirate comedian, but keep it clean!",
         "tags": []
     }
 ```
 
 This should produce the following:
 
-![SWIRL Prompt Object](./images/swirl_prompt_form.png)
+![SWIRL Prompt Object](images/swirl_prompt_form.png)
 
 ## Specifying a Saved Prompt in a Query
 
@@ -391,7 +376,7 @@ swirl ai connect prompt:pirate
 
 The response should be in pirate-speak:
 
-![SWIRL RAG response in pirate speak](./images/swirl_prompt_pirate.png)
+![SWIRL RAG response in pirate speak](images/swirl_prompt_pirate.png)
 
 ## Understanding Saved Prompts
 

diff --git a/docs/Admin-Guide.md b/docs/Admin-Guide.md
@@ -179,6 +179,14 @@ python swirl.py restart
 {: .highlight }
 Consult the [release notes](https://github.com/swirlai/swirl-search/releases) for more information on each release.
 
+## Resetting Prompts
+
+To reset AI prompts to their default state, run the following command:
+
+`python swirl.py reload_ai_prompts`
+
+This will restore the system prompts to the "factory" settings. New prompts are not impacted.
+
 # Configuring SWIRL
 
 SWIRL uses the following configuration items, defined in the [`swirl_server/settings.py`](https://github.com/swirlai/swirl-search/blob/main/swirl_server/settings.py):

diff --git a/docs/Developer-Guide.md b/docs/Developer-Guide.md
@@ -34,8 +34,16 @@ This document applies to all SWIRL Editions.
 
 # Architecture
 
-![SWIRL Federated Search Architecture -1](images/swirl_architecture-1.png)
-![SWIRL Federated Search Architecture -2](images/swirl_architecture-2.png)
+## SWIRL AI Connect
+![SWIRL AI Connect Architecture](images/swirl_architecture_1.png)
+
+![SWIRL AI Connect Architecture Part 2](images/swirl_architecture_2.png)
+
+## SWIRL RAG Architecture
+![SWIRL RAG Architecture](images/swirl_architecture_3.png)
+
+## SWIRL AI Co-Pilot
+![SWIRL AI Co-Pilot Architecture](images/swirl_architecture_4.png)
 
 # Workflow
 
@@ -191,11 +199,11 @@ SWIRL AI Connect, Community Edition, supports this using the ChatGPTQueryProcess
 
 ## Adjusting the swirl_score that causes Galaxy UI to star results
 
-The `swirl_score` configuration is available in `theminimumSwirlScore` entry of `static/api/config/default`. 
+For SWIRL Community, this configuration is the `theminimumSwirlScore` entry of `static/api/config/default`. The default value is `100`. Higher values will produce fewer starred results.
 
-The default value is `100`. Higher values will produce fewer starred results.
+For SWIRL Enterprise, the configuration is the `minimumConfidenceScore` entry of the `static/api/config/default`. The default value is .7. Higher values will produce fewer starred results. 
 
-![Galaxy UI with stars](https://raw.githubusercontent.com/swirlai/swirl-search/main/docs/images/3_2_0-Galaxy-star.png)
+![SWIRL AI Connect 4.0 Results](images/swirl_40_results.png)
 
 ## Handle NOTted queries
 
@@ -391,7 +399,29 @@ It is intended for SearchProviders that would otherwise have few (or no) good re
 
 Specify `DATASET` in the `result_mappings` to have SWIRL organize a columnar response into a single result, with the columns in the payload.
 
-![Galaxy UI with Chart Generated from Thoughtspot Dataset Result](images/3_5_0-Chart_UI.png)
+![Galaxy UI with charts displayed](images/swirl_40_chart_display.png)
+
+`DATASET` is fully compatible with `result_mappings`, including `NO_PAYLOAD`. 
+
+SWIRL uses `chart.js` to visualize data sets. The following list explains how it selects the type of chart:
+
+* Field Analysis via checkSupported():
+This method examines the first row of the extracted data:
+It counts how many fields are numeric versus non‑numeric.
+
+* No Numeric Fields:
+A pseudo-count field `count` is added to simulate numeric data. Depending on how many fields are present, it will default to a bar or a stacked bar chart.
+
+* One Numeric Field:
+If there’s only one numeric field (and at least one string/discrete field), the component chooses a bar chart.
+
+* Two Numeric Fields:
+If exactly two numeric fields are present, the component checks the range (difference between maximum and minimum values) of each field. If both ranges are positive, a scatter chart is used; otherwise, it defaults to a bar chart.
+
+* Three or More Numeric Fields:
+With three or more numeric fields, the third field is used to size the bubbles. If the third field’s range is positive, the component chooses a bubble chart; if not, it defaults to a bar chart.
+
+Please [contact support](mailto:[email protected]) if you need help with this feature.
 
 ## Expire Search Objects
 

diff --git a/docs/Developer-Reference.md b/docs/Developer-Reference.md
@@ -946,7 +946,7 @@ Note: Putting a fixed SQL query in the `query_template` is perfectly acceptable.
 
 # Processing Pipelines
 
-![SWIRL Processing Pipelines](images/processing_pipielines.png)
+![SWIRL Processing Pipelines](images/swirl_architecture_1.png)
 
 Processors are intended to be single purpose and executed in a sequence called a "pipeline". Pipelines are specified as JSON lists in their respective properties.  There are four processing pipelines in SWIRL:
 * `Search.pre_query_processors`

diff --git a/docs/M365-Guide.md b/docs/M365-Guide.md
@@ -240,22 +240,17 @@ To:
 }
 ```
 
-# Authenticate to Microsoft
-
-{: .highlight }
-As of SWIRL 3.2.0, when signing in to SWIRL via Microsoft OIDC, users are now automatically authorized to any active M365 SearchProviders.
+# Authenticating with Microsoft
 
 To verify that the SWIRL-M365 integration is working, open the Galaxy UI: <http://localhost:8000> (or <http://localhost:8000/galaxy/>)
 
-Click on the profile icon at the top right and then click the Microsoft toggle to activate it. Depending on how the above configuration was done, M365 may require authentication and/or granting of access. After that, the toggle next to the Microsoft logo will light up, indicating a successful connection.
-
-![SWIRL Search box Auth](images/swirl_searchbox_auth-galaxy_dark.png)
+Click on the profile icon at the top right and then click the Microsoft toggle to activate it. Depending on how the above configuration was done, M365 may require authentication and/or granting of access. 
 
-Now, enter a search in the UI: `metasearch`
+<img src="images/swirl_40_ms_login.png" alt="SSO Provider Login Page" width="300">
 
-Assuming any document in your M365 contains the word `metasearch`, results should appear in just a few seconds:
+After that, the toggle next to the Microsoft logo will light up, indicating a successful connection. You can then query any of the Microsoft sources:
 
-![SWIRL Results Auth](images/swirl_results_auth-galaxy_dark.png)
+![SWIRL Co-Pilot discussion](images/swirl_40_search_msft.png)
 
 {: .warning }
 If the toggle does not light up after authenticating with Microsoft, please [contact support](#support). The [related documentation](#related-documentation) below may also be useful.

diff --git a/docs/Quick-Start-Enterprise.md b/docs/Quick-Start-Enterprise.md
@@ -73,13 +73,13 @@ The output should look like this, or similar:
 
 If the search page appears, click `Log Out` at the top, right. The SWIRL login page will appear:
 
-![SWIRL Login](images/swirl_login-galaxy_dark.png)
+![SWIRL Login](images/swirl_40_login.png)
 
 * Enter the username `admin` and password `password`, then click `Login`.
 
 * Enter a search in the search box and press the `Search` button. Ranked results appear in just a few seconds:
 
-![SWIRL Results No M365](images/swirl_results_no_m365-galaxy_dark.png)
+![SWIRL Results No M365](images/swirl_40_results.png)
 
 If they don’t appear, or an error occurs, please [contact support](#support).
 
@@ -108,7 +108,7 @@ To use different AI providers for RAG and AI Co-Pilot, set the “defaults” li
 * When there is at least one active, default provider for RAG, you may click the “Generate AI Insights” button at the top of the AI Connect form. 
 
 For example:
-![SWIRL Results with RAG](images/swirl_rag_pulmonary_1.png)
+![SWIRL Results with RAG](images/swirl_40_community_rag.png)
 
 * To try the SWIRL AI Co-Pilot, navigate to: [http://localhost:8000/galaxy/chat](http://localhost:8000/galaxy/chat)
 
@@ -118,7 +118,7 @@ Be sure to verify that RAG is working before attempting to use Co-Pilot.
 
 * Direct the co-pilot to search for something of interest, and you should get a RAG result as part of the conversation, including citations:
 
-![SWIRL Co-Pilot Conversation with RAG Result](images/swirl_copilot_chat_rag.png)
+![SWIRL Co-Pilot Conversation with RAG Result](images/swirl_40_enterprise_assistant_rag.png)
 
 * SWIRL ships with admin/password pre-configured as noted. This may cause a warning to appear if using Galaxy with the Google Chrome browser.  
 

diff --git a/docs/Quick-Start.md b/docs/Quick-Start.md
@@ -64,71 +64,42 @@ docker compose up
 After a few minutes, the following or similar should appear:
 
 ``` shell
-redis-1  | 1:C 05 Feb 2024 22:03:51.503 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
-redis-1  | 1:C 05 Feb 2024 22:03:51.503 * Redis version=7.2.4, bits=64, commit=00000000, modified=0, pid=1, just started
-redis-1  | 1:C 05 Feb 2024 22:03:51.503 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
-redis-1  | 1:M 05 Feb 2024 22:03:51.504 * monotonic clock: POSIX clock_gettime
-redis-1  | 1:M 05 Feb 2024 22:03:51.504 * Running mode=standalone, port=6379.
-redis-1  | 1:M 05 Feb 2024 22:03:51.505 * Server initialized
-redis-1  | 1:M 05 Feb 2024 22:03:51.505 * Ready to accept connections tcp
-app-1    | __S_W_I_R_L__3_._2_._0__________________________________________________________
-app-1    | 
-app-1    | Setting Up Swirl:
-app-1    | Checking Migrations:
-app-1    | 
-app-1    | Migrating:
-app-1    | 
-app-1    | Operations to perform:
-app-1    |   Apply all migrations: admin, auth, authtoken, contenttypes, django_celery_beat, sessions, swirl
-app-1    | Running migrations:
-app-1    |   No migrations to apply.
-app-1    | 
-app-1    | 
-app-1    | Collecting Statics:
-app-1    | 
-app-1    | 
-app-1    | 246 static files copied to '/app/static'.
-app-1    | 
-app-1    | Ok
-app-1    | Command successful!
-app-1    | __S_W_I_R_L__3_._2_._0__________________________________________________________
-app-1    | 
-app-1    | Warning: logs directory does not exist, creating it
-app-1    | ParseResult(scheme='redis', netloc='redis:6379', path='/0', params='', query='', fragment='') checked.
-app-1    | ParseResult(scheme='redis', netloc='redis:6379', path='/0', params='', query='', fragment='') checked.
-app-1    | Start: celery-worker -> celery -A swirl_server worker --loglevel INFO ... Ok, pid: 50
-app-1    | Start: celery-beats -> celery -A swirl_server beat --scheduler django_celery_beat.schedulers:DatabaseScheduler ... Ok, pid: 72
+app-1    | Start: celery-worker -> celery -A swirl_server worker ... Ok, pid: 55
+app-1    | Start: celery-beats -> celery -A swirl_server beat --scheduler django_celery_beat.schedulers:DatabaseScheduler ... Ok, pid: 81
 app-1    | Updating .swirl... Ok
 app-1    | 
 app-1    |   PID TTY          TIME CMD
-app-1    |    50 ?        00:00:03 celery
+app-1    |    55 ?        00:00:05 celery
+app-1    |    81 ?        00:00:04 celery
 app-1    | 
-app-1    | You're using version 3.2.0 of Swirl, the current version.
+app-1    | setting up logging...
+app-1    | setting up logging DONE
+app-1    | You're using version 4.0.0.0 of Swirl, the current version.
 app-1    | Command successful!
-app-1    | 2024-02-05 17:04:14,131 INFO     Starting server at tcp:port=8000:interface=0.0.0.0
-app-1    | 2024-02-05 17:04:14,131 INFO     HTTP/2 support not enabled (install the http2 and tls Twisted extras)
-app-1    | 2024-02-05 17:04:14,131 INFO     Configuring endpoint tcp:port=8000:interface=0.0.0.0
-app-1    | 2024-02-05 17:04:14,132 INFO     Listening on TCP address 0.0.0.0:8000
+app-1    | INFO 2025-02-17 15:45:00 cli Starting server at tcp:port=8000:interface=0.0.0.0
+app-1    | INFO 2025-02-17 15:45:00 server HTTP/2 support not enabled (install the http2 and tls Twisted extras)
+app-1    | INFO 2025-02-17 15:45:00 server Configuring endpoint tcp:port=8000:interface=0.0.0.0
+app-1    | INFO 2025-02-17 15:45:00 server Listening on TCP address 0.0.0.0:8000
 ```
 
 * Open this URL with a browser: <http://localhost:8000> (or <http://localhost:8000/galaxy>)
 
 If the search page appears, click `Log Out` at the top, right. The SWIRL login page will appear:
 
-![SWIRL Login](images/swirl_login-galaxy_dark.png)
+<img src="images/swirl_40_login.png" alt="SWIRL 4.0 Login" width="300">
 
 * Enter the username `admin` and password `password`, then click `Login`.
 
 * Enter a search in the search box and press the `Search` button. Ranked results appear in just a few seconds:
 
-![SWIRL Results No M365](images/swirl_results_no_m365-galaxy_dark.png)
+![SWIRL AI Connect 4.0 Results](images/swirl_40_results.png)
 
 * Click the `Generate AI Insight` button to RAG using the most relevant results. 
 
 {: .warning }
-As noted above, you must have setup OpenAI or Azure/OpenAI prior to executing this step.
+As noted above, if using the Community Edition, you *must* setup OpenAI or Azure/OpenAI prior to executing this step.
 
-![SWIRL Results with RAG](images/swirl_rag_pulmonary_1.png)
+![SWIRL AI Connect 4.0 Results with RAG](images/swirl_40_community_rag.png)
 
 * Click the profile avatar in the upper right corner of the Galaxy UI. Then click [Manage SWIRL](http://localhost:8000/swirl/) to explore the rest of SWIRL's features.