Reduce amount of stuff we fetch on `/persona` #4988

Weves · 2025-07-04T00:48:51Z

Description

Fixes https://linear.app/danswer/issue/DAN-2161/improve-latency-of-persona-endpoint

How Has This Been Tested?

Tested persona / assistant functionality locally

Backporting (check the box to trigger backport action)

Note: You have to check that the action passes, otherwise resolve the conflicts manually and tag the patches.

This PR should be backported (make sure to check that the backport attempt succeeds)
[Optional] Override Linear Check

vercel · 2025-07-04T00:48:56Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
internal-search	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 9, 2025 6:36pm

greptile-apps

PR Summary

Optimizes the persona/assistants API performance by introducing a MinimalPersonaSnapshot type to reduce payload size and database load time.

Introduced PersonaLoadType enum in backend to control granularity of data loading (NONE, MINIMAL, FULL) in backend/onyx/db/persona.py
Replaced full Persona type with MinimalPersonaSnapshot across frontend components, significantly reducing data transfer payload
Switched from selectinload to joinedload in persona queries for better SQL query performance in backend/onyx/db/persona.py
Consolidated assistant data fetching into new useAdminPersonas hook with SWR caching in web/src/app/admin/assistants/hooks.ts
Simplified persona retrieval logic by removing unnecessary user files/folders checks in web/src/app/chat/lib.tsx

_{48 files reviewed, 14 comments}
_{Edit PR Review Bot Settings | Greptile}

web/src/lib/assistants/fetchAssistantsSS.ts

web/src/components/Field.tsx

greptile-apps · 2025-07-04T23:27:47Z

web/src/components/context/AssistantsContext.tsx


-      let filteredAssistants = filterAssistants(
-        assistants,
-        hasAnyConnectors,
-        hasImageCompatibleModel
-      );
+      let filteredAssistants = filterAssistants(assistants);


logic: filterAssistants is called without previously required params. Make sure this function has been updated to handle this change in its implementation

greptile-apps · 2025-07-04T23:27:47Z

web/src/components/context/AssistantsContext.tsx

        .map((id) => assistants.find((assistant) => assistant.id === id))
-        .filter((assistant): assistant is Persona => assistant !== undefined);
+        .filter(
+          (assistant): assistant is MinimalPersonaSnapshot =>
+            assistant !== undefined
+        );


style: Consider extracting this common pinned assistants filter logic into a shared utility function since it's duplicated in the useEffect

greptile-apps · 2025-07-04T23:27:50Z

web/src/app/admin/assistants/PersonaTable.tsx

+    const reorderedPersonas = orderedPersonaIds.map(
+      (id) => personas.find((persona) => persona.id.toString() === id)!
    );


logic: Add null check here. The ! operator assumes the find will always succeed, but this could throw if the ID doesn't exist

Suggested change

const reorderedPersonas = orderedPersonaIds.map(

(id) => personas.find((persona) => persona.id.toString() === id)!

);

const reorderedPersonas = orderedPersonaIds.map((id) => {

const persona = personas.find((p) => p.id.toString() === id);

if (!persona) {

throw new Error(`Persona with id ${id} not found`);

}

return persona;

});

web/src/app/admin/assistants/interfaces.ts

backend/onyx/db/persona.py

greptile-apps · 2025-07-04T23:27:52Z

web/src/app/admin/assistants/page.tsx

+            error?.info?.message ||
+            error?.info?.detail ||
+            "An unknown error occurred"


style: Consider using more specific error types instead of optional chaining through generic error object

greptile-apps · 2025-07-04T23:27:53Z

backend/onyx/server/features/persona/models.py

+            owner=(
+                MinimalUserSnapshot(id=persona.user.id, email=persona.user.email)
+                if persona.user
+                else None
+            ),


logic: persona.user is accessed before the null check. Move the persona.user.id and email access inside the parentheses to prevent potential NullPointerException.

Suggested change

owner=(

MinimalUserSnapshot(id=persona.user.id, email=persona.user.email)

if persona.user

else None

),

owner=(

MinimalUserSnapshot(id=user.id, email=user.email)

if (user := persona.user)

else None

),

greptile-apps · 2025-07-04T23:27:54Z

web/src/app/admin/assistants/hooks.ts

+  const url = buildApiPath("/api/admin/persona", {
+    include_deleted: includeDeleted.toString(),
+    get_editable: getEditable.toString(),
+  });


style: Converting booleans to strings manually is error-prone. Consider adding URLSearchParams utility

evan-onyx

some nits and efficiency things that might help

backend/onyx/db/persona.py

evan-onyx · 2025-07-05T22:59:34Z

backend/onyx/db/persona.py

+            # Used for filtering
+            joinedload(Persona.labels),
+            # only show document sets in the UI that the assistant has access to
+            joinedload(Persona.document_sets),


Probably should include these from below since they get used in the minimalsnapshot:

.joinedload(DocumentSet.connector_credential_pairs) .joinedload(ConnectorCredentialPair.connector), joinedload(Persona.document_sets) .joinedload(DocumentSet.connector_credential_pairs) .joinedload(ConnectorCredentialPair.credential),

Also these:

joinedload(Persona.document_sets).joinedload(DocumentSet.users), joinedload(Persona.document_sets).joinedload(DocumentSet.groups),

Also on user

evan-onyx · 2025-07-05T23:03:07Z

web/src/app/admin/assistants/PersonaTable.tsx

@@ -119,7 +120,7 @@ export function PersonasTable() {
    if (personaToDelete) {
      const response = await deletePersona(personaToDelete.id);
      if (response.ok) {
-        await refreshAssistants();
+        refreshPersonas();


potential greptile w?

web/src/app/admin/assistants/interfaces.ts

evan-onyx · 2025-07-05T23:08:18Z

web/src/lib/assistants/utils.ts

  let filteredAssistants = assistants.filter(
    (assistant) => assistant.is_visible
  );
-
-  if (!hasAnyConnectors) {


don't need these anymore?

This reverts commit adf48de.

#5024)" This reverts commit f7ed7cd.

* Revert "Revert "Reduce amount of stuff we fetch on `/persona` (#4988)" (#5024)" This reverts commit f7ed7cd. * Enhancements / fix re-render * re-arrange * greptile

vercel bot deployed to Preview July 4, 2025 00:53 View deployment

vercel bot had a problem deploying to Preview July 4, 2025 18:24 Failure

vercel bot deployed to Preview July 4, 2025 18:35 View deployment

vercel bot deployed to Preview July 4, 2025 22:24 View deployment

vercel bot deployed to Preview July 4, 2025 22:37 View deployment

Weves changed the title ~~claude stuff~~ Reduce amount of stuff we fetch on /persona Jul 4, 2025

Weves marked this pull request as ready for review July 4, 2025 23:27

Weves requested a review from a team as a code owner July 4, 2025 23:27

greptile-apps bot reviewed Jul 4, 2025

View reviewed changes

vercel bot deployed to Preview July 4, 2025 23:31 View deployment

evan-onyx approved these changes Jul 5, 2025

View reviewed changes

Weves added 8 commits July 9, 2025 10:44

claude stuff

0e53f34

Send over less Assistant data

3176c67

more

5baff11

Fix build

cb0c9b3

Fix mypy

1e50a4a

fix

fad732f

small tweak

3ee5653

Address EL cmments

47ea4a0

Weves force-pushed the improve-persona-api-speed branch from e1e3a01 to 47ea4a0 Compare July 9, 2025 17:45

vercel bot had a problem deploying to Preview July 9, 2025 17:47 Failure

fix

7cc6845

vercel bot had a problem deploying to Preview July 9, 2025 18:09 Failure

Fix build

ed6c756

vercel bot deployed to Preview July 9, 2025 18:36 View deployment

Weves merged commit adf48de into main Jul 12, 2025
13 of 14 checks passed

Weves deleted the improve-persona-api-speed branch July 12, 2025 21:15

Weves added a commit that referenced this pull request Jul 14, 2025

Revert "Reduce amount of stuff we fetch on /persona (#4988)"

2b16c3e

This reverts commit adf48de.

greptile-apps bot mentioned this pull request Jul 14, 2025

Revert "Reduce amount of stuff we fetch on /persona (#4988)" #5024

Merged

2 tasks

Weves added a commit that referenced this pull request Jul 14, 2025

Revert "Reduce amount of stuff we fetch on /persona (#4988)" (#5024)

f7ed7cd

This reverts commit adf48de.

Weves added a commit that referenced this pull request Jul 15, 2025

Revert "Revert "Reduce amount of stuff we fetch on /persona (#4988)" (

6604e1f

#5024)" This reverts commit f7ed7cd.

Weves added a commit that referenced this pull request Jul 15, 2025

Persona simplification r2 (#5031)

3bb58a3

* Revert "Revert "Reduce amount of stuff we fetch on `/persona` (#4988)" (#5024)" This reverts commit f7ed7cd. * Enhancements / fix re-render * re-arrange * greptile

Weves added a commit that referenced this pull request Jul 15, 2025

Persona simplification r2 (#5031)

8646247

* Revert "Revert "Reduce amount of stuff we fetch on `/persona` (#4988)" (#5024)" This reverts commit f7ed7cd. * Enhancements / fix re-render * re-arrange * greptile

-    const reorderedPersonas = orderedPersonaIds.map(
-      (id) => personas.find((persona) => persona.id.toString() === id)!
-    );
+    const reorderedPersonas = orderedPersonaIds.map((id) => {
+      const persona = personas.find((p) => p.id.toString() === id);
+      if (!persona) {
+        throw new Error(`Persona with id ${id} not found`);
+      }
+      return persona;
+    });

Reduce amount of stuff we fetch on /persona #4988

Reduce amount of stuff we fetch on /persona #4988

Uh oh!

Conversation

Weves commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

How Has This Been Tested?

Backporting (check the box to trigger backport action)

Uh oh!

vercel bot commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

PR Summary

Uh oh!

Uh oh!

Uh oh!

greptile-apps bot Jul 4, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Jul 4, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Jul 4, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

greptile-apps bot Jul 4, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Jul 4, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Jul 4, 2025

Choose a reason for hiding this comment

Uh oh!

evan-onyx left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

evan-onyx Jul 5, 2025

Choose a reason for hiding this comment

Uh oh!

evan-onyx Jul 5, 2025

Choose a reason for hiding this comment

Uh oh!

evan-onyx Jul 5, 2025

Choose a reason for hiding this comment

Uh oh!

evan-onyx Jul 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

evan-onyx Jul 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reduce amount of stuff we fetch on `/persona` #4988

Reduce amount of stuff we fetch on `/persona` #4988

Weves commented Jul 4, 2025 •

edited

Loading

vercel bot commented Jul 4, 2025 •

edited

Loading