Skip to content

Internationalization Support for Leo system prompts (For locales fr, de, it, es) #41554

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
5 of 6 tasks
bbondy opened this issue Oct 10, 2024 · 4 comments · Fixed by brave/brave-core#25931
Closed
5 of 6 tasks

Comments

@bbondy
Copy link
Member

bbondy commented Oct 10, 2024

Description

Internationalization Support for Leo

Brave browser sends up a selected_language and a system_language with prompts.

The selected_language is initially an empty string, but the server can send a SelectedLanguage event to set the language for future.

The server sets a selected_language based on what it detects the first message's language to be in. If it can't make that determine or that message is not long enough it can fallback to using the system language.
Or if there is no system language specified it will use English.

More details here:
https://docs.google.com/document/d/1IWQNX-E40ujaQFQVavZEtgK3J068PgZ_xDbSLDWZOD8/edit?tab=t.0#heading=h.miolul91b7va

Steps to reproduce

Type in "Bonjour, ça vas" without quotes to Leo?
Repeat similarly for Spanish, German, and Italian.
You can test with Release channel to see if you reproduce before verifying the fix in this issue.

Here are some other test strings:
English
was there an assassination attempt against Donald Trump?

French
y a-t-il eu une nouveau tentative d'assassinat contre Donald Trump ?

Spanish:
¿Hubo un intento de asesinato contra Donald Trump?

Italian:
c'è stato un tentativo di omicidio contro Donald Trump?

German:
var der et mordforsøg mod Donald Trump?

Actual result

Reply is in English

Expected result

Reply should be in the user's language. We require 3 words at least to determine the language, and otherwise it uses the system language to convert the system prompts for.

Reproduces how often

Easily reproduced

Desktop Brave version (brave://version info)

1.72.61

Android device

n/a

Channel information

  • release (stable)
  • beta
  • nightly

Reproducibility

  • with Brave Shields disabled
  • with Brave Rewards disabled
  • in the latest version of Chrome

Miscellaneous information

No response

@bbondy bbondy added OS/Android Fixes related to Android browser functionality OS/Desktop OS/iOS Fixes related to iOS browser functionality QA/Yes release-notes/include labels Oct 10, 2024
@bbondy bbondy changed the title Internationalization Support for Leo system prompts Internationalization Support for Leo system prompts (For locales fr, de, it, es) Oct 10, 2024
@brave-builds brave-builds added this to the 1.73.x - Nightly milestone Oct 21, 2024
@rebron rebron added the i18n label Oct 21, 2024
@kjozwiak
Copy link
Member

The above requires 1.72.90 or higher for 1.72.x verification 👍

@Uni-verse Uni-verse added the QA/In-Progress Indicates that QA is currently in progress for that particular issue label Nov 1, 2024
@Uni-verse
Copy link
Contributor

Uni-verse commented Nov 1, 2024

Verified on Samsung Galaxy S21 using the following version(s):

Brave	1.72.93 Chromium: 131.0.6778.14 (Official Build) beta (64-bit) 
Revision	69e5dc20ca483a13316632df5bcd9279bb2a3cb7
OS	Android 13; Build/TP1A.220624.014; 33; REL

Test Plan: #41554 (comment)

  1. Set device language to non-english
  2. Sent some test strings in fr, de, it, es-us and en-us
  3. Confirmed response matches the language used by the user
  4. Sent test strings under 3 words to test conversion logic
  5. Confirmed response falls back to system language when less than 3 words are used.
  6. Confirmed internationalization is working with the opt-in flow.
Spanish Italian German then English German French < 3 words
Image Image Image Image Image Image

@Uni-verse Uni-verse added QA Pass - Android ARM and removed QA/In-Progress Indicates that QA is currently in progress for that particular issue labels Nov 1, 2024
@srirambv
Copy link
Contributor

srirambv commented Nov 8, 2024

Verification passed on

Brave 1.73.86 Chromium: 131.0.6778.39 (Official Build) (64-bit)
Revision 52163bcf4e40f27ddb76ffa79c90a8833084a9bd
OS Windows 11 Version 23H2 (Build 22631.4317)
  • Verified steps from brave/brave-core#25931
  • Verified query input in non-English languages produce response in the same language
  • Verified when input is less than 3 words, Leo fallback to system language for response
41554.mp4

@hffvld hffvld added the QA/In-Progress Indicates that QA is currently in progress for that particular issue label Nov 14, 2024
@hffvld
Copy link
Contributor

hffvld commented Nov 14, 2024

Verified on iPhone 14using version(s):

Device/OS: iPhone 14 / iOS 17.7.1
Brave build: 1.73 (89)
BraveCore: 1.73.89 (131.0.6778.69)

Language/Region set to en_US

STEPS:

  1. Set device language/region to en_US
  2. Launch Brave
  3. Open Leo
  4. Send a query:
  • de_DE var der et mordforsøg mod Donald Trump?
  • it_IT c'è stato un tentativo di omicidio contro Donald Trump?
  • es_ES ¿Hubo un intento de asesinato contra Donald Trump?
  • fr_FR y a-t-il eu une nouveau tentative d'assassinat contre Donald Trump ?
  • en_US was there an assassination attempt against Donald Trump?
  1. Verify response
  2. Send another query:
  • de_DE Guten Morgen
  • it_IT Buongiorno
  • es_ES Buen día
  • fr_FR Bonjour
  • en_US Good morning

ACTUAL RESULTS:

  • Verified that if device language/region is set to en_US, then Leo's response returned in the same language used to send a query if a query is longer than 3 words.

en_US German Italian Spanish French English
1 2 3 4 5 6
Language/Region set to de_DE

STEPS:

  1. Set device language/region to de_DE
  2. Launch Brave
  3. Open Leo
  4. Send a query:
  • de_DE Guten Morgen
  • it_IT Buongiorno
  • es_ES Buen día
  • fr_FR Bonjour
  • en_US Good morning
  1. Verify response

ACTUAL RESULTS:

  • Verified that if a query is less than 3 words, then the response comes in the same language that is set in device settings.

de_DE German Italian Spanish French English
1 2 3 4 5 6

@hffvld hffvld added QA Pass - iPhone and removed QA/In-Progress Indicates that QA is currently in progress for that particular issue labels Nov 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants