Error no in app info in response error on initial IAP via production - follow up of #37007

[kjozwiak]

Description:

When attempting to do a IAP for Leo via production once the builds are uploaded/available via the App Store, users who initially purchase Leo will receive an Error message due to the following:

{
  "message": "Error no in app info in response",
  "errorCode": "validation_failed",
  "code": 400,
  "data": {
    "validationErrors": {
      "receiptErrors": "no in app info in response"
    }
  }
}

Once the user hits the above error/issue, Leo has technically been purchased but the user isn't converted to a Premium user. When a user is in the above state, they'll need to basically Restore via the IAP paywall model. However, when you're in the above state, you'll notice that Brave Leo is under the OS's valid/available subscriptions.

Steps to Reproduce

1. download/install 1.63.182 Chromium: 122.0.6261.128 via the App Store (this is the 1.63.1 release)
2. run through a IAP for Leo via Hamburger Menu -> Settings -> Leo -> Go Premium
3. run through the Leo paywall modal and complete a purchase

You'll get the 400 - Error no in app info in response error and you'll notice that the user wasn't converted into Premium.

Actual result:

Example	Example

RPReplay_Final1711466857.MP4

Expected result:

IAP should work

Reproduces how often: [Easily reproduced, Intermittent Issue]

1.63.182 | Chromium: 122.0.6261.128 (Official Build) unknown (64-bit)
--- | ---
Revision | aba689383f95...
OS | iOS

Brave Version:

• Can you reproduce this issue with the most recent build from TestFlight? Yes
• Can you reproduce this issue with the previous version of the current build from TestFlight? Yes
• Can you reproduce this issue with the current build from AppStore? Yes

Device details:

• iPhone 11 running iOS 17.4.1 - Reproduced

Website problems only:

• Does the issue resolve itself when disabling Brave Shields? N/A
• Is the issue reproducible on the latest version of Mobile Safari? N/A

Additional Information

[kjozwiak]

Closing as the above is hopefully resolved by brave/brave-core#22791.

[kjozwiak]

The above requires 1.63.183 or higher for 1.63.x verification 👍

[hffvld]

Verified in TF on iPhone 14 using version(s):

Device/OS: iPhone 14 / iOS 17.4.1
Brave build: 1.63.2 (183)
BraveCore: 1.63.183 (122.0.6261.128)

---

STEPS:

1. Follow the steps from Error no in app info in response error on initial IAP via production - follow up of #37007 #37150 (comment)
2. Verify

ACTUAL RESULTS:

• Verified that no errors are shown when purchasing a new Leo subscription
• Verified that no rate limit is seen

---

2024-03-28_10-11-18.mp4

[kjozwiak]

Verification PASSED on iPhone 11 running iOS 17.4.1 using the following build(s):

1.63.183 | Chromium: 122.0.6261.128 (Official Build) unknown (64-bit) 
--- | ---
Revision | 442293d4d758...
OS | iOS

So unfortunately I ran into the Error again with the latest 1.63.2 build but this time around, when I pressed OK on the error modal, the subscription/credentials looked like they were loaded. I then summarized a few pages/follow up questions and didn't get the rate limit. However, I also ran into a few issues.

Here's the case that I ran through:

• using a brand new account that has never used TF before, I bought VPN via TF and that worked without any issues (however, I didn't install the profile/enable the VPN as all I cared about was the purchase)
• once VPN was purchased, I went through the Leo purchase using Settings -> Leo -> Go Premium

Notes:

• ~38s into the video, you'll notice that I'm starting to run through the Leo purchase on staging via TF using a brand new Apple ID
• ~1:07min -- you'll notice that I hit the Error message. I waited ~5-6s to see if anything happened but it wasn't being dismissed. I clicked on OK and then noticed that all my subscription information was filled in and I was set as Premium
• ~4:15min -- When Mixtral started summarizing this question, it basically stopped at ~4:30s and completely froze the entire UI/application. I waited ~10-15s or so and it finally completed the summarization, unfroze the UI thread and then displayed the last remaining Suggested question
• once the above happened, Leo basically stopped working in that instance/tab. I tried asking it a question via ~5:10min and attempted to summarize another page within the same instance via ~6:06min and it wouldn't do anything (waited ~15-20s each time). I could never get Mixtral to work in that instance after the UI froze.
• once I opened a new tab after experiencing ☝️, Mixtral worked in the new instance

So re: the Error, there's a high chance that users might still see the error. Maybe we'll just experience it on TF and not production. If users are still getting the Error but recover/end up pulling/receiving the credentials, we'll probably continue pushing the build live and in the meantime, we'll look into improving the purchase/IAP flow. Hopefully we'll get more data back from users (this is assuming we continue pushing live if production looks good once we go live with 1.63.2).

RPReplay_Final1711660340.MP4

[kjozwiak]

iPhone 11 running iOS 17 verifications

Account #1 (Brand new account/never ran through Leo IAP)

IAP before hitting rate limit

Verification PASSED on iPhone 11 running iOS 14 using the following build(s):

1.63.183 Chromium: 122.0.6261.128 (Official Build) unknown (64-bit)
--- | --- 
Revision | 442293d4d758...
OS | iOS

• installed 1.63.183 Chromium: 122.0.6261.128 on a new account that has never completed a IAP re: Leo
• once installed, quickly went through the opt-in/onboarding and T&S
• once I was opted into Leo, went through the IAP flow via Hamburger Menu -> Settings -> Leo -> Get Premium
• ensured that there was no Error modal that appeared once the purchase has been completed
• ensured that the UI/UX indicated that the user has been converted into a Premium user
• went through ~20 Mixtral queries/summarizations and suggested questions and ensured that there was no rate limit
• went through ~20 Claude queries/summarizations and suggested questions and ensured that there was no rate limit

RPReplay_Final1712083763.MP4

There's an example of the network calls/flows via https://github.com/brave/internal/issues/1133#issue-2221255905. We're still seeing the Error but after ~4-5 POST calls, we eventually send the orderId and get the receipt.

Linking current account with account.brave.com

Once the IAP was completed and verified that the user is set as Premium user and is not hitting the rate limit, ensured that you can link the IAP with account.brave.com without any issues via the Link purchase to your Brave account feature as per the following:

• opened Hamburger Menu -> Settings -> Leo and tapped on Link purchase to your Brave account
• ensured that https://account.brave.com/?intent=link-order&product=leo was being opened
• used the same email that was used for the IAP/Apple ID and ensured that I received a Access your Brave account email
• clicked on the Confirmation link and ensured that account.brave.com opened
• ensured that tapping on Link my subscription now worked and a user the Success! message
• ensured Leo is listed as a Subscription plan and currently has active credentials loaded
• ensured that there's a Purchased with iOS indicator on the Leo subscription

Example	Example	Example	Example	Example	Example

Linking from account.brave.com from a previous iOS purchase

• installed 1.63.183 Chromium: 122.0.6261.128 on a new account that has never completed a IAP re: Leo
• once installed, quickly went through the opt-in/onboarding and T&S
• once I was opted into Leo, went through the linking process via Hamburger Menu -> Settings -> Leo
• ensured that tapping on Refresh Leo worked and a user the YOU HAVE ACTIVE CREDENTIALS LOADED! message
• ensured Leo is listed as a Subscription plan and currently has active credentials loaded
• went through ~20 Mixtral queries/summarizations and suggested questions and ensured that there was no rate limit
• went through ~20 Claude queries/summarizations and suggested questions and ensured that there was no rate limit

Example	Example	Example	Example	Example	Example	Example

Account #2 (Brand new account/never ran through Leo IAP)

IAP before hitting rate limit

Verification PASSED on iPhone 11 running iOS 14 using the following build(s):

1.63.183 Chromium: 122.0.6261.128 (Official Build) unknown (64-bit)
--- | --- 
Revision | 442293d4d758...
OS | iOS

• installed 1.63.183 Chromium: 122.0.6261.128 on a new account that has never completed a IAP re: Leo
• once installed, quickly went through the opt-in/onboarding and T&S
• once I was opted into Leo, went through the IAP flow via Hamburger Menu -> Settings -> Leo -> Get Premium
• ensured that there was no Error modal that appeared once the purchase has been completed
• ensured that the UI/UX indicated that the user has been converted into a Premium user
• went through ~20 Mixtral queries/summarizations and suggested questions and ensured that there was no rate limit
• went through ~20 Claude queries/summarizations and suggested questions and ensured that there was no rate limit

RPReplay_Final1712106824.MP4

[Uni-verse]

Verified on iPhone X using version 16.7 using version 1.63.2

IAP after hitting rate limit (Brand new account with clean install)

• Ensured that Premium upgrade screen was shown when reaching rate limit.
• Ensured that IAP purchase flow is successful and Premium status is activated.
• Ensured that sending 20+ queries after upgrading does not limit user.
• Ensured that voice recognition is working in chat.

IAP_flow1.mp4

Restore credentials (linked from Android IAP)

• Ensured that loading account.brave.com will enable Premium status.
• Ensured that sending 20+ queries between multiple tabs does not.

Restore_flow1.mp4

[hffvld]

Verified on iPhone 14using version(s):

Device/OS: iPhone 14 / iOS 17.4.1
Brave build: 1.63.2 (183)
BraveCore: 1.63.183 (122.0.6261.128)

---

IAP subscription before hitting the rate limit

STEPS:

1. Add brand new account on device
2. Install and launch Brave
3. Go to Leo or Settings > Leo
4. Purchase new subscription
5. Open Leo > Verify that Premium subscription is shown there
6. Set language model to Mixtral > Run ~20 queries > Verify that no rate limit
7. Set language model to Claude > Run ~20 queries > Verify that no rate limit

ACTUAL RESULTS:

• Verified that IAP for new Leo subscription is working.
• Verified that rate limit message is not shown after sending ~20 queries.

---

2024-04-02_14-21-28.mp4