[Due for payment 2025-04-28] [$250] Unable to save search query name that contains before and after dates

[m-natarajan]

If you haven’t already, check out our contributing guidelines for onboarding and email [email protected] to request to join our Slack channel!

---

Version Number: 9.1.20-1
Reproducible in staging?: Y
Reproducible in production?:
If this was caught on HybridApp, is this reproducible on New Expensify Standalone?:
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: @dominictb
Slack conversation (hyperlinked to channel name): #Expensify Bugs

Action Performed:

1. Go to Reports
2. Click filters
3. Click Date field
4. Select before and after date
5. Click Save
6. Click save search
7. Click three dot menu on the created search
8. Click Rename
9. Click Save

Expected Result:

Should save successfully

Actual Result:

“Invalid character.” Error shows up

Workaround:

Unknown

Platforms:

Which of our officially supported platforms is this issue occurring on?

• Android: Standalone
• Android: HybridApp
• Android: mWeb Chrome
• iOS: Standalone
• iOS: HybridApp
• iOS: mWeb Safari
• MacOS: Chrome / Safari
• MacOS: Desktop

Screenshots/Videos

Add any screenshot/video evidence

Screen.Recording.2025-03-27.at.22.06.33.mov

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~021905581416164321466
• Upwork Job ID: 1905581416164321466
• Last Price Increase: 2025-04-04
• Automatic offers:
• dominictb | Contributor | 106824880

Issue Owner

Current Issue Owner: @marcaaron

[melvin-bot]

Triggered auto assignment to @twisterdotcom (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details. Please add this bug to a GH project, as outlined in the SO.

[m-natarajan]

proposal by @dominictb

Proposal

Please re-state the problem that we are trying to solve in this issue.

Cannot save search query name that contains before and after date

What is the root cause of that problem?

When saving the search, we validate using logics here:

App/src/components/Form/FormProvider.tsx

Line 116 in da9797b

const onValidate = useCallback(

There's a part where we check if the string contains html tags:

App/src/components/Form/FormProvider.tsx

Lines 142 to 160 in da9797b

	const matchedHtmlTags = inputValue.match(CONST.VALIDATE_FOR_HTML_TAG_REGEX);
	let isMatch = CONST.WHITELISTED_TAGS.some((regex) => regex.test(inputValue));
	// Check for any matches that the original regex (foundHtmlTagIndex) matched
	if (matchedHtmlTags) {
	// Check if any matched inputs does not match in WHITELISTED_TAGS list and return early if needed.
	for (const htmlTag of matchedHtmlTags) {
	isMatch = CONST.WHITELISTED_TAGS.some((regex) => regex.test(htmlTag));
	if (!isMatch) {
	break;
	}
	}
	}
	if (isMatch && leadingSpaceIndex === -1) {
	return;
	}
	// Add a validation error here because it is a string value that contains HTML characters
	validateErrors[inputID] = translate('common.error.invalidCharacter');

Looking at the query name here:

type:expense status:all date<2025-03-13 date>2025-03-12

We see there's a part here:

<2025-03-13 date>

This part will match VALIDATE_FOR_HTML_TAG_REGEX:

App/src/CONST.ts

Line 1695 in da9797b

VALIDATE_FOR_HTML_TAG_REGEX: /<([^>\s]+)(?:[^>]*?)>/g,

But it's not one of the whitelisted HTML tags:

https://github.com/Expensify/App/blob/main/src/CONST.ts#L1704

So the conditions here will not be returned early:

App/src/components/Form/FormProvider.tsx

Lines 145 to 157 in da9797b

	if (matchedHtmlTags) {
	// Check if any matched inputs does not match in WHITELISTED_TAGS list and return early if needed.
	for (const htmlTag of matchedHtmlTags) {
	isMatch = CONST.WHITELISTED_TAGS.some((regex) => regex.test(htmlTag));
	if (!isMatch) {
	break;
	}
	}
	}
	if (isMatch && leadingSpaceIndex === -1) {
	return;
	}

And error is shown

App/src/components/Form/FormProvider.tsx

Line 160 in da9797b

validateErrors[inputID] = translate('common.error.invalidCharacter');

What changes do you think we should make in order to solve the problem?

The regex used to check for HTML here is good, but it leaves some false positive cases like the one above, while fails to match some tags like this one:

I found a regex on this blog post that works pretty well in this case:

/</?\w*((\s+\w+(\s*=\s*(?:"(.|\n)?"|'(.|\n)?'|[^'">\s]+))?)+\s*|\s*)/?>/g

We should apply this regex to VALIDATE_FOR_HTML_TAG_REGEX, and based on the comments on the constant, we should update BE as well.

What specific scenarios should we cover in automated tests to prevent reintroducing this issue in the future?

Create unit test to test for possible HTML tags with VALIDATE_FOR_HTML_TAG_REGEX

What alternative solutions did you explore? (Optional)

Or we can pass allowHTML to FormProvider to ignore the check.

[dominictb]

Repost my proposal here due to formatting issue

---

Proposal

Please re-state the problem that we are trying to solve in this issue.

Cannot save search query name that contains before and after date

What is the root cause of that problem?

When saving the search, we validate using logics here:

App/src/components/Form/FormProvider.tsx

Line 116 in da9797b

const onValidate = useCallback(

There's a part where we check if the string contains html tags:

App/src/components/Form/FormProvider.tsx

Lines 142 to 160 in da9797b

	const matchedHtmlTags = inputValue.match(CONST.VALIDATE_FOR_HTML_TAG_REGEX);
	let isMatch = CONST.WHITELISTED_TAGS.some((regex) => regex.test(inputValue));
	// Check for any matches that the original regex (foundHtmlTagIndex) matched
	if (matchedHtmlTags) {
	// Check if any matched inputs does not match in WHITELISTED_TAGS list and return early if needed.
	for (const htmlTag of matchedHtmlTags) {
	isMatch = CONST.WHITELISTED_TAGS.some((regex) => regex.test(htmlTag));
	if (!isMatch) {
	break;
	}
	}
	}
	if (isMatch && leadingSpaceIndex === -1) {
	return;
	}
	// Add a validation error here because it is a string value that contains HTML characters
	validateErrors[inputID] = translate('common.error.invalidCharacter');

Looking at the query name here:

type:expense status:all date<2025-03-13 date>2025-03-12

We see there's a part here:

<2025-03-13 date>

This part will match VALIDATE_FOR_HTML_TAG_REGEX:

App/src/CONST.ts

Line 1695 in da9797b

VALIDATE_FOR_HTML_TAG_REGEX: /<([^>\s]+)(?:[^>]*?)>/g,

But it's not one of the whitelisted HTML tags:

https://github.com/Expensify/App/blob/main/src/CONST.ts#L1704

So the conditions here will not be returned early:

App/src/components/Form/FormProvider.tsx

Lines 145 to 157 in da9797b

	if (matchedHtmlTags) {
	// Check if any matched inputs does not match in WHITELISTED_TAGS list and return early if needed.
	for (const htmlTag of matchedHtmlTags) {
	isMatch = CONST.WHITELISTED_TAGS.some((regex) => regex.test(htmlTag));
	if (!isMatch) {
	break;
	}
	}
	}
	if (isMatch && leadingSpaceIndex === -1) {
	return;
	}

And error is shown

App/src/components/Form/FormProvider.tsx

Line 160 in da9797b

validateErrors[inputID] = translate('common.error.invalidCharacter');

What changes do you think we should make in order to solve the problem?

The regex used to check for HTML here is good, but it leaves some false positive cases like the one above, while fails to match some tags like this one:

<comment id="<123 >">

I found a regex on this blog post that works pretty well in this case:

/<\/?\w*((\s+\w+(\s*=\s*(?:"(.|\n)*?"|'(.|\n)*?'|[^'">\s]+))?)+\s*|\s*)\/?>/g

We should apply this regex to VALIDATE_FOR_HTML_TAG_REGEX, and based on the comments on the constant, we should update BE as well.

What specific scenarios should we cover in automated tests to prevent reintroducing this issue in the future?

Create unit test to test for possible HTML tags with VALIDATE_FOR_HTML_TAG_REGEX

What alternative solutions did you explore? (Optional)

Or we can pass allowHTML to FormProvider to ignore the check.

[samranahm]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Unable to save search query name that contains before and after dates

What is the root cause of that problem?

App/src/components/Form/FormProvider.tsx

Lines 142 to 143 in 3bfb30a

	const matchedHtmlTags = inputValue.match(CONST.VALIDATE_FOR_HTML_TAG_REGEX);
	let isMatch = CONST.WHITELISTED_TAGS.some((regex) => regex.test(inputValue));

onValidate checks input values for HTML tags using CONST.VALIDATE_FOR_HTML_TAG_REGEX.
If any part of the input string matches an HTML tag that is NOT whitelisted, it assigns an error.

currently

App/src/CONST.ts

Line 1704 in 3bfb30a

WHITELISTED_TAGS: [/<>/, /< >/, /<->/, /<-->/, /<br>/, /<br\/>/],

we have not added tag regarding date in WHITELISTED_TAGS
that does not pass the validation for
type:expense status:all date<2025-03-02 date>2025-03-01

What changes do you think we should make in order to solve the problem?

We should add /[<>]\d{4}-\d{2}-\d{2}/ in WHITELISTED_TAGS
update current regex

App/src/CONST.ts

Line 1704 in 3bfb30a

WHITELISTED_TAGS: [/<>/, /< >/, /<->/, /<-->/, /<br>/, /<br\/>/],

to

    WHITELISTED_TAGS: [/<>/, /< >/, /<->/, /<-->/, /<br>/, /<br\/>/, /[<>]\d{4}-\d{2}-\d{2}/],

What specific scenarios should we cover in automated tests to prevent reintroducing this issue in the future?

N/A

What alternative solutions did you explore? (Optional)

Reminder: Please use plain English, be brief and avoid jargon. Feel free to use images, charts or pseudo-code if necessary. Do not post large multi-line diffs or write walls of text. Do not create PRs unless you have been hired for this job.

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~021905581416164321466

[melvin-bot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @eVoloshchak (External)

[melvin-bot]

@eVoloshchak Whoops! This issue is 2 days overdue. Let's get this updated quick!

[eVoloshchak]

@dominictb's proposal:
Changing VALIDATE_FOR_HTML_TAG_REGEX would work, but I have a couple of arguments against it

• The server has a WAF (Web Application Firewall), which will strip out HTML/XML tags using this regex pattern. It's copied here so that the same regex pattern can be used in form validations to be consistent with the server.
  This means we use the same regex on the BE. If we change it here, we have to change it on the BE. I don't have access to the BE, but I suspect this regex is used in multiple places throughout the app (we use it only since on FE), which poses a risk of introducing new regressions

• /<\/?\w*((\s+\w+(\s*=\s*(?:"(.|\n)*?"|'(.|\n)*?'|[^'">\s]+))?)+\s*|\s*)\/?>/g - this is not very readable. If we have this regex both on FE and BE, we need comments overexplaining what it does

Saying all that, I still think this is the correct approach. VALIDATE_FOR_HTML_TAG_REGEX is simply incorrect, and as @dominictb's pointed out, it produces some false positives. We could work around this (@samranahm's proposal), but ultimately, to fix the root cause, we have to change VALIDATE_FOR_HTML_TAG_REGEX on both FE and BE.

🎀👀🎀 C+ reviewed!

[melvin-bot]

Triggered auto assignment to @marcaaron, see https://stackoverflow.com/c/expensify/questions/7972 for more details.

[samranahm]

@marcaaron gentle bump #59283 (comment)

[melvin-bot]

📣 It's been a week! Do we have any satisfactory proposals yet? Do we need to adjust the bounty for this issue? 💸

[melvin-bot]

@marcaaron Huh... This is 4 days overdue. Who can take care of this?

[melvin-bot]

📣 @dominictb 🎉 An offer has been automatically sent to your Upwork account for the Contributor role 🎉 Thanks for contributing to the Expensify app!

Offer link
Upwork job
Please accept the offer and leave a comment on the Github issue letting us know when we can expect a PR to be ready for review 🧑‍💻
Keep in mind: Code of Conduct | Contributing 📖

[marcaaron]

I think it's fine to update this on the FE for now.

@eVoloshchak did you test this out and confirm that the WAF will strip this value?

[samranahm]

@marcaaron Sorry the confusion but for the FE part I proposed adding another tag (tested with WAF using multiple queries) and @eVoloshchak is mentioned that here

We could work around this (@samranahm's proposal), but ultimately, to fix the root cause, we have to change VALIDATE_FOR_HTML_TAG_REGEX on both FE and BE.

🎀👀🎀 C+ reviewed!

@eVoloshchak Would you mind confirming here please

[marcaaron]

No confusion here. @dominictb you may begin.

[melvin-bot]

Reviewing label has been removed, please complete the "BugZero Checklist".

[melvin-bot]

The solution for this issue has been 🚀 deployed to production 🚀 in version 9.1.30-4 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

• fix: unable to save search query with before and after dates #59846

If no regressions arise, payment will be issued on 2025-04-28. 🎊

For reference, here are some details about the assignees on this issue:

• @eVoloshchak requires payment through NewDot Manual Requests
• @dominictb requires payment automatic offer (Contributor)

[melvin-bot]

@eVoloshchak / @dominictb @twisterdotcom @eVoloshchak / @dominictb The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed. Please copy/paste the BugZero Checklist from here into a new comment on this GH and complete it. If you have the K2 extension, you can simply click: [this button]

[bernhardoj]

After updating the regex, we can now enter <.> (for example) as the WS name, but BE will "reject" it (issue). I think we need to update the regex on the BE too.

[twisterdotcom]

@marcaaron is :this: something quick, or should I lobby in #engineering for somebody to do it?

[marcaaron]

Personally, I don't think that's something we need to urgently address. Mostly, I can't think of why we want to support setting <.> as a workspace name?

[twisterdotcom]

Fair point. Is that the only failure the backend will block now @bernhardoj? If so, I agree with @marcaaron it's not really a problem.

[bernhardoj]

It will fail for <{any length special character}> format. It happens when updating profile display name too.

[twisterdotcom]

I think this is a perfectly fine error for us to throw in this instance:

I am happy for them to be considered invalid in display name, workspace name etc. We really only require them in search because they serve a function in the UI.

[bernhardoj]

The issue is when using special characters between <>.

web.mp4

[twisterdotcom]

Okay, I agree it's less obvious but I am happy for us to leave this bug in prod and fix it when the first cyborg/gen z kid named <?> with their company named <.> signs up.

[twisterdotcom]

Payment Summary:

• @eVoloshchak paid $250 via NewDot Manual Request
• @dominictb paid $250 here (Contributor)