issue in _WD_ExecuteScript / __WD_EscapeString when comments // was used in JavaScript string #487

mlipok · 2023-07-30T13:13:55Z

Bug report

Describe the bug

$sData = StringRegExpReplace($sData, '[\v\t]', '')
from here:

Lines 1747 to 1749 in 42dda7d

    
           Func __WD_EscapeString($sData, $iOption = 0) 
        
           	If BitAND($iOption, $JSON_MLREFORMAT) Then 
        
           		$sData = StringRegExpReplace($sData, '[\v\t]', '') ; Strip tabs and CR/LFs

strips [\v] chars and when // comment is used entire $sJavaScript becames commented out.

How to reproduce

_Example()
Func _Example()
	Local $sJavaScript = _
			"// comment" & @CRLF & _
			"return document.readyState;" & @CRLF & _
			""
	_WD_DebugSwitch($_WD_DEBUG_Full)
	_WD_ExecuteScript($sSession, $sJavaScript)
	_WD_DebugSwitch()
EndFunc   ;==>_Example

Expected behavior

get result from: return document.readyState;

Screenshots

none

Additional context

https://www.autoitscript.com/forum/topic/208640-webdriver-udf-help-support-iv/?do=findComment&comment=1521863

System under test

not related

The text was updated successfully, but these errors were encountered:

mlipok · 2023-07-30T13:20:15Z

interesting.
Using:

_Example()
Func _Example()
	Local $sJavaScript = _
			"// comment" & @CRLF & _
			"return document.readyState;" & @CRLF & _
			""
	_WD_DebugSwitch($_WD_DEBUG_Full)
	_WD_ExecuteScript($sSession, $sJavaScript)
	_WD_DebugSwitch()
EndFunc   ;==>_Example

I get proper results:

__WD_Post: URL=HTTP://127.0.0.1:4444/session/e1192324-9090-4743-8d1c-c390f190cdda/execute/sync; Data={"script":"\/\/ comment\r\nreturn document.readyState;\r\n", "args":[]}
__WD_Post ==> Success [0] : HTTP status = 200 ResponseText={"value":"complete"}

the same result with:

_Example()
Func _Example()
	Local $sJavaScript = _
			"// comment" & @CR & _
			"return document.readyState;" & @CR & _
			""
	_WD_DebugSwitch($_WD_DEBUG_Full)
	_WD_ExecuteScript($sSession, $sJavaScript)
	_WD_DebugSwitch()
EndFunc   ;==>_Example

and with:

_Example()
Func _Example()
	Local $sJavaScript = _
			"// comment" & @LF & _
			"return document.readyState;" & @LF & _
			""
	_WD_DebugSwitch($_WD_DEBUG_Full)
	_WD_ExecuteScript($sSession, $sJavaScript)
	_WD_DebugSwitch()
EndFunc   ;==>_Example

Danp2 · 2023-07-30T13:30:45Z

I get proper results:

Not me. I got this --

     __WD_Post: URL=HTTP://127.0.0.1:4444/session/0cab983f-b61b-4acf-afa7-055cb4397c0a/execute/sync; Data={"script":"\/\/ commentreturn document.readyState;", "args":[]}
     __WD_Post ==> Success [0] : HTTP status = 200 ResponseText={"value":null}

Edit: Notice the difference in the Data portion of mine VS yours. Are you sure you are testing with the latest release?

mlipok · 2023-07-30T13:39:07Z

I get proper results:

__WD_Post: URL=HTTP://127.0.0.1:4444/session/e1192324-9090-4743-8d1c-c390f190cdda/execute/sync; Data={"script":"\/\/ comment\r\nreturn document.readyState;\r\n", "args":[]}
__WD_Post ==> Success [0] : HTTP status = 200 ResponseText={"value":"complete"}

oops.. my mistake. I had little modified verison because I was testing this issue.

my results are:

_WD_DebugSwitch:  /  stack size: 2
__WD_Post: URL=HTTP://127.0.0.1:4444/session/758b3de6-2777-4a45-a73b-c4171a079a57/execute/sync; Data={"script":"\/\/ commentreturn document.readyState;", "args":[]}
__WD_Post ==> Success [0] : HTTP status = 200 ResponseText={"value":null}
_WD_ExecuteScript ==> Success [0]

the issue can be solved by:

Func __WD_EscapeString($sData, $iOption = 0)
	If BitAND($iOption, $JSON_MLREFORMAT) Then
;~ 		$sData = StringRegExpReplace($sData, '[\v\t]', '') ; Strip tabs and CR/LFs ; original
		$sData = StringRegExpReplace($sData, '\t', '') ; Strip tabs ; modified

and results are:

_WD_DebugSwitch:  /  stack size: 2
__WD_Post: URL=HTTP://127.0.0.1:4444/session/cbaddcad-3adb-4f2e-91da-8fb50c008226/execute/sync; Data={"script":"\/\/ comment\nreturn document.readyState;\n", "args":[]}
__WD_Post ==> Success [0] : HTTP status = 200 ResponseText={"value":"complete"}
_WD_ExecuteScript ==> Success [0]

mlipok · 2023-07-30T13:40:57Z

we should re-test:
#465

mlipok · 2023-07-30T13:49:04Z

from: #465
REPRO:
Using file: UserTesting_EscapeString.au3 with following content:

ConsoleWrite("! Start: UserTesting_EscapeString.au3" & @CRLF)

ConsoleWrite("- Test 1:" & @CRLF)
_WD_Navigate($sSession, 'https://www.google.com')
_WD_LoadWait($sSession)
MsgBox($MB_OK + $MB_TOPMOST + $MB_ICONWARNING, "Warning #" & @ScriptLineNumber, "If you see COOKIE accept panel close them before you will continue.")

ConsoleWrite("- Test 2:" & @CRLF)
; REMARK:  __SetVAR($IDX_VAR, $value) will set  $_VAR[$IDX_VAR] = $value
__SetVAR(0, _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//textarea[@name='q']"))

ConsoleWrite("- Test 3:" & @CRLF)
__SetVAR(1, ' " ! @ # $ % ^ & * ( ) - _ + = { } [ ] ; : \ | , . < > / ? ~ ' & " ' " & ' @CR' & @CR & ' @CRLF' & @CRLF & ' @LF' & @LF & ' @TAB' & @TAB & 'END')
_WD_ElementAction($sSession, $_VAR[0], "VALUE", $_VAR[1])
;~ _WD_SetElementValue($sSession, $_VAR[0], $_VAR[1], $_WD_OPTION_Advanced)

ConsoleWrite("! $_VAR[1] -" & @CRLF & $_VAR[1] & @CRLF)

ConsoleWrite("! __WD_EscapeString($_VAR[0]) -" & @CRLF & __WD_EscapeString($_VAR[1]) & @CRLF)
ConsoleWrite("! END: UserTesting_EscapeString.au3" & @CRLF)

together with current modification:

Func __WD_EscapeString($sData, $iOption = 0)
	If BitAND($iOption, $JSON_MLREFORMAT) Then
;~ 		$sData = StringRegExpReplace($sData, '[\v\t]', '') ; Strip tabs and CR/LFs ; original
		$sData = StringRegExpReplace($sData, '\t', '') ; Strip tabs ; modified

I get:

_WD_FindElement ==> Success [0] : Parameters:   Strategy=xpath   Selector=//textarea[@name='q']   StartNodeID=Default   Multiple=Default   ShadowRoot=Default
- Setting $_VAR[0] = 9559b64a-277b-4171-b3fb-e1ace0755842
- Test 3:
- Setting $_VAR[1] =  " ! @ # $ % ^ & * ( ) - _ + = { } [ ] ; : \ | , . < > / ? ~  '  @CR
 @CRLF
 @LF
 @TAB	END
__WD_Post ==> Success [0] : HTTP status = 200
_WD_ElementAction ==> Success [0] : Parameters:   Command=VALUE   Option=<masked>
! $_VAR[1] -
 " ! @ # $ % ^ & * ( ) - _ + = { } [ ] ; : \ | , . < > / ? ~  '  @CR
 @CRLF
 @LF
 @TAB	END
! __WD_EscapeString($_VAR[0]) -
 \" ! @ # $ % ^ & * ( ) - _ + = { } [ ] ; : \\ | , . < > \/ ? ~  '  @CR\r @CRLF\r\n @LF\n @TAB\tEND
! END: UserTesting_EscapeString.au3

for me the following fix

Func __WD_EscapeString($sData, $iOption = 0)
	If BitAND($iOption, $JSON_MLREFORMAT) Then
;~ 		$sData = StringRegExpReplace($sData, '[\v\t]', '') ; Strip tabs and CR/LFs ; original
		$sData = StringRegExpReplace($sData, '\t', '') ; Strip tabs ; modified

works fine.

mlipok · 2023-07-30T13:53:27Z

DemoUpload from wd_demo.au3 also works fine.

mlipok · 2023-07-30T13:57:33Z

@Danp2 If you agree with my testing route and results, I pass the solution of this problem to you (please make PR).

mlipok · 2023-07-30T14:00:05Z

btw.
__WD_EscapeString tested also with:
#486 (comment)

results - works fine.

Danp2 · 2023-07-30T14:19:28Z

we should re-test: #465

Yes, I think it is important to understand why the original change was made before reverting so that we can hopefully avoid creating a different issue.

mlipok · 2023-07-30T14:20:38Z

I think this was my mistake.

mlipok · 2023-07-30T14:22:21Z

quick review and I think it starts here:
#465 (comment)

mlipok · 2023-07-30T14:27:57Z

finall fix proposal:

; #INTERNAL_USE_ONLY# ===========================================================================================================
; Name ..........: __WD_EscapeString
; Description ...: Escapes designated characters in string.
; Syntax ........: __WD_EscapeString($sData[, $iOption = 0])
; Parameters ....: $sData   - the string to be escaped
;                  $iOption - [optional] Any combination of $JSON_* constants. Default is 0.
; Author ........: Danp2, mLipok
; Modified ......:
; Remarks .......: $JSON_MLREFORMAT will strip tabs from a multiline string (ie. indentation) as they are not supported by WebDriver, use \t where it is needed
;                  See $JSON_* constants in json.au3 for other $iOption possibilities.
; Related .......:
; Link ..........:
; Example .......: No
; ===============================================================================================================================
Func __WD_EscapeString($sData, $iOption = 0)
	If BitAND($iOption, $JSON_MLREFORMAT) Then
		$sData = StringRegExpReplace($sData, '\t', '') ; Strip tabs
		$iOption = BitXOR($iOption, $JSON_MLREFORMAT) ; Flip bit off
	EndIf

	$sData = Json_StringEncode($sData, $iOption) ; Escape JSON Strings
	Return SetError($_WD_ERROR_Success, 0, $sData)
EndFunc   ;==>__WD_EscapeString

Danp2 · 2023-07-30T23:20:07Z

An alternative is to drop the use of $JSON_MLREFORMAT in _WD_ExecuteScript, but still awaiting your clarification here.

* _WD_ElementSelectAction 'selectedOptions' group (#375) * _WD_ElementSelectAction 'options' added group name (#374) * _WD_ElementSelectAction 'selectAll' JS validation (#373) * _WD_ElementSelectAction 'selectedLabels' JS refact (#376) * _WD_ElementSelectAction + SINGLESELECT (#367) * _ElementSelectAction : JS refactoring + case insensivity (#381) * Return result from `selectAll` * selectAll + $_WD_ERROR_NoMatch

mlipok · 2023-07-31T05:31:25Z

take a look here:
5bb7a7a#r123141584

from this time my experience says to mee each time I use TABS in $sJavaScript they need to be stripped.

au3WebDriver/wd_helper.au3

Line 2276 in 5bb7a7a

    
           ; StringReplace() removes all possible @TAB's because they are used only for indentation and are not needed in JSON string

mlipok · 2023-07-31T09:21:34Z

I will do test today to clarify if TABS are allowed or not.

btw.
https://www.autoitscript.com/forum/topic/208640-webdriver-udf-help-support-iv/?do=findComment&comment=1521937

Danp2 · 2023-07-31T12:50:47Z

If your results match mine, then I would suggest that we drop the usage of $JSON_MLREFORMAT

mlipok · 2023-07-31T13:53:35Z

So please do this test:

run DemoSelectOptions from wd_demo.au3 for Chrome browser
check results
open wd_helper.au3
change all , @TAB, '') to , '@TAB', '')
skip If BitAND($iOption, $JSON_MLREFORMAT) Then
run DemoSelectOptions from wd_demo.au3 for Chrome browser
check results

this should explain why I said that TABS used directly in $sJavaScript are not supported by WebDriver

my results are:

! Error = 8 occurred on: DemoSelectOptions
! _WD_LastHTTPResult = 404
! _WD_LastHTTPResponse = {"value":{"error":"no such element","message":"no such element: Unable to locate element: {\"method\":\"xpath\",\"selector\":\"//*[@name=\"pets\"]\"}\n  (Session info: chrome=115.0.5790.110)","stacktrace":"Backtrace:\n\tGetHandleVerifier [0x0058A813+48355]\n\t(No symbol) [0x0051C4B1]\n\t(No symbol) [0x00425358]\n\t(No symbol) [0x004509A5]\n\t(No symbol) [0x00450B3B]\n\t(No symbol) [0x0047E232]\n\t(No symbol) [0x0046A784]\n\t(No symbol) [0x0047C922]\n\t(No symbol) [0x0046A536]\n\t(No symbol) [0x004482DC]\n\t(No symbol) [0x004493DD]\n\tGetHandleVerifier [0x007EAABD+2539405]\n\tGetHandleVerifier [0x0082A78F+2800735]\n\tGetHandleVerifier [0x0082456C+2775612]\n\tGetHandleVerifier [0x006151E0+616112]\n\t(No symbol) [0x00525F8C]\n\t(No symbol) [0x00522328]\n\t(No symbol) [0x0052240B]\n\t(No symbol) [0x00514FF7]\n\tBaseThreadInitThunk [0x767D7D59+25]\n\tRtlInitializeExceptionChain [0x77A5B79B+107]\n\tRtlClearBits [0x77A5B71F+191]\n"}}

mlipok · 2023-07-31T14:10:27Z

last post edited (sorry for the confusion)

mlipok · 2023-07-31T14:12:40Z

my results are:

! Error = 8 occurred on: DemoSelectOptions
! _WD_LastHTTPResult = 404
! _WD_LastHTTPResponse = {"value":{"error":"no such element","message":"no such element: Unable to locate element: {\"method\":\"xpath\",\"selector\":\"//*[@name=\"pets\"]\"}\n  (Session info: chrome=115.0.5790.110)","stacktrace":"Backtrace:\n\tGetHandleVerifier [0x0058A813+48355]\n\t(No symbol) [0x0051C4B1]\n\t(No symbol) [0x00425358]\n\t(No symbol) [0x004509A5]\n\t(No symbol) [0x00450B3B]\n\t(No symbol) [0x0047E232]\n\t(No symbol) [0x0046A784]\n\t(No symbol) [0x0047C922]\n\t(No symbol) [0x0046A536]\n\t(No symbol) [0x004482DC]\n\t(No symbol) [0x004493DD]\n\tGetHandleVerifier [0x007EAABD+2539405]\n\tGetHandleVerifier [0x0082A78F+2800735]\n\tGetHandleVerifier [0x0082456C+2775612]\n\tGetHandleVerifier [0x006151E0+616112]\n\t(No symbol) [0x00525F8C]\n\t(No symbol) [0x00522328]\n\t(No symbol) [0x0052240B]\n\t(No symbol) [0x00514FF7]\n\tBaseThreadInitThunk [0x767D7D59+25]\n\tRtlInitializeExceptionChain [0x77A5B79B+107]\n\tRtlClearBits [0x77A5B71F+191]\n"}}

oops.
Something went really wrong.
Wait a moment I need some more time to investigate this new issue.

Danp2 · 2023-07-31T14:31:45Z

I think the simplest way to perform this testing is to change the following line in _WD_ExecuteScript --

$sScript = __WD_EscapeString($sScript, $JSON_MLREFORMAT)

to

$sScript = __WD_EscapeString($sScript)

Then test all JS functionality to see if there are any negative effects.

mlipok · 2023-08-01T00:20:10Z

I think the simplest way to perform this testing is to change the following line in _WD_ExecuteScript --

$sScript = __WD_EscapeString($sScript, $JSON_MLREFORMAT)

to

$sScript = __WD_EscapeString($sScript)

Then test all JS functionality to see if there are any negative effects.

After some more testing I think this should be fine.

I think that here:
81154ab

au3WebDriver/wd_core.au3

Lines 1690 to 1699 in 81154ab

    
           Func __WD_EscapeString($sData, $iOption = 0) 
        
           	If BitAND($iOption, $JSON_MLREFORMAT) Then 
        
           		$sData = StringRegExpReplace($sData, '[\v\t]', '') ; Strip tabs and CR/LFs 
        
           		$iOption = BitXOR($iOption, $JSON_MLREFORMAT)      ; Flip bit off 
        
           	Endif 
        
           	$sData = Json_StringEncode($sData, $iOption) ; Escape JSON Strings 
        
           	Return SetError($_WD_ERROR_Success, 0, $sData) 
        
           EndFunc   ;==>__WD_EscapeString

this following part was bad as it was no taking all cases:

Local $sRegEx = "([" & $_WD_ESCAPE_CHARS & "])"

and thus I wanted to add:

$sData = StringRegExpReplace($sData, '[\v\t]', '') ; Strip tabs and CR/LFs

but you propose to use:

$sData = Json_StringEncode($sData, $iOption) ; Escape JSON Strings

your proposal was covering also my request about fixing \t so currently $JSON_MLREFORMAT should be treated as useless and can be romoved from wd_core.au3 especially from:

au3WebDriver/wd_core.au3

Lines 1691 to 1694 in 81154ab

    
           If BitAND($iOption, $JSON_MLREFORMAT) Then 
        
           	$sData = StringRegExpReplace($sData, '[\v\t]', '') ; Strip tabs and CR/LFs 
        
           	$iOption = BitXOR($iOption, $JSON_MLREFORMAT)      ; Flip bit off 
        
           Endif

mlipok · 2023-08-01T09:21:56Z

If your results match mine, then I would suggest that we drop the usage of $JSON_MLREFORMAT

#489

This comment was marked as off-topic.

Sign in to view

This was referenced Jul 31, 2023

Frame issues #488

Open

_WD_ExecuteScript: don't reformat JS code #489

Merged

Danp2 closed this as completed in #489 Aug 1, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

issue in _WD_ExecuteScript / __WD_EscapeString when comments // was used in JavaScript string #487

issue in _WD_ExecuteScript / __WD_EscapeString when comments // was used in JavaScript string #487

mlipok commented Jul 30, 2023 •

edited

Loading

mlipok commented Jul 30, 2023

Danp2 commented Jul 30, 2023 •

edited

Loading

mlipok commented Jul 30, 2023

mlipok commented Jul 30, 2023

mlipok commented Jul 30, 2023

mlipok commented Jul 30, 2023

mlipok commented Jul 30, 2023

mlipok commented Jul 30, 2023 •

edited

Loading

Danp2 commented Jul 30, 2023

mlipok commented Jul 30, 2023

mlipok commented Jul 30, 2023

mlipok commented Jul 30, 2023 •

edited

Loading

Danp2 commented Jul 30, 2023

mlipok commented Jul 31, 2023 •

edited

Loading

mlipok commented Jul 31, 2023

Danp2 commented Jul 31, 2023

mlipok commented Jul 31, 2023 •

edited

Loading

mlipok commented Jul 31, 2023

mlipok commented Jul 31, 2023 •

edited

Loading

Danp2 commented Jul 31, 2023

This comment was marked as off-topic.

This comment was marked as off-topic.

mlipok commented Aug 1, 2023 •

edited

Loading

mlipok commented Aug 1, 2023

issue in _WD_ExecuteScript / __WD_EscapeString when comments // was used in JavaScript string #487

issue in _WD_ExecuteScript / __WD_EscapeString when comments // was used in JavaScript string #487

Comments

mlipok commented Jul 30, 2023 • edited Loading

Bug report

Describe the bug

How to reproduce

Expected behavior

Screenshots

Additional context

System under test

mlipok commented Jul 30, 2023

Danp2 commented Jul 30, 2023 • edited Loading

mlipok commented Jul 30, 2023

mlipok commented Jul 30, 2023

mlipok commented Jul 30, 2023

mlipok commented Jul 30, 2023

mlipok commented Jul 30, 2023

mlipok commented Jul 30, 2023 • edited Loading

Danp2 commented Jul 30, 2023

mlipok commented Jul 30, 2023

mlipok commented Jul 30, 2023

mlipok commented Jul 30, 2023 • edited Loading

Danp2 commented Jul 30, 2023

mlipok commented Jul 31, 2023 • edited Loading

mlipok commented Jul 31, 2023

Danp2 commented Jul 31, 2023

mlipok commented Jul 31, 2023 • edited Loading

mlipok commented Jul 31, 2023

mlipok commented Jul 31, 2023 • edited Loading

Danp2 commented Jul 31, 2023

This comment was marked as off-topic.

This comment was marked as off-topic.

mlipok commented Aug 1, 2023 • edited Loading

mlipok commented Aug 1, 2023

mlipok commented Jul 30, 2023 •

edited

Loading

Danp2 commented Jul 30, 2023 •

edited

Loading

mlipok commented Jul 30, 2023 •

edited

Loading

mlipok commented Jul 30, 2023 •

edited

Loading

mlipok commented Jul 31, 2023 •

edited

Loading

mlipok commented Jul 31, 2023 •

edited

Loading

mlipok commented Jul 31, 2023 •

edited

Loading

mlipok commented Aug 1, 2023 •

edited

Loading