Skip to content

Commit 0d71702

Browse files
vincent-zurczakvincent-zurczak
authored
Merge pull request #37 from bats-core/#34
#34 Support regex validation of property values
2 parents e56f901 + 7927bb1 commit 0d71702

18 files changed

+1484
-97
lines changed

‎README.md

+69-6
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ This kind of test is the ultimate set of verifications to run for a project, lon
2727
- [Syntax Reference](#syntax-reference)
2828
- [Counting Resources](#counting-resources)
2929
- [Verifying Property Values](#verifying-property-values)
30+
- [Using Regular Expressions](#using-regular-expressions)
3031
- [Property Names](#property-names)
3132
- [Errors](#errors)
3233
- [Error Codes](#error-codes)
@@ -167,6 +168,11 @@ try "at most 5 times every 30s " \
167168
try at most 5 times every 30s \
168169
to get svc named "'nginx'" \
169170
and verify that "'.spec.ports[*].targetPort'" is "'8484'"
171+
172+
# Regular expressions can also be used
173+
try at most 5 times every 30s \
174+
to get svc named 'nginx' \
175+
and verify that '.spec.ports[*].targetPort' matches '[[:digit:]]+'
170176
```
171177

172178
If you work with OpenShift and would prefer to use **oc** instead of **kubectl**...
@@ -304,9 +310,7 @@ try "at most <number> times every <number>s \
304310
For services, you may directly use the simple count assertions.
305311

306312
This is a checking loop.
307-
It breaks the loop if as soon as the assertion is verified. If it reaches the end of the loop
308-
without having been verified, an error is thrown. Please, refer to [this section](#property-names) for details
309-
about the property names.
313+
It breaks the loop if as soon as the assertion is verified. If it reaches the end of the loop without having been verified, an error is thrown. Please, refer to [this section](#property-names) for details about the property names.
310314

311315

312316
### Verifying Property Values
@@ -334,6 +338,65 @@ about the property names.
334338
But unlike the assertion type to [count resources](#counting-resources), you do not verify _how many instances_ have this value. Notice however that **if it finds 0 item verifying the property, the assertion fails**.
335339

336340

341+
### Using Regular Expressions
342+
343+
It is also possible to verify property values against a regular expression.
344+
This can be used, as an example, to verify values in a JSON array.
345+
346+
```bash
347+
# Verifying a property
348+
verify "'<property-name>' matches '<regular-experession>' for <resource-type> named '<regular-expression>'"
349+
350+
# Finding elements with a matching property
351+
try "at most <number> times every <number>s \
352+
to get <resource-type> named '<regular-expression>' \
353+
and verify that '<property-name>' matches '<regular-experession>'"
354+
355+
# Counting elements with a matching property
356+
try "at most <number> times every <number>s \
357+
to find <number> <resource-type> named '<regular-expression>' \
358+
with '<property-name>' matching '<regular-expression>'"
359+
```
360+
361+
The regular expression used for property values relies on
362+
[BASH regexp](https://en.wikibooks.org/wiki/Regular_Expressions/POSIX-Extended_Regular_Expressions).
363+
More exactly, it uses extended regular expressions (EREs). You can simulate the result of such an assertion
364+
with `grep`, as it is the command used internally. Hence, you can use `echo your-value | grep -E your-regex`
365+
to prepare your assertions.
366+
367+
> Unlike the assertions with the verb « to be », those with the verb « to match » are case-sensitive.
368+
369+
All the assertions using the verb « to be » make case-insensitive comparison.
370+
It means writing `is 'running'` or `is 'Running'` does not change anything.
371+
If you want case-sensitive equality, then use a regular expression, i.e. write
372+
`matches '^Running$'`.
373+
374+
If for some reasons, one needs case-insensitive matches, you can set the DETIK_CASE_INSENSITIVE_PROPERTIES
375+
property to `true` in your test. All the retrieved values by the DETIK_CLIENT will be lower-cased. It means
376+
you can write a pattern that only considers lower-case characters. The following sample illustrates this situation:
377+
378+
```bash
379+
# Assuming the status of the POD is "Running"...
380+
# ... then the following assertion will fail.
381+
verify "'status' matches 'running' for pods named 'nginx'"
382+
383+
# Same for...
384+
verify "'status' matches '^running$' for pods named 'nginx'"
385+
386+
# This is because the value returned by the client starts with an upper-case letter.
387+
# For case-insensivity operations with a regular expression, just use...
388+
DETIK_CASE_INSENSITIVE_PROPERTIES="true"
389+
verify "'status' matches 'running' for pods named 'nginx'"
390+
391+
# The assertion will now be verified.
392+
# Just make sure the pattern ONLY INCLUDES lower-case characters.
393+
394+
# If you set DETIK_CASE_INSENSITIVE_PROPERTIES directly in a "@test" function,
395+
# there is no need to reset it for the other tests. Its scope is limited to the
396+
# function that defines it. It is recommended to NOT make this variable a global one.
397+
```
398+
399+
337400
### Property Names
338401

339402
In all assertions, *property-name* is one of the column names supported by K8s.
@@ -428,9 +491,9 @@ DEBUG_DETIK=""
428491

429492
### Linting
430493

431-
Because Bash is not a compiled language, it is easy to make mistakes.
432-
Even if the library was designed to be simple. This is why a linter was created, to help to
433-
locate syntax errors when writing DETIK assertions. You can use it with BATS in your tests.
494+
Despite the efforts to make the DETIK syntax as simple as possible, BASH remains a non-compiled
495+
language and mistakes happen. To prevent them, a linter was created to help locating
496+
syntax errors when writing DETIK assertions. You can use it with BATS in your tests.
434497

435498
```bash
436499
#!/usr/bin/env bats

‎lib/detik.bash

+84-20
Original file line numberDiff line numberDiff line change
@@ -38,18 +38,30 @@ try() {
3838
property=""
3939
expected_value=""
4040
expected_count=""
41+
verify_strict_equality="true"
4142

42-
if [[ "$exp" =~ $try_regex_verify ]]; then
43+
if [[ "$exp" =~ $try_regex_verify_is ]]; then
4344

4445
# Extract parameters
4546
times="${BASH_REMATCH[1]}"
4647
delay="${BASH_REMATCH[2]}"
4748
resource=$(to_lower_case "${BASH_REMATCH[3]}")
4849
name="${BASH_REMATCH[4]}"
4950
property="${BASH_REMATCH[5]}"
50-
expected_value=$(to_lower_case "${BASH_REMATCH[6]}")
51+
expected_value="${BASH_REMATCH[6]}"
5152

52-
elif [[ "$exp" =~ $try_regex_find ]]; then
53+
elif [[ "$exp" =~ $try_regex_verify_matches ]]; then
54+
55+
# Extract parameters
56+
times="${BASH_REMATCH[1]}"
57+
delay="${BASH_REMATCH[2]}"
58+
resource=$(to_lower_case "${BASH_REMATCH[3]}")
59+
name="${BASH_REMATCH[4]}"
60+
property="${BASH_REMATCH[5]}"
61+
expected_value="${BASH_REMATCH[6]}"
62+
verify_strict_equality="false"
63+
64+
elif [[ "$exp" =~ $try_regex_find_being ]]; then
5365

5466
# Extract parameters
5567
times="${BASH_REMATCH[1]}"
@@ -58,7 +70,19 @@ try() {
5870
resource=$(to_lower_case "${BASH_REMATCH[4]}")
5971
name="${BASH_REMATCH[5]}"
6072
property="${BASH_REMATCH[6]}"
61-
expected_value=$(to_lower_case "${BASH_REMATCH[7]}")
73+
expected_value="${BASH_REMATCH[7]}"
74+
75+
elif [[ "$exp" =~ $try_regex_find_matching ]]; then
76+
77+
# Extract parameters
78+
times="${BASH_REMATCH[1]}"
79+
delay="${BASH_REMATCH[2]}"
80+
expected_count="${BASH_REMATCH[3]}"
81+
resource=$(to_lower_case "${BASH_REMATCH[4]}")
82+
name="${BASH_REMATCH[5]}"
83+
property="${BASH_REMATCH[6]}"
84+
expected_value="${BASH_REMATCH[7]}"
85+
verify_strict_equality="false"
6286
fi
6387

6488
# Do we have something?
@@ -73,7 +97,7 @@ try() {
7397
for ((i=1; i<=times; i++)); do
7498

7599
# Verify the value
76-
verify_value "$property" "$expected_value" "$resource" "$name" "$expected_count" && code=$? || code=$?
100+
verify_value "$verify_strict_equality" "$property" "$expected_value" "$resource" "$name" "$expected_count" && code=$? || code=$?
77101

78102
# Break the loop prematurely?
79103
if [[ "$code" == "0" ]]; then
@@ -158,7 +182,20 @@ verify() {
158182
name="${BASH_REMATCH[4]}"
159183

160184
echo "Valid expression. Verification in progress..."
161-
verify_value "$property" "$expected_value" "$resource" "$name"
185+
verify_value true "$property" "$expected_value" "$resource" "$name"
186+
187+
if [[ "$?" != "0" ]]; then
188+
return 3
189+
fi
190+
191+
elif [[ "$exp" =~ $verify_regex_property_matches ]]; then
192+
property="${BASH_REMATCH[1]}"
193+
expected_value="${BASH_REMATCH[2]}"
194+
resource=$(to_lower_case "${BASH_REMATCH[3]}")
195+
name="${BASH_REMATCH[4]}"
196+
197+
echo "Valid expression. Verification in progress..."
198+
verify_value false "$property" "$expected_value" "$resource" "$name"
162199

163200
if [[ "$?" != "0" ]]; then
164201
return 3
@@ -172,6 +209,7 @@ verify() {
172209

173210

174211
# Verifies the value of a column for a set of elements.
212+
# @param {boolean} true to verify equality, false to match a regex
175213
# @param {string} A K8s column or one of the supported aliases.
176214
# @param {string} The expected value.
177215
# @param {string} The resouce type (e.g. pod).
@@ -183,11 +221,12 @@ verify() {
183221
verify_value() {
184222

185223
# Make the parameters readable
186-
property="$1"
187-
expected_value=$(to_lower_case "$2")
188-
resource="$3"
189-
name="$4"
190-
expected_count="$5"
224+
verify_strict_equality=$(to_lower_case "$1")
225+
property="$2"
226+
expected_value="$3"
227+
resource="$4"
228+
name="$5"
229+
expected_count="$6"
191230

192231
# List the items and remove the first line (the one that contains the column names)
193232
query=$(build_k8s_request "$property")
@@ -224,22 +263,47 @@ verify_value() {
224263
for line in $result; do
225264

226265
# Keep the second column (property to verify)
227-
# and put it in lower case
228-
value=$(to_lower_case "$line" | awk '{ print $2 }')
266+
value=$(echo "$line" | awk '{ print $2 }')
229267
element=$(echo "$line" | awk '{ print $1 }')
230-
if [[ "$value" != "$expected_value" ]]; then
231-
echo "Current value for $element is $value..."
232-
invalid=$((invalid + 1))
268+
269+
# Compare with an exact value (case insensitive)
270+
if [[ "$verify_strict_equality" == "true" ]]; then
271+
value=$(to_lower_case "$value")
272+
expected_value=$(to_lower_case "$expected_value")
273+
if [[ "$value" != "$expected_value" ]]; then
274+
echo "Current value for $element is $value..."
275+
invalid=$((invalid + 1))
276+
else
277+
echo "$element has the right value ($value)."
278+
valid=$((valid + 1))
279+
fi
280+
281+
# Verify a regex (we preserve the case)
233282
else
234-
echo "$element has the right value ($value)."
235-
valid=$((valid + 1))
283+
# We do not want another syntax for case-insensitivity
284+
if [ "$DETIK_REGEX_CASE_INSENSITIVE_PROPERTIES" = "true" ]; then
285+
value=$(to_lower_case "$value")
286+
fi
287+
288+
reg=$(echo "$value" | grep -E "$expected_value")
289+
if [[ "$?" -ne 0 ]]; then
290+
echo "Current value for $element is $value..."
291+
invalid=$((invalid + 1))
292+
else
293+
echo "$element matches the regular expression (found $reg)."
294+
valid=$((valid + 1))
295+
fi
236296
fi
237297
done
238298

239299
# Do we have the right number of elements?
240300
if [[ "$expected_count" != "" ]]; then
241301
if [[ "$valid" != "$expected_count" ]]; then
242-
echo "Expected $expected_count $resource named $name to have this value ($expected_value). Found $valid."
302+
if [[ "$verify_strict_equality" == "true" ]]; then
303+
echo "Expected $expected_count $resource named $name to have this value ($expected_value). Found $valid."
304+
else
305+
echo "Expected $expected_count $resource named $name to match this pattern ($expected_value). Found $valid."
306+
fi
243307
invalid=101
244308
else
245309
invalid=0
@@ -279,7 +343,7 @@ build_k8s_client_with_options() {
279343

280344
client_with_options="$DETIK_CLIENT_NAME"
281345
if [[ -n "$DETIK_CLIENT_NAMESPACE" ]]; then
282-
# eval does not like '-n'
346+
# eval does not "like" the '-n' syntax
283347
client_with_options="$DETIK_CLIENT_NAME --namespace=$DETIK_CLIENT_NAMESPACE"
284348
fi
285349

‎lib/linter.bash

+23-8
Original file line numberDiff line numberDiff line change
@@ -132,15 +132,24 @@ check_line() {
132132
part=$(clean_regex_part "${BASH_REMATCH[2]}")
133133
context="$context\nRegex part: $part"
134134

135-
verify_against_pattern "$part" "$try_regex_verify"
136-
p_verify="$?"
135+
verify_against_pattern "$part" "$try_regex_verify_is"
136+
p_verify_is="$?"
137137

138-
verify_against_pattern "$part" "$try_regex_find"
139-
p_find="$?"
138+
verify_against_pattern "$part" "$try_regex_verify_matches"
139+
p_verify_matches="$?"
140+
141+
verify_against_pattern "$part" "$try_regex_find_being"
142+
p_find_being="$?"
143+
144+
verify_against_pattern "$part" "$try_regex_find_matching"
145+
p_find_matching="$?"
140146

141147
# detik_debug "p_verify=$p_verify, p_find=$p_find, part=$part"
142-
if [[ "$p_verify" != "0" ]] && [[ "$p_find" != "0" ]]; then
143-
handle_error "Invalid TRY statement at line $line_number." "$context"
148+
if [[ "$p_verify_is" != "0" ]] && \
149+
[[ "$p_verify_matches" != "0" ]] && \
150+
[[ "$p_find_being" != "0" ]] && \
151+
[[ "$p_find_matching" != "0" ]]; then
152+
handle_error "Invalid TRY statement at line $line_number." "$context"
144153
fi
145154

146155
# We have "verify" or "run verify" followed by something
@@ -159,9 +168,15 @@ check_line() {
159168
verify_against_pattern "$part" "$verify_regex_property_is"
160169
p_prop="$?"
161170

171+
verify_against_pattern "$part" "$verify_regex_property_matches"
172+
p_matches="$?"
173+
162174
# detik_debug "p_is=$p_is, p_are=$p_are, p_prop=$p_prop, part=$part"
163-
if [[ "$p_is" != "0" ]] && [[ "$p_are" != "0" ]] && [[ "$p_prop" != "0" ]] ; then
164-
handle_error "Invalid VERIFY statement at line $line_number." "$context"
175+
if [[ "$p_is" != "0" ]] && \
176+
[[ "$p_are" != "0" ]] && \
177+
[[ "$p_prop" != "0" ]] && \
178+
[[ "$p_matches" != "0" ]] ; then
179+
handle_error "Invalid VERIFY statement at line $line_number." "$context"
165180
fi
166181
fi
167182
}

‎lib/utils.bash

+14-3
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,16 @@
22

33

44
# The regex for the "try" key word
5-
try_regex_verify="^at +most +([0-9]+) +times +every +([0-9]+)s +to +get +([a-z]+) +named +'([^']+)' +and +verify +that +'([^']+)' +is +'([^']+)'$"
6-
try_regex_find="^at +most +([0-9]+) +times +every +([0-9]+)s +to +find +([0-9]+) +([a-z]+) +named +'([^']+)' +with +'([^']+)' +being +'([^']+)'$"
5+
try_regex_verify_is="^at +most +([0-9]+) +times +every +([0-9]+)s +to +get +([a-z]+) +named +'([^']+)' +and +verify +that +'([^']+)' +is +'([^']+)'$"
6+
try_regex_verify_matches="^at +most +([0-9]+) +times +every +([0-9]+)s +to +get +([a-z]+) +named +'([^']+)' +and +verify +that +'([^']+)' +matches +'([^']+)'$"
7+
try_regex_find_being="^at +most +([0-9]+) +times +every +([0-9]+)s +to +find +([0-9]+) +([a-z]+) +named +'([^']+)' +with +'([^']+)' +being +'([^']+)'$"
8+
try_regex_find_matching="^at +most +([0-9]+) +times +every +([0-9]+)s +to +find +([0-9]+) +([a-z]+) +named +'([^']+)' +with +'([^']+)' +matching +'([^']+)'$"
79

810
# The regex for the "verify" key word
911
verify_regex_count_is="^there +is +(0|1) +([a-z]+) +named +'([^']+)'$"
1012
verify_regex_count_are="^there +are +([0-9]+) +([a-z]+) +named +'([^']+)'$"
1113
verify_regex_property_is="^'([^']+)' +is +'([^']+)' +for +([a-z]+) +named +'([^']+)'$"
12-
14+
verify_regex_property_matches="^'([^']+)' +matches +'([^']+)' +for +([a-z]+) +named +'([^']+)'$"
1315

1416

1517
# Prints a string in lower case.
@@ -73,3 +75,12 @@ reset_detik_debug() {
7375
reset_debug
7476
fi
7577
}
78+
79+
80+
# Dumps the argument and return the previous error code.
81+
# @return the previous error code
82+
ddump() {
83+
res="$?"
84+
echo "$1"
85+
return $res
86+
}

0 commit comments

Comments
 (0)