Skip to content

Form RequestFieldMatcher can not be constructed from Simulation #290

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
matthiastz opened this issue Oct 24, 2023 · 10 comments
Closed

Form RequestFieldMatcher can not be constructed from Simulation #290

matthiastz opened this issue Oct 24, 2023 · 10 comments

Comments

@matthiastz
Copy link

matthiastz commented Oct 24, 2023 •
edited
Loading

Description of the bug

With bumping our Dependencies to ("io.specto:hoverfly-java-junit5:0.15.0") ("io.specto:hoverfly-java:0.15.0") one of our test breaks, because the form request is not matched anymore.

Steps to reproduce the issue

Observed result

Hoverfly Java and Hoverfly error messages seen (If none, say none)


io.specto.hoverfly.junit.api.HoverflyClientException: Failed to get simulation: Cannot construct instance of `io.specto.hoverfly.junit.core.model.RequestFieldMatcher$MatcherType`, problem: No enum constant io.specto.hoverfly.junit.core.model.RequestFieldMatcher.MatcherType.FORM
 at [Source: (String)"{"data":{"pairs":[{"request":{"path":[{"matcher":"exact","value":"/oauth2/token"}],"method":[{"matcher":"exact","value":"POST"}],"destination":[{"matcher":"exact","value":"www.my-test.com"}],"body":[{"matcher":"form","value":{"client_id":[{"matcher":"exact","value":"some-id"}],"client_secret":[{"matcher":"exact","value":""}],"code":[{"matcher":"exact","value":"some-code-value"}],"grant_type":[{"matcher":"exact","value":"authorization_code"}],"redirect_uri":[{"m"[truncated 588 chars]; line: 1, column: 211] (through reference chain: io.specto.hoverfly.junit.core.model.Simulation["data"]->io.specto.hoverfly.junit.core.model.HoverflyData["pairs"]->java.util.LinkedHashSet[0]->io.specto.hoverfly.junit.core.model.RequestResponsePair["request"]->io.specto.hoverfly.junit.core.model.Request["body"]->java.util.ArrayList[0]->io.specto.hoverfly.junit.core.model.RequestFieldMatcher["matcher"])

	at io.specto.hoverfly.junit.api.OkHttpHoverflyClient.getSimulation(OkHttpHoverflyClient.java:118)
	at io.specto.hoverfly.junit.core.Hoverfly.verifyAll(Hoverfly.java:507)

If possible, add screenshots to help explain your problem

So I read about the schema change for form and created a json file for my test:

{
  "data": {
    "pairs": [
      {
        "request": {
          "body": [
            {
              "matcher": "form",
              "value": {
                "grant_type": [
                  {
                    "matcher": "exact",
                    "value": "authorization_code"
                  }
                ],
                "code": [
                  {
                    "matcher": "exact",
                    "value": "some-code"
                  }
                ],
                "client_secret": [
                  {
                    "matcher": "exact",
                    "value": ""
                  }
                ],
                "client_id": [
                  {
                    "matcher": "exact",
                    "value": "some-client-id"
                  }
                ],
                "redirect_uri": [
                  {
                    "matcher": "exact",
                    "value": "https://example.org/callback"
                  }
                ]
              }
            }
          ],
          "path": [
            {
              "value": "/oauth2/token",
              "matcher": "exact"
            }
          ],
          "method": [
            {
              "value": "POST",
              "matcher": "exact"
            }
          ],
          "destination": [
            {
              "value": "www.my-test.com",
              "matcher": "exact"
            }
          ]
        },
        "response": {
          "body": "{\"access_token\":\"eyJ0.foo.bar\",\"expires_in\":9999999999,\"refresh_token\":\"eyJ0.foo.bar\",\"id_token\":\"eyJ0.foo.bar\",\"token_type\":\"Bearer\"}",
          "status": 200,
          "headers": {
            "Content-Type": [
              "application/json"
            ]
          },
          "templated": false,
          "encodedBody": false
        }
      }
    ],
    "globalActions": {
      "delays": [],
      "delaysLogNormal": []
    }
  },
  "meta": {
    "timeExported": "2023-10-23T14:13:54Z",
    "schemaVersion": "v5.2",
    "hoverflyVersion": "v1.5.0"
  }
}

my simplified test code (Kotlin and Junit5):

hoverfly.simulate(SimulationSource.classpath("sso/file.json"))

// do something

hoverfly.verifyAll()

before the update I used a slightly different setup (with ("io.specto:hoverfly-java-junit5:0.14.4") and ("io.specto:hoverfly-java:0.14.4")):

14.4. version 
  hoverfly.simulate(
      dsl(
         service(baseUrl)
           .post(startsWith(oauthTokenPath))
           .header("Content-Type", "application/x-www-form-urlencoded; charset=UTF-8")
           .anyBody()
           .willReturn(success(tokenResponse, "application/json")),
      ),
    )

// do something

hoverfly.verify(
      service(baseUrl)
        .post(startsWith(oauthTokenPath))
        .header("Content-Type", "application/x-www-form-urlencoded; charset=UTF-8")
        .anyQueryParams()
        // need to do regex based matching since those are not "real" query params: https://github.com/SpectoLabs/hoverfly/issues/903
        .body(
          RequestFieldMatcher.newRegexMatcher(
            "code=$codeParam.*grant_type=authorization_code.*client_secret=.*redirect_uri=${
              URLEncoder.encode(
                redirectUri,
                "UTF-8",
              )
            }.*client_id=$fakeClientId",
          ),
        ),
    )

Expected result

Additional relevant information

the underlying http request/framework that is responsible to construct and send the HTTP request (which is observed in this tests) uses oauth2-oidc-sdk-5.45. Complete project is Kotlin/Java with Junit5.
tommysitu added a commit that referenced this issue Dec 22, 2023
@tommysitu
#290 mapping form matcher to simulation object
cfc2d07
@tommysitu
Copy link
Member

tommysitu commented Dec 22, 2023 •
edited
Loading

I've raised a PR which should fix this issue: #292

tommysitu added a commit that referenced this issue Dec 22, 2023
@tommysitu
#290 mapping form matcher to simulation object
06ad4d3
@tommysitu
Copy link
Member

it's fixed in 0.16.0

@matthiastz
Copy link
Author

Hey @tommysitu 👋 , I tried to update the test in our Repo today to make use of the new form matcher (body).

Unfortunately, it still does not work.

My simplified test code is still the same (Kotlin and Junit5):

hoverfly.simulate(SimulationSource.classpath("sso/file.json"))

// do something

hoverfly.verifyAll()

I am 100% certain that the issue is related to the body verification (if I leave out the body matcher my test works fine - but thats obviously not what I need), unfortunately I do not know how to debug this issue further, as it seem to happen in the underlying Go service. Error:

WARN  2024-01-08T16:04:49,253 io.specto.hoverfly.junit.api.HoverflyClient - Failed to search journal: Unexpected response (code=500, message=PANIC: interface conversion: interface {} is nil, not func(interface {}, string) bool
goroutine 19 [running]:
github.com/codegangsta/negroni.(*Recovery).ServeHTTP.func1()
	/home/circleci/hoverfly/vendor/github.com/codegangsta/negroni/recovery.go:34 +0x94
panic({0x1052b5b80, 0x1400046cff0})
	/usr/local/go/src/runtime/panic.go:838 +0x204
github.com/SpectoLabs/hoverfly/core/matching.isMatching({{0x140004470f4, 0x4}, {0x1052aa480, 0x1400046cf30}, 0x0, 0x0}, {0x0?, 0x1400007e000?})
	/home/circleci/hoverfly/core/matching/field_matcher.go:43 +0x1d8
github.com/SpectoLabs/hoverfly/core/matching.FieldMatcher({0x1400046cfc0?, 0x1, 0x1052cae00?}, {0x0, 0x0})
	/home/circleci/hoverfly/core/matching/field_matcher.go:19 +0xc8
github.com/SpectoLabs/hoverfly/core/journal.(*Journal).GetFilteredEntries(0x140002ed860, {0x105263f00?})
	/home/circleci/hoverfly/core/journal/journal.go:191 +0x354
github.com/SpectoLabs/hoverfly/core/handlers/v2.(*JournalHandler).Post(0x14000267c10, {0x12c9c1b58, 0x14000402900}, 0x140003f4c40?, 0x0?)
	/home/circleci/hoverfly/core/handlers/v2/journal_handler.go:86 +0x80
github.com/codegangsta/negroni.HandlerFunc.ServeHTTP(0x14000425598?, {0x12c9c1b58?, 0x14000402900?}, 0x0?, 0x0?)
	/home/circleci/hoverfly/vendor/github.com/codegangsta/negroni/negroni.go:24 +0x40
github.com/codegangsta/negroni.middleware.ServeHTTP({{0x105366a98?, 0x140003fe150?}, 0x1400009f9b0?}, {0x12c9c1b58, 0x14000402900}, 0xa3?)
	/home/circleci/hoverfly/vendor/github.com/codegangsta/negroni/negroni.go:33 +0xc4
github.com/SpectoLabs/hoverfly/core/handlers.(*AuthHandler).RequireTokenAuthentication(0x105a805b8?, {0x12c9c1b58?, 0x14000402900?}, 0x140003f4c00?, 0x0?)
	/home/circleci/hoverfly/core/handlers/auth_handler.go:50 +0x158
github.com/codegangsta/negroni.HandlerFunc.ServeHTTP(0x1400006a480?, {0x12c9c1b58?, 0x14000402900?}, 0x1058d5ae8?, 0x14000425698?)
	/home/circleci/hoverfly/vendor/github.com/codegangsta/negroni/negroni.go:24 +0x40
github.com/codegangsta/negroni.middleware.ServeHTTP({{0x105366a98?, 0x140003fe140?}, 0x1400009f998?}, {0x12c9c1b58, 0x14000402900}, 0x104c36754?)
	/home/circleci/hoverfly/vendor/github.com/codegangsta/negroni/negroni.go:33 +0xc4
github.com/codegangsta/negroni.(*Negroni).ServeHTTP(0x140003f6b10, {0x12c9c1b58?, 0x140004028c0}, 0x140003ee168?)
	/home/circleci/hoverfly/vendor/github.com/codegangsta/negroni/negroni.go:73 +0xd0
github.com/go-zoo/bone.(*Route).parse(0x140003fc9a0, {0x12c9c1b58, 0x140004028c0}, 0x14000310500)
	/home/circleci/hoverfly/vendor/github.com/go-zoo/bone/route.go:171 +0x1b8
github.com/go-zoo/bone.(*Mux).parse(0x1400026bbc0, {0x12c9c1b58, 0x140004028c0}, 0x14000310500)
	/home/circleci/hoverfly/vendor/github.com/go-zoo/bone/helper.go:23 +0x9c
github.com/go-zoo/bone.(*Mux).DefaultServe(0x14000404320?, {0x12c9c1b58, 0x140004028c0}, 0x104b53f00?)
	/home/circleci/hoverfly/vendor/github.com/go-zoo/bone/bone.go:71 +0x30
github.com/go-zoo/bone.(*Mux).ServeHTTP(0x105a805b8?, {0x12c9c1b58?, 0x140004028c0?}, 0x140003f4be0?)
	/home/circleci/hoverfly/vendor/github.com/go-zoo/bone/bone.go:90 +0xb0
github.com/codegangsta/negroni.Wrap.func1({0x12c9c1b58, 0x140004028c0}, 0x1052bd601?, 0x140003f4be0)
	/home/circleci/hoverfly/vendor/github.com/codegangsta/negroni/negroni.go:41 +0x58
github.com/codegangsta/negroni.HandlerFunc.ServeHTTP(0x140000b1560?, {0x12c9c1b58?, 0x140004028c0?}, 0x1049c0a50?, 0x5000?)
	/home/circleci/hoverfly/vendor/github.com/codegangsta/negroni/negroni.go:24 +0x40
github.com/codegangsta/negroni.middleware.ServeHTTP({{0x105366a98?, 0x1400009fe60?}, 0x1400009fe90?}, {0x12c9c1b58, 0x140004028c0}, 0x12c9998a8?)
	/home/circleci/hoverfly/vendor/github.com/codegangsta/negroni/negroni.go:33 +0xc4
github.com/codegangsta/negroni.(*Recovery).ServeHTTP(0x1400006a480?, {0x12c9c1b58?, 0x140004028c0?}, 0x0?, 0x14000425a08?)
	/home/circleci/hoverfly/vendor/github.com/codegangsta/negroni/recovery.go:45 +0x78
github.com/codegangsta/negroni.middleware.ServeHTTP({{0x1053654b8?, 0x1400009fe18?}, 0x1400009fe78?}, {0x12c9c1b58, 0x140004028c0}, 0x104a06c9c?)
	/home/circleci/hoverfly/vendor/github.com/codegangsta/negroni/negroni.go:33 +0xc4
github.com/codegangsta/negroni.(*Negroni).ServeHTTP(0x140003f6ed0, {0x10536ab78?, 0x140004a81c0}, 0x1049c0a50?)
	/home/circleci/hoverfly/vendor/github.com/codegangsta/negroni/negroni.go:73 +0xd0
net/http.serverHandler.ServeHTTP({0x1053689d8?}, {0x10536ab78, 0x140004a81c0}, 0x14000310500)
	/usr/local/go/src/net/http/server.go:2916 +0x3fc
net/http.(*conn).serve(0x140002f1f40, {0x10536b280, 0x140003f7110})
	/usr/local/go/src/net/http/server.go:1966 +0x56c
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:3071 +0x450

I think the most important lines here are:

  • field_matcher.go:43 +0x1d8
  • field_matcher.go:19 +0xc8
  • journal.go:191

field_matcher.go:43 then panics, since it somehow tries to convert nil to func(interface {}, string) bool which is not possible. I have no clue where the nil comes from.
https://github.com/SpectoLabs/hoverfly/blob/d8b55cae005caec655a6b9440dd878d8b6e47af7/core/matching/field_matcher.go#L43

my simplified json schema, with which you hopefully can reproduce the issue:

{
  "data": {
    "pairs": [
      {
        "request": {
          "method": [
            {
              "value": "POST",
              "matcher": "exact"
            }
          ],
          "body": [
            {
              "matcher": "form",
              "value": {
                "grant_type": [
                  {
                    "matcher": "exact",
                    "value": "authorization_code"
                  }
                ]
              }
            }
          ]
        },
        "response": {
          "body": "{\"access_token\":\"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.foo.bar\",\"expires_in\":3600,\"refresh_token\":\"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.foo.bar\",\"id_token\":\"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.foo.bar\",\"token_type\":\"Bearer\"}",
          "status": 200,
          "headers": {
            "Content-Type": [
              "application/json"
            ]
          }
        }
      }
    ],
    "globalActions": {
      "delays": []
    }
  },
  "meta": {
    "schemaVersion": "v5.2"
  }
}

Is there any way with hoverfly + junit to somehow debug what is going on in the go service while running? Any extended logging, e.g. on the current field/currentMatcher? This would help.

Ty!

@tommysitu tommysitu reopened this Jan 8, 2024
@tommysitu
Copy link
Member

Hi @matthiastz thanks for the info. I will take a look as soon as I can!

@tommysitu
Copy link
Member

debugging time: #294 😬

@tommysitu
Copy link
Member

tommysitu commented Jan 9, 2024 •
edited
Loading

If you wanna debug hoverfly, you will need to download the go project, run it in debug mode in your IDE (I use Intellij for both java and go), and then configure the hoverfly rule in your junit test to point to the local instance on the default ports like this:

@ClassRule
    public static HoverflyRule hoverflyRule = HoverflyRule.inSimulationMode(simulationSource, 
HoverflyConfig.remoteConfigs().host("localhost").proxyPort(8500).adminPort(8888));

so when you run your junit test, it will pause the execution when it hits the breakpoint on the golang project (which is hoverfly running in debug mode).

@tommysitu
Copy link
Member

I've raised a PR on hoverfly to fix this issue: SpectoLabs/hoverfly#1110

@tommysitu
Copy link
Member

fixed in v0.16.1

@matthiastz
Copy link
Author

Hey @tommysitu - thx so much for your support and the hint to debug hoverfly!! Can confirm that its working now as expected :)

@tommysitu
Copy link
Member

🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tommysitu @matthiastz