Can't use matchers in body with `'Content-Type': 'application/x-www-form-urlencoded'` #767

colossatr0n · 2021-11-05T01:19:43Z

Software versions

Please provide at least OS and version of pact-js

OS: Mac OSX 10.15.7
Versions:
@pact-foundation/karma-pact: 3.1.0,
@pact-foundation/pact: 9.16.5,
@pact-foundation/pact-web: 9.16.5,

Issue Checklist

Please confirm the following:

I have upgraded to the latest
I have the read the FAQs in the Readme
I have triple checked, that there are no unhandled promises in my code and have read the section on intermittent test failures
I have set my log level to debug and attached a log file showing the complete request/response cycle
For bonus points and virtual high fives, I have created a reproduceable git repository (see below) to illustrate the problem

Expected behaviour

When using pact-web, matchers should be able to be used in the body when the content type is application/x-www-form-urlencoded.

Actual behaviour

Mock server logs report the following:

E, [2021-11-04T18:50:22.192860 #41190] ERROR -- : Error ocurred in mock service: NoMethodError - undefined method `ascii_only?' for #<Pact::Term:0x007fa5d60b4f40>

I believe this is related to #633.

Steps to reproduce

Create a test with the following interaction:

// Failing interaction that uses body matchers and form url encoded content type.
beforeAll(done => {
    provider.addInteraction({
        state: 'some state',
        uponReceiving: 'some request',
        withRequest: {
            method: 'POST',
            path: "/some/path",
            // ISSUE HERE 
            body: Matchers.regex(
                {
                    matcher:  "someFormParam=\\d+",
                    generate: "someFormParam=1",
                }
            ),
            headers: { 'Content-Type': 'application/x-www-form-urlencoded'}
        },
        willRespondWith: {
           status: 200,
            body: {}
        }
    }).then(done);
})

The above test will fail with the log error noted earlier.

Contrast that with this interaction, which will pass:

// Passing interaction that removes the body matcher.
beforeAll(done => {
    provider.addInteraction({
        state: 'some state',
        uponReceiving: 'some request',
        withRequest: {
            method: 'POST',
            path: "/some/path",
            // Body matcher replaced with string 
            body: "someFormParam=1"
            headers: { 'Content-Type': 'application/x-www-form-urlencoded'}
        },
        willRespondWith: {
           status: 200,
            body: {}
        }
    }).then(done);
})

Relevant log files

I, [2021-11-04T18:50:22.136830 #41190]  INFO -- : Received OPTIONS request for mock service administration endpoint DELETE /interactions. Returning CORS headers: {"Access-Control-Allow-Origin"=>"http://localhost:9876", "Access-Control-Allow-Headers"=>"content-type,x-pact-mock-service", "Access-Control-Allow-Methods"=>"DELETE, POST, GET, HEAD, PUT, TRACE, CONNECT, PATCH"}.
I, [2021-11-04T18:50:22.145129 #41190]  INFO -- : Cleared interactions
I, [2021-11-04T18:50:22.156703 #41190]  INFO -- : Registered expected interaction POST /some/path
D, [2021-11-04T18:50:22.157159 #41190] DEBUG -- : {
  "description": "some description",
  "providerState": "some state",
  "request": {
    "method": "POST",
    "path": "/some/path",
    "headers": {
      "Content-Type": "application/x-www-form-urlencoded"
    },
    "body": {
      "json_class": "Pact::Term",
      "data": {
        "generate": "someFormParam=1",
        "matcher": {
          "json_class": "Regexp",
          "o": 0,
          "s": "someFormParam=\\d+"
        }
      }
    }
  },
  "response": {
    "status": 200,
    "headers": {
    },
    "body": { }
  },
  "metadata": null
}
I, [2021-11-04T18:50:22.191358 #41190]  INFO -- : Received request POST /some/path
D, [2021-11-04T18:50:22.191627 #41190] DEBUG -- : {
  "path": "/some/path",
  "query": "",
  "method": "post",
  "body": "someFormParam=1",
  "headers": {
    "Content-Length": "15",
    "Content-Type": "application/x-www-form-urlencoded",
    "X-Forwarded-Host": "localhost:9876",
    "X-Forwarded-Proto": "http",
    "X-Forwarded-Port": "9876",
    "X-Forwarded-For": "127.0.0.1",
    "Accept-Language": "en-US",
    "Accept-Encoding": "gzip, deflate, br",
    "Referer": "http://localhost:9876/context.html",
    "Sec-Fetch-Dest": "empty",
    "Sec-Fetch-Mode": "cors",
    "Sec-Fetch-Site": "same-origin",
    "Origin": "http://localhost:9876",
    "Sec-Ch-Ua-Platform": "",
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/95.0.4638.69 Safari/537.36",
    "Sec-Ch-Ua-Mobile": "?0",
    "Accept": "application/json, text/plain, */*",
    "Sec-Ch-Ua": "",
    "Connection": "keep-alive",
    "Host": "localhost:9876",
    "Version": "HTTP/1.1"
  }
}
E, [2021-11-04T18:50:22.192860 #41190] ERROR -- : Error ocurred in mock service: NoMethodError - undefined method `ascii_only?' for #<Pact::Term:0x007fa5d60b4f40>
E, [2021-11-04T18:50:22.192981 #41190] ERROR -- : /Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/ruby/lib/ruby/2.2.0/uri/common.rb:450:in `decode_www_form'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/pact-support-1.17.0/lib/pact/shared/form_differ.rb:26:in `decode_www_form'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/pact-support-1.17.0/lib/pact/shared/form_differ.rb:15:in `to_hash'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/pact-support-1.17.0/lib/pact/shared/form_differ.rb:8:in `call'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/pact-support-1.17.0/lib/pact/consumer_contract/request.rb:72:in `body_diff'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/pact-support-1.17.0/lib/pact/consumer_contract/request.rb:42:in `difference'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/pact-support-1.17.0/lib/pact/consumer_contract/request.rb:28:in `matches?'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/interactions/candidate_interactions.rb:8:in `block in matching_interactions'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/interactions/candidate_interactions.rb:7:in `select'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/interactions/candidate_interactions.rb:7:in `matching_interactions'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/request_handlers/interaction_replay.rb:55:in `find_response'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/request_handlers/interaction_replay.rb:45:in `respond'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/request_handlers/base_request_handler.rb:17:in `call'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/rack-2.1.4/lib/rack/cascade.rb:35:in `block in call'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/rack-2.1.4/lib/rack/cascade.rb:26:in `each'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/rack-2.1.4/lib/rack/cascade.rb:26:in `call'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/consumer/mock_service/cors_origin_header_middleware.rb:11:in `call'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/consumer/mock_service/error_handler.rb:13:in `call'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/app.rb:34:in `call'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/consumer/mock_service/set_location.rb:14:in `call'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/rack-2.1.4/lib/rack/handler/webrick.rb:88:in `service'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/webrick-1.3.1/lib/webrick/httpserver.rb:138:in `service'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/webrick-1.3.1/lib/webrick/httpserver.rb:94:in `run'
/Users/someuser/webapp/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/lib/vendor/ruby/2.2.0/gems/webrick-1.3.1/lib/webrick/server.rb:191:in `block in start_thread'
I, [2021-11-04T18:50:22.201372 #41190]  INFO -- : Received OPTIONS request for mock service administration endpoint GET /interactions/verification. Returning CORS headers: {"Access-Control-Allow-Origin"=>"http://localhost:9876", "Access-Control-Allow-Headers"=>"content-type,x-pact-mock-service", "Access-Control-Allow-Methods"=>"DELETE, POST, GET, HEAD, PUT, TRACE, CONNECT, PATCH"}.
W, [2021-11-04T18:50:22.205640 #41190]  WARN -- : Verifying - actual interactions do not match expected interactions. 
Missing requests:
	POST /some/path



W, [2021-11-04T18:50:22.205722 #41190]  WARN -- : Missing requests:
	POST /some/path



I, [2021-11-04T18:50:22.209135 #41190]  INFO -- : Received OPTIONS request for mock service administration endpoint POST /pact. Returning CORS headers: {"Access-Control-Allow-Origin"=>"http://localhost:9876", "Access-Control-Allow-Headers"=>"content-type,x-pact-mock-service", "Access-Control-Allow-Methods"=>"DELETE, POST, GET, HEAD, PUT, TRACE, CONNECT, PATCH"}.
I, [2021-11-04T18:50:22.222491 #41190]  INFO -- : Cleared interactions

The text was updated successfully, but these errors were encountered:

mefellows · 2021-11-05T01:26:20Z

We don't support matchers on non-JSON or XML bodies at this time.

This could be marked as an enhancement, or (preferably) be supported via the new plugins process when ready.

mefellows · 2022-07-29T11:09:13Z

cc @uglyog

jcadavez · 2024-12-13T20:35:25Z

Curious, is there an alternative way to implement checking application/x-www-form-urlencoded requests with Pact if it doesn't accept in format below?

        .withRequest('POST', '/url/path', builder => {
          builder
            .headers({
              Authorization: 'Basic xxx',
              'Content-Type': 'application/x-www-form-urlencoded',
            })
            .body('application/x-www-form-urlencoded', Buffer.from('[email protected]'));
          return builder;
        })

mefellows · 2024-12-17T22:34:10Z

I'd need to check. It should support the content-type, just without matching rules.

FYI see also this recent RFC: https://github.com/pact-foundation/roadmap/blob/master/rfc/99-define-matching-rules-for-form-urlencoded-body.md

mefellows · 2024-12-19T04:53:17Z

Matcher support (and generators) are now added to the core engine here: pact-foundation/pact-reference#462.

This might mean things "just work" for this once the new FFI is updated in pact-core.

Note to review this once the new FFI is released.

colossatr0n added the bug Indicates an unexpected problem or unintended behavior label Nov 5, 2021

TimothyJones added enhancement and removed bug Indicates an unexpected problem or unintended behavior labels Nov 5, 2021

mefellows removed the enhancement label Jul 12, 2023

mefellows added the enhancement Indicates new feature requests label Aug 18, 2023

mefellows added this to Pact Triage (not yet in use) Aug 18, 2023

github-project-automation bot moved this to New Issue in Pact Triage (not yet in use) Aug 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can't use matchers in body with `'Content-Type': 'application/x-www-form-urlencoded'` #767

Can't use matchers in body with `'Content-Type': 'application/x-www-form-urlencoded'` #767

colossatr0n commented Nov 5, 2021

mefellows commented Nov 5, 2021

mefellows commented Jul 29, 2022

jcadavez commented Dec 13, 2024

mefellows commented Dec 17, 2024

mefellows commented Dec 19, 2024

Can't use matchers in body with 'Content-Type': 'application/x-www-form-urlencoded' #767

Can't use matchers in body with 'Content-Type': 'application/x-www-form-urlencoded' #767

Comments

colossatr0n commented Nov 5, 2021

Software versions

Issue Checklist

Expected behaviour

Actual behaviour

Steps to reproduce

Relevant log files

mefellows commented Nov 5, 2021

mefellows commented Jul 29, 2022

jcadavez commented Dec 13, 2024

mefellows commented Dec 17, 2024

mefellows commented Dec 19, 2024

Can't use matchers in body with `'Content-Type': 'application/x-www-form-urlencoded'` #767

Can't use matchers in body with `'Content-Type': 'application/x-www-form-urlencoded'` #767