Non-unique http request body parameters are overwritten

[johndwalker]

Brief summary

Making an http request whose body includes multiple parameters of the same name results in only one of those parameters being included in the actual request.

Consider the following http request:

response = http.post(`https://www.myserver.com/api/`, {
    "flag": `A`,
    "flag": `B`,
    "flag": `C`,
    "flag": `D`,
    "flag": `E`,
    "flag": `F`
  }, {
    "headers": {
      "Content-Type": `application/x-www-form-urlencoded; charset=UTF-8`,
      "Accept": `text/html, */*; q=0.01`,
      "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.70 Safari/537.36"
    },
  }
)

When executed, a Fiddler trace shows that only a single body parameter was included in the request (flag=F). The same request from JMeter correctly includes all six body parameters of the same name.

According to the HTML 5 standard, web forms are allowed to have multiple controls share the same name (see here).

Although I am not a Go developer, I looked at the code that parses requests, and it seems that it uses a map to store body parameters. Due to the nature of how a map works, it would naturally lead to data loss as it would override each previous value mapped to the specified key. Apologies if I am mistaken here.

If anyone has any input as to a potential workaround, I am all-ears. In the meantime, I will try to come up with a solution.

k6 version

k6 v0.41.0 ((devel), go1.19.3, darwin/arm64)

OS

macOS 12.6.1

Docker version and image (if applicable)

No response

Steps to reproduce the problem

Execute an http request with multiple body parameters of the same name.

Expected behaviour

All body parameters are included in the actual http request as seen in a Fiddler trace.

Actual behaviour

Only one of the parameters with a non-unique name is included in the actual request.

[na--]

You are right, though the problem here comes not from k6 itself, but from JavaScript: you can't have a JS object with duplicate keys. If you just do this:

console.log({
    "flag": `A`,
    "flag": `B`,
    "flag": `C`,
    "flag": `D`,
    "flag": `E`,
    "flag": `F`
});

You would always get just {"flag":"F"} on the console, whether in k6 or in Node.js or in a browser. The workaround is to encode the request body yourself in a application/x-www-form-urlencoded format and pass it to http.post() as a string or an ArrayBuffer. You can use the form-urlencoded library from jslib.k6.io or something like it to do that.

This is a limitation of the current k6/http API that we can't do anything about, at the moment, without making a major breaking change to the API... 😞 There are many other similar issues with that API, so we'll need a new HTTP API (#2461) eventually, to address most of them. I'll close this issue, since the problem is not actually a bug in k6 and has an easy workaround.