From 0560b4f25c94e140a8f4261f721ebe7a18b99513 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ivan=20Miri=C4=87?= <ivan.miric@grafana.com>
Date: Fri, 10 Mar 2023 16:19:08 +0100
Subject: [PATCH 01/16] Add design document for the new HTTP API

---
 docs/design/018-new-http-api.md | 248 ++++++++++++++++++++++++++++++++
 1 file changed, 248 insertions(+)
 create mode 100644 docs/design/018-new-http-api.md

diff --git a/docs/design/018-new-http-api.md b/docs/design/018-new-http-api.md
new file mode 100644
index 00000000000..1bf589796c5
--- /dev/null
+++ b/docs/design/018-new-http-api.md
@@ -0,0 +1,248 @@
+# New HTTP API
+
+## Authors
+The k6 core team
+
+## Why is this needed?
+
+The HTTP API (in k6 <=v0.43.0) used by k6 scripts has many limitations, inconsistencies and performance issues, that lead to a poor user experience. Considering that it's the most commonly used JS API, improving it would benefit most k6 users.
+
+The list of issues with the current API is too long to mention in this document, but you can see a detailed list of [GitHub issues labeled `new-http`](https://github.com/grafana/k6/issues?q=is%3Aopen+is%3Aissue+label%3Anew-http) that should be fixed by this proposal, as well as the [epic issue #2461](https://github.com/grafana/k6/issues/2461). Here we'll only mention the relatively more significant ones:
+
+* [#2311](https://github.com/grafana/k6/issues/2311): files being uploaded are copied several times in memory, causing more memory usage than necessary. Related issue: [#1931](https://github.com/grafana/k6/issues/1931)
+* [#857](https://github.com/grafana/k6/issues/857), [#1045](https://github.com/grafana/k6/issues/1045): it's not possible to configure transport options, such as proxies or DNS, per VU or group of requests.
+* [#761](https://github.com/grafana/k6/issues/761): specifying configuration options globally is not supported out-of-the-box, and workarounds like the [httpx library](https://k6.io/docs/javascript-api/jslib/httpx/) are required.
+* [#746](https://github.com/grafana/k6/issues/746): async functionality like Server-sent Events is not supported.
+* Related to the previous point, all (except asyncRequest) current methods are synchronous, which is inflexible, and doesn't align with modern APIs from other JS runtimes.
+* [#436](https://github.com/grafana/k6/issues/436): the current API is not very friendly or ergonomic. Different methods also have parameters that change places, e.g. `params` is the second argument in `http.get()`, but the third one in `http.post()`.
+
+
+## Proposed solution(s)
+
+### Design
+
+In general, the design of the API should follow these guidelines:
+
+- It should be familiar to users of HTTP APIs from other JS runtimes, and easy for new users to pick up.
+
+  As such, it would serve us well to draw inspiration from existing runtimes and frameworks. Particularly:
+
+  - The [Fetch API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API), a [WHATWG standard](https://fetch.spec.whatwg.org/) supported by most modern browsers.
+    [Deno's implementation](https://deno.land/manual/examples/fetch_data) and [GitHub's polyfill](https://github.com/github/fetch) are good references to follow.
+
+    This was already suggested in [issue #2424](https://github.com/grafana/k6/issues/2424).
+
+  - The [Streams API](https://developer.mozilla.org/en-US/docs/Web/API/Streams_API), a [WHATWG standard](https://streams.spec.whatwg.org/) supported by most modern browsers.
+    [Deno's implementation](https://deno.land/manual@v1.30.3/examples/fetch_data#files-and-streams) is a good reference to follow.
+
+    There's a related, but very old [proposal](https://github.com/grafana/k6/issues/592) before the Streams API was standardized, so we shouldn't use it, but it's clear there's community interest in such an API.
+
+    Streaming files both from disk to RAM to the network, and from network to RAM and possibly disk, would also partly solve our [performance and memory issues with loading large files](https://github.com/grafana/k6/issues/2311).
+
+  - Native support for the [`FormData` API](https://developer.mozilla.org/en-US/docs/Web/API/FormData).
+
+    Currently this is supported with a [JS polyfill](https://k6.io/docs/examples/data-uploads/#advanced-multipart-request), which should be deprecated.
+
+  - Aborting requests or any other async process with the [`AbortSignal`/`AbortController` API](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal), part of the [WHATWG DOM standard](https://dom.spec.whatwg.org/#aborting-ongoing-activities).
+
+    This is slightly out of scope for the initial phases of implementation, but aborting async processes like `fetch()` is an important feature.
+
+- The Fetch API alone would not address all our requirements (e.g. specifying global and transport options), so we still need more flexible and composable interfaces.
+
+  One source of inspiration is the Go `net/http` package, which the k6 team is already familiar with. Based on this, our JS API could have similar entities:
+
+  - `Dialer`: a low-level interface for configuring TCP/IP options, such as TCP timeouts, keep-alive duration, DNS, and IP version preferences.
+
+  - `Transport`: interface for configuring HTTP connection options, such as proxies, TLS, HTTP version preferences, etc.
+
+    It enables advanced behaviors like intercepting requests before they're sent to the server.
+
+  - `Client`: the main entrypoint for making requests, it encompasses all of the above options. A k6 script should be able to initialize more than one `Client`, each with their separate configuration.
+
+    In order to simplify the API, the creation of a `Client` should use sane defaults for `Dialer` and `Transport`.
+
+  There should be some research into existing JS APIs that offer similar features (e.g. Node/Deno), as we want to offer an API familiar to JS developers, not necessarily Go developers.
+
+  - `Request`/`Response`: represent objects sent by the client, and received from the server. In contrast to the current API, the k6 script should be able to construct `Request` objects declaratively, and then reuse them to make multiple requests with the same (or similar) data.
+
+- All methods that perform I/O calls must be asynchronous. Now that we have `Promise`, event loop and `async`/`await` support natively in k6, there's no reason for these to be synchronous anymore.
+
+- The API should avoid any automagic behavior. That is, it should not attempt to infer desired behavior or options based on some implicit value.
+
+  We've historically had many issues with this ([#878](https://github.com/grafana/k6/issues/878), [#1185](https://github.com/grafana/k6/issues/1185)), resulting in confusion for users, and we want to avoid it in the new API. Even though we want to have sane defaults for most behavior, instead of guessing what the user wants, all behavior should be explicitly configured by the user. In cases where some behavior is ambiguous, the API should raise an error indicating so.
+
+
+### Implementation
+
+Trying to solve all `new-http` issues with a single large and glorious change wouldn't be reasonable, so improvements will undoubtedly need to be done gradually, in several phases, and over several k6 development cycles.
+
+With this in mind, we propose the following phases:
+
+#### Phase 1: create initial k6 extension
+
+**Goals**:
+
+- Implement a barebones async API that serves as a proof-of-concept for what the final developer experience will look and feel like.
+  The code should be in a state that allows it to be easily extended.
+
+  By barebones, we mean:
+
+  - The `Client` interface with only one method: `request()`, which will work similarly to the current `http.asyncRequest()`.
+
+    For the initial PoC, it's fine if only `GET` and `POST` methods are supported.
+
+    It's not required to make `Dialer` and `Transport` fully configurable at this point, but they should use sane defaults, and it should be clear how the configuration will be done.
+
+- This initial API should solve a minor, but concrete, issue of current API. It should fix something that's currently not possible and doesn't have a good workaround.
+
+  Good candidates: [#936](https://github.com/grafana/k6/issues/936), [#970](https://github.com/grafana/k6/issues/970).
+
+- Features like configuring options globally, or per VU or request, should be implemented.
+  Deprecating the `httpx` library should be possible after this phase.
+
+
+**Non-goals**:
+
+- We won't yet try to solve performance/memory issues of the current API, or implement major new features like data streaming.
+
+
+#### Phase 2: work on major issues
+
+**Goals**:
+
+- Work should be started on some of the most impactful issues from the current API.
+  Issues like high memory usage when uploading files ([#2311](https://github.com/grafana/k6/issues/2311)), and data streaming ([#592](https://github.com/grafana/k6/issues/592)), are good candidates to focus on first.
+
+
+#### Phase 3: work on leftover issues
+
+**Goals**:
+
+- All leftover `new-http` issues should be worked on in this phase.
+  **TODO**: Specify which issues and in what order should be worked on here.
+
+- The extension should be thoroughly tested, by both internal and external users.
+
+
+#### Phase 4: expand, polish and stabilize the API
+
+**Goals**:
+
+- The API should be expanded to include all HTTP methods supported by the current API.
+  For the most part, it should reach feature parity with the current API.
+
+- A standalone `fetch()` function should be added that resembles the web Fetch API. There will be some differences in the options compared to the web API, as we want to make parts of the transport/client configurable.
+
+    Internally, this function will create a new client (or reuse a global one?), and will simply act as a convenience wrapper over the underlying `Client`/`Dialer`/`Transport` implementations, which will be initialized with sane default values.
+
+- Towards the end of this phase, the API should be mostly stable, based on community feedback.
+  Small changes will be inevitable, but there should be no discussion about the overall design.
+
+
+#### Phase 5: merge into k6-core, more testing
+
+At this point the extension should be relatively featureful and stable to be useful to all k6 users.
+
+**Goals**:
+
+- Merge the extension into k6 core, and make it available to k6 Cloud users.
+
+- Continue to gather and address feedback from users, thorough testing and polishing.
+
+
+#### Phase 6: deprecate `k6/http`
+
+As the final phase, we should add deprecation warnings when `k6/http` is used, and point users to the new API.
+Eventually, months down the line, we can consider replacing `k6/http` altogether with the new module.
+
+
+### Examples
+
+- `basic-get.js`:
+  ```javascript
+  import { fetch } from 'k6/x/net/http';
+
+  export default async function () {
+    // Creates a new default Client internally.
+    const response = await fetch('https://httpbin.test.k6.io/get');
+    const jsonData = await response.json();
+    console.log(jsonData);
+  }
+  ```
+
+- `basic-post.js`:
+  ```javascript
+  import { fetch } from 'k6/x/net/http';
+
+  export default async function () {
+    await fetch('https://httpbin.test.k6.io/post', {
+      method: 'POST',
+      json: { name: 'k6' }, // automatically adds 'Content-Type: application/json' header
+    });
+  }
+  ```
+
+- `basic-get-request.js`:
+  ```javascript
+  import { fetch, Request } from 'k6/x/net/http';
+
+  export default async function () {
+    const request = new Request('https://httpbin.test.k6.io/get', {
+      headers: { 'Case-Sensitive-Header': 'somevalue' },
+      // Other options that can also be passed directly to fetch()
+    });
+    const response = await fetch(request, {
+      // Some options that will merge with or override the options specified
+      // in the Request.
+    });
+    const jsonData = await response.json();
+    console.log(jsonData);
+  }
+  ```
+
+- `advanced-get.js`:
+  ```javascript
+  import { Client } from 'k6/x/net/http';
+
+  const client = new Client({
+    proxy: 'https://myproxy',
+    forceHTTP1: true,
+    dns: { ... },
+    // other transport options: h2c, TLS, etc.
+  });
+
+  export default async function () {
+    const response = await client.get('https://httpbin.test.k6.io/get');
+    const jsonData = await response.json();
+    console.log(jsonData);
+  }
+  ```
+
+- `advanced-post-streaming.js`:
+  ```javascript
+  import { File } from 'k6/x/file';
+  import { fetch } from 'k6/x/net/http';
+
+  // Will need supporting await in init context
+  const file = await File.open('./logo.svg');  // by default assumes 'read'
+
+  export default async function () {
+    await fetch('https://httpbin.test.k6.io/post', {
+      method: 'POST',
+      body: file.readable,
+    });
+  }
+  ```
+
+
+## Potential risks
+
+* Long implementation time.
+
+  Not so much of a risk, but more of a necessary side-effect of spreading the work in phases, and over several development cycles. We need this approach in order to have ample time for community feedback, to implement any unplanned features, and to make sure the new API fixes all existing issues.
+  Given this, it's likely that the entire process will take many months, possibly more than a year to finalize.
+
+
+## Technical decisions
+
+TBD after team discussion. In the meantime, see the "Proposed solution(s)" section.

From 529e4fd0a93454e566b4d6a22b6456b116ec51ee Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ivan=20Miri=C4=87?= <ivan.miric@grafana.com>
Date: Fri, 17 Mar 2023 19:27:11 +0100
Subject: [PATCH 02/16] Add design details about Sockets, Client and other APIs

Also, move examples to their specific section.
---
 docs/design/018-new-http-api.md | 306 +++++++++++++++++++++++---------
 1 file changed, 227 insertions(+), 79 deletions(-)

diff --git a/docs/design/018-new-http-api.md b/docs/design/018-new-http-api.md
index 1bf589796c5..a5c0701da0a 100644
--- a/docs/design/018-new-http-api.md
+++ b/docs/design/018-new-http-api.md
@@ -72,6 +72,233 @@ In general, the design of the API should follow these guidelines:
   We've historically had many issues with this ([#878](https://github.com/grafana/k6/issues/878), [#1185](https://github.com/grafana/k6/issues/1185)), resulting in confusion for users, and we want to avoid it in the new API. Even though we want to have sane defaults for most behavior, instead of guessing what the user wants, all behavior should be explicitly configured by the user. In cases where some behavior is ambiguous, the API should raise an error indicating so.
 
 
+#### Sockets
+
+A Socket represents the file or network socket over which client/server or peer communication happens.
+
+It can be of three types:
+- `tcp`: a stream-oriented network socket using the Transmission Control Protocol.
+- `udp`: a message-oriented network socket using the User Datagram Protocol.
+- `ipc`: a mechanism for communicating between processes on the same machine, typically using files.
+
+The Socket state can either be _active_—meaning connected for a TCP socket, bound for a UDP socket, or open for an IPC socket—, or _inactive_—meaning disconnected, unbound, or closed, respectively.
+
+##### Example
+
+- TCP:
+```javascript
+import { dialTCP } from 'k6/x/net';
+import { Client } from 'k6/x/net/http';
+
+export default async function () {
+  const socket = await dialTCP('192.168.1.1:80', {
+                            // default      | possible values
+    ipVersion: 0,           // 0            | 4 (IPv4), 6 (IPv6), 0 (both)
+    keepAlive: true,        // false        |
+    lookup: null,           // dns.lookup() |
+    proxy: 'myproxy:3030',  // ''           |
+  });
+  console.log(socket.active); // true
+
+  // Writing directly to the socket.
+  // Requires TextEncoder implementation, otherwise typed arrays can be used as well.
+  await socket.write(new TextEncoder().encode('GET / HTTP/1.1\r\n\r\n'));
+
+  // And reading...
+  socket.on('data', (data) => {
+    console.log(`received ${data}`);
+    socket.close();
+  });
+}
+```
+
+- UDP:
+```javascript
+import { dialUDP } from 'k6/x/net';
+import { Client } from 'k6/x/net/http';
+
+export default async function () {
+  const socket = new dialUDP('192.168.1.1:9090');
+
+  await socket.write(new TextEncoder().encode('GET / HTTP/1.1\r\n\r\n'));
+}
+```
+
+- IPC:
+```javascript
+import { open } from 'k6/x/file';
+import { Client } from 'k6/x/net/http';
+
+export default async function () {
+  const file = await open('/tmp/unix.sock');
+
+  // The HTTP client supports communicating over a Unix socket.
+  // Otherwise it can also be read from and written to directly.
+  const client = new Client({
+    socket: file,
+  });
+  await client.get('http://127.0.0.1/get');
+}
+```
+
+#### Client
+
+An HTTP Client is used to communicate with an HTTP server.
+
+##### Examples
+
+- Using a client with default transport settings, and making a GET request:
+```javascript
+import { Client } from 'k6/x/net/http';
+
+export default async function () {
+  const client = new Client();
+  const response = await client.get('https://httpbin.test.k6.io/get');
+  const jsonData = await response.json();
+  console.log(jsonData);
+}
+```
+
+- Passing a socket with custom transport settings, some HTTP options, and making a POST request:
+```javascript
+import { dialTCP } from 'k6/x/net';
+import { Client } from 'k6/x/net/http';
+
+export default async function () {
+  const socket = await dialTCP('10.0.0.10:80, { keepAlive: true });
+  const client = new Client({
+    socket: socket,
+    proxy: 'https://myproxy',
+    version: 1.1,                     // force a specific HTTP version
+    headers: { 'User-Agent': 'k6' },  // set some global headers
+  });
+  await client.post('http://10.0.0.10/post', {
+    json: { name: 'k6' }, // automatically adds 'Content-Type: application/json' header
+  });
+}
+```
+
+- A tentative HTTP/3 example:
+```javascript
+import { dialUDP } from 'k6/x/net';
+import { Client } from 'k6/x/net/http';
+
+export default async function () {
+  const socket = new dialUDP('192.168.1.1:9090');
+
+  const client = new Client({
+    socket: socket,
+    version: 3,      // A UDP socket would imply HTTP/3, but this makes it explicit.
+  });
+  await client.get('https://httpbin.test.k6.io/get');
+}
+```
+
+
+#### Host name resolution
+
+Host names can be resolved to IP addresses in several ways:
+
+- Via a static lookup map defined in the script.
+- Via the operating system's facilities (`/etc/hosts`, `/etc/resolv.conf`, etc.).
+- By querying specific DNS servers.
+
+When connecting to an address using a host name, the resolution can be controlled via the `lookup` function passed to the socket constructor. By default, the mechanism provided by the operating system is used (`dns.lookup()`).
+
+For example:
+```javascript
+import { dialTCP } from 'k6/x/net';
+import dns from 'k6/x/net/dns';
+
+const hosts = {
+  'hostA': '10.0.0.10',
+  'hostB': '10.0.0.11',
+};
+
+export default async function () {
+  const socket = await dialTCP('myhost', {
+    lookup: async hostname => {
+      // Return either the IP from the static map, or do an OS lookup,
+      // or fallback to making a DNS query to specific servers.
+      return hosts[hostname] || await dns.lookup(hostname) ||
+        await dns.resolve(hostname, {
+          rrtype: 'A',
+          servers: ['1.1.1.1:53', '8.8.8.8:53'],
+        });
+    },
+  });
+}
+```
+
+#### Requests and responses
+
+HTTP requests can be created declaratively, and sent only when needed. This allows reusing request data to send many similar requests.
+
+For example:
+```javascript
+import { Client, Request } from 'k6/x/net/http';
+
+export default async function () {
+  const client = new Client({
+    headers: { 'User-Agent': 'k6' },  // set some global headers
+  });
+  const request = new Request('https://httpbin.test.k6.io/get', {
+    // These will be merged with the Client options.
+    headers: { 'Case-Sensitive-Header': 'somevalue' },
+  });
+  const response = await client.get(request, {
+    // These will override any options for this specific submission.
+    headers: { 'Case-Sensitive-Header': 'anothervalue' },
+  });
+  const jsonData = await response.json();
+  console.log(jsonData);
+}
+```
+
+
+#### Data streaming
+
+The [Streams API](https://developer.mozilla.org/en-US/docs/Web/API/Streams_API) allows streaming data that is received or sent over the network, or read from or written to the local filesystem. This enables more efficient usage of memory, as only chunks of it need to be allocated at once.
+
+This is a separate project from the HTTP API, tracked in [issue #2978](https://github.com/grafana/k6/issues/2978), and involves changes in other parts of k6. Certain HTTP API functionality, however, depends on this API being available.
+
+An example inspired by [Deno](https://deno.land/manual/examples/fetch_data#files-and-streams) of how this might work in k6:
+```javascript
+import { open } from 'k6/x/file';
+import { Client } from 'k6/x/net/http';
+
+// Will need supporting await in init context
+const file = await open('./logo.svg');  // by default assumes 'read'
+
+export default async function () {
+  const client = new Client();
+  await client.post('https://httpbin.test.k6.io/post', { body: file.readable });
+}
+```
+
+
+#### Fetch API
+
+The [Fetch API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) is a convenience wrapper over existing Client, Socket and other low-level interfaces, with the benefit of being easy to use, and having sane defaults. It's a quick way to fire off some HTTP requests and get some responses, without worrying about advanced configuration.
+
+The implementation in k6 differs slightly from the web API, but we've tried to make it familiar to use wherever possible.
+
+Example:
+```
+import { fetch } from 'k6/x/net/http';
+
+export default async function () {
+  await fetch('https://httpbin.test.k6.io/get');
+  await fetch('https://httpbin.test.k6.io/post', {
+    // Supports the same options as Client.request()
+    method: 'POST',
+    headers: { 'User-Agent': 'k6' },
+    json: { name: 'k6' },
+  });
+}
+```
+
+
 ### Implementation
 
 Trying to solve all `new-http` issues with a single large and glorious change wouldn't be reasonable, so improvements will undoubtedly need to be done gradually, in several phases, and over several k6 development cycles.
@@ -156,85 +383,6 @@ As the final phase, we should add deprecation warnings when `k6/http` is used, a
 Eventually, months down the line, we can consider replacing `k6/http` altogether with the new module.
 
 
-### Examples
-
-- `basic-get.js`:
-  ```javascript
-  import { fetch } from 'k6/x/net/http';
-
-  export default async function () {
-    // Creates a new default Client internally.
-    const response = await fetch('https://httpbin.test.k6.io/get');
-    const jsonData = await response.json();
-    console.log(jsonData);
-  }
-  ```
-
-- `basic-post.js`:
-  ```javascript
-  import { fetch } from 'k6/x/net/http';
-
-  export default async function () {
-    await fetch('https://httpbin.test.k6.io/post', {
-      method: 'POST',
-      json: { name: 'k6' }, // automatically adds 'Content-Type: application/json' header
-    });
-  }
-  ```
-
-- `basic-get-request.js`:
-  ```javascript
-  import { fetch, Request } from 'k6/x/net/http';
-
-  export default async function () {
-    const request = new Request('https://httpbin.test.k6.io/get', {
-      headers: { 'Case-Sensitive-Header': 'somevalue' },
-      // Other options that can also be passed directly to fetch()
-    });
-    const response = await fetch(request, {
-      // Some options that will merge with or override the options specified
-      // in the Request.
-    });
-    const jsonData = await response.json();
-    console.log(jsonData);
-  }
-  ```
-
-- `advanced-get.js`:
-  ```javascript
-  import { Client } from 'k6/x/net/http';
-
-  const client = new Client({
-    proxy: 'https://myproxy',
-    forceHTTP1: true,
-    dns: { ... },
-    // other transport options: h2c, TLS, etc.
-  });
-
-  export default async function () {
-    const response = await client.get('https://httpbin.test.k6.io/get');
-    const jsonData = await response.json();
-    console.log(jsonData);
-  }
-  ```
-
-- `advanced-post-streaming.js`:
-  ```javascript
-  import { File } from 'k6/x/file';
-  import { fetch } from 'k6/x/net/http';
-
-  // Will need supporting await in init context
-  const file = await File.open('./logo.svg');  // by default assumes 'read'
-
-  export default async function () {
-    await fetch('https://httpbin.test.k6.io/post', {
-      method: 'POST',
-      body: file.readable,
-    });
-  }
-  ```
-
-
 ## Potential risks
 
 * Long implementation time.

From 1084875a8f67b163f6563cc462a40a03815e51c7 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ivan=20Miri=C4=87?= <ivan.miric@grafana.com>
Date: Wed, 22 Mar 2023 19:01:23 +0100
Subject: [PATCH 03/16] Slightly reword Go net/http section

---
 docs/design/018-new-http-api.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/docs/design/018-new-http-api.md b/docs/design/018-new-http-api.md
index a5c0701da0a..14cf5994d42 100644
--- a/docs/design/018-new-http-api.md
+++ b/docs/design/018-new-http-api.md
@@ -51,9 +51,9 @@ In general, the design of the API should follow these guidelines:
 
   One source of inspiration is the Go `net/http` package, which the k6 team is already familiar with. Based on this, our JS API could have similar entities:
 
-  - `Dialer`: a low-level interface for configuring TCP/IP options, such as TCP timeouts, keep-alive duration, DNS, and IP version preferences.
+  - `Dialer`: a low-level interface for configuring TCP/IP options, such as TCP timeout and keep-alive, TLS settings, DNS resolution, IP version preference, etc.
 
-  - `Transport`: interface for configuring HTTP connection options, such as proxies, TLS, HTTP version preferences, etc.
+  - `Transport`: interface for configuring HTTP connection options, such as proxies, HTTP version preferences, etc.
 
     It enables advanced behaviors like intercepting requests before they're sent to the server.
 

From d2e5e2c6ecb0b763d17ba21a527c0fb883776cba Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ivan=20Miri=C4=87?= <ivan.miric@grafana.com>
Date: Wed, 22 Mar 2023 19:04:19 +0100
Subject: [PATCH 04/16] Add socket done blocking call to TCP example

---
 docs/design/018-new-http-api.md | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/docs/design/018-new-http-api.md b/docs/design/018-new-http-api.md
index 14cf5994d42..50d62335cb0 100644
--- a/docs/design/018-new-http-api.md
+++ b/docs/design/018-new-http-api.md
@@ -109,6 +109,8 @@ export default async function () {
     console.log(`received ${data}`);
     socket.close();
   });
+
+  await socket.done();
 }
 ```
 

From 7c49aef72c6c90b3f90b96115827c9c040e94145 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ivan=20Miri=C4=87?= <ivan.miric@grafana.com>
Date: Wed, 22 Mar 2023 19:02:18 +0100
Subject: [PATCH 05/16] Update IPC example to match the network API

---
 docs/design/018-new-http-api.md | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/docs/design/018-new-http-api.md b/docs/design/018-new-http-api.md
index 50d62335cb0..cd59e67fd02 100644
--- a/docs/design/018-new-http-api.md
+++ b/docs/design/018-new-http-api.md
@@ -128,18 +128,19 @@ export default async function () {
 
 - IPC:
 ```javascript
-import { open } from 'k6/x/file';
+import { dialIPC } from 'k6/x/net;
 import { Client } from 'k6/x/net/http';
 
 export default async function () {
-  const file = await open('/tmp/unix.sock');
+  const socket = await dialIPC('/tmp/unix.sock');
+
+  console.log(socket.file.path); // /tmp/unix.sock
 
   // The HTTP client supports communicating over a Unix socket.
-  // Otherwise it can also be read from and written to directly.
   const client = new Client({
-    socket: file,
+    socket: socket,
   });
-  await client.get('http://127.0.0.1/get');
+  await client.get('http://unix/get');
 }
 ```
 

From 0d8233a09c7968669ec4a99705378c00129dd384 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ivan=20Miri=C4=87?= <ivan.miric@grafana.com>
Date: Wed, 29 Mar 2023 11:37:24 +0200
Subject: [PATCH 06/16] Apply suggestions from Mihail

Co-authored-by: Mihail Stoykov <312246+mstoykov@users.noreply.github.com>
---
 docs/design/018-new-http-api.md | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/docs/design/018-new-http-api.md b/docs/design/018-new-http-api.md
index cd59e67fd02..190f68a5394 100644
--- a/docs/design/018-new-http-api.md
+++ b/docs/design/018-new-http-api.md
@@ -88,7 +88,6 @@ The Socket state can either be _active_—meaning connected for a TCP socket, bo
 - TCP:
 ```javascript
 import { dialTCP } from 'k6/x/net';
-import { Client } from 'k6/x/net/http';
 
 export default async function () {
   const socket = await dialTCP('192.168.1.1:80', {
@@ -117,7 +116,6 @@ export default async function () {
 - UDP:
 ```javascript
 import { dialUDP } from 'k6/x/net';
-import { Client } from 'k6/x/net/http';
 
 export default async function () {
   const socket = new dialUDP('192.168.1.1:9090');
@@ -128,7 +126,7 @@ export default async function () {
 
 - IPC:
 ```javascript
-import { dialIPC } from 'k6/x/net;
+import { dialIPC } from 'k6/x/net';
 import { Client } from 'k6/x/net/http';
 
 export default async function () {
@@ -168,7 +166,7 @@ import { dialTCP } from 'k6/x/net';
 import { Client } from 'k6/x/net/http';
 
 export default async function () {
-  const socket = await dialTCP('10.0.0.10:80, { keepAlive: true });
+  const socket = await dialTCP('10.0.0.10:80', { keepAlive: true });
   const client = new Client({
     socket: socket,
     proxy: 'https://myproxy',
@@ -287,7 +285,7 @@ The [Fetch API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) is a
 The implementation in k6 differs slightly from the web API, but we've tried to make it familiar to use wherever possible.
 
 Example:
-```
+```javascript
 import { fetch } from 'k6/x/net/http';
 
 export default async function () {

From 98841cb464857a6f03c220df47057981c55dd3b1 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ivan=20Miri=C4=87?= <ivan.miric@grafana.com>
Date: Mon, 27 Mar 2023 18:27:46 +0200
Subject: [PATCH 07/16] Add introduction to Design section

---
 docs/design/018-new-http-api.md | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/docs/design/018-new-http-api.md b/docs/design/018-new-http-api.md
index 190f68a5394..f995286c223 100644
--- a/docs/design/018-new-http-api.md
+++ b/docs/design/018-new-http-api.md
@@ -21,7 +21,14 @@ The list of issues with the current API is too long to mention in this document,
 
 ### Design
 
-In general, the design of the API should follow these guidelines:
+HTTP is an application-layer protocol that is built upon lower level transport protocols, such as TCP, UDP, or local IPC mechanisms in most operating systems. It's also closely related to the DNS protocol, which all browsers use when resolving a host name before establishing an HTTP connection. As such, we can't implement a flexible and modern HTTP API without exposing APIs for these lower level protocols. In fact, some user requested functionality would be difficult, if not impossible, without access to these APIs (e.g. issues [#1393](https://github.com/grafana/k6/issues/1393), [#1098](https://github.com/grafana/k6/issues/1098), [#2510](https://github.com/grafana/k6/issues/2510), [#857](https://github.com/grafana/k6/issues/857), [#2366](https://github.com/grafana/k6/issues/2366)).
+
+In this sense, we propose designing the HTTP API in such a way that it's built _on top_ of these lower level APIs. By making our networking namespace composable in this way, we open the door for other application-layer protocols to be implemented using the same low level primitives. For example, WebSockets could be implemented on top of the TCP API, gRPC on top of the HTTP/2 API, and so on.
+
+This approach also follows other modern JavaScript runtimes such as Node and Deno, which ensures we're building a familiar and extensible API, instead of a purpose-built library just for HTTP and unique to k6.
+
+
+With that said, the design of the API should follow these guidelines:
 
 - It should be familiar to users of HTTP APIs from other JS runtimes, and easy for new users to pick up.
 

From d5ffcd28053ec064a97b254c1f28ce368283c00f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ivan=20Miri=C4=87?= <ivan.miric@grafana.com>
Date: Wed, 29 Mar 2023 11:34:19 +0200
Subject: [PATCH 08/16] Reference Streams API issue

---
 docs/design/018-new-http-api.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/design/018-new-http-api.md b/docs/design/018-new-http-api.md
index f995286c223..9df35808ea2 100644
--- a/docs/design/018-new-http-api.md
+++ b/docs/design/018-new-http-api.md
@@ -42,7 +42,7 @@ With that said, the design of the API should follow these guidelines:
   - The [Streams API](https://developer.mozilla.org/en-US/docs/Web/API/Streams_API), a [WHATWG standard](https://streams.spec.whatwg.org/) supported by most modern browsers.
     [Deno's implementation](https://deno.land/manual@v1.30.3/examples/fetch_data#files-and-streams) is a good reference to follow.
 
-    There's a related, but very old [proposal](https://github.com/grafana/k6/issues/592) before the Streams API was standardized, so we shouldn't use it, but it's clear there's community interest in such an API.
+    The work to implement it is tracked in [issue #2978](https://github.com/grafana/k6/issues/2978).
 
     Streaming files both from disk to RAM to the network, and from network to RAM and possibly disk, would also partly solve our [performance and memory issues with loading large files](https://github.com/grafana/k6/issues/2311).
 

From 336abcd11f55e81f9eed659cf68979451cff93f5 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ivan=20Miri=C4=87?= <ivan.miric@grafana.com>
Date: Wed, 29 Mar 2023 11:42:49 +0200
Subject: [PATCH 09/16] Remove mention of replacing k6/http

---
 docs/design/018-new-http-api.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/design/018-new-http-api.md b/docs/design/018-new-http-api.md
index 9df35808ea2..0696311ab2d 100644
--- a/docs/design/018-new-http-api.md
+++ b/docs/design/018-new-http-api.md
@@ -388,7 +388,7 @@ At this point the extension should be relatively featureful and stable to be use
 #### Phase 6: deprecate `k6/http`
 
 As the final phase, we should add deprecation warnings when `k6/http` is used, and point users to the new API.
-Eventually, months down the line, we can consider replacing `k6/http` altogether with the new module.
+We'll have to maintain both `k6/http` and `k6/net/http` for likely years to come, though any new development will happen in `k6/net/http`, and `k6/http` would only receive bug and security fixes.
 
 
 ## Potential risks

From 1e71d7eb8c764ddfa3589d5c2fbf7a63a81a02ac Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ivan=20Miri=C4=87?= <ivan.miric@grafana.com>
Date: Wed, 29 Mar 2023 11:50:45 +0200
Subject: [PATCH 10/16] Suggest merging into k6 core as experimental earlier,
 remove phase 6

Resolves https://github.com/grafana/k6/pull/2971#discussion_r1150460161
---
 docs/design/018-new-http-api.md | 16 +++++++---------
 1 file changed, 7 insertions(+), 9 deletions(-)

diff --git a/docs/design/018-new-http-api.md b/docs/design/018-new-http-api.md
index 0696311ab2d..7837588ce46 100644
--- a/docs/design/018-new-http-api.md
+++ b/docs/design/018-new-http-api.md
@@ -341,13 +341,16 @@ With this in mind, we propose the following phases:
 - We won't yet try to solve performance/memory issues of the current API, or implement major new features like data streaming.
 
 
-#### Phase 2: work on major issues
+#### Phase 2: work on major issues, merge into k6 core as experimental module
 
 **Goals**:
 
 - Work should be started on some of the most impactful issues from the current API.
   Issues like high memory usage when uploading files ([#2311](https://github.com/grafana/k6/issues/2311)), and data streaming ([#592](https://github.com/grafana/k6/issues/592)), are good candidates to focus on first.
 
+- At the end of this phase the API should resolve major limitations of `k6/http`, and it would be a good time to merge it into k6 core as an experimental module (`k6/experimental/net/http`).
+  This would make it available to more users, including in the k6 Cloud.
+
 
 #### Phase 3: work on leftover issues
 
@@ -374,21 +377,16 @@ With this in mind, we propose the following phases:
   Small changes will be inevitable, but there should be no discussion about the overall design.
 
 
-#### Phase 5: merge into k6-core, more testing
+#### Phase 5: more testing, deprecating old API
 
 At this point the extension should be relatively featureful and stable to be useful to all k6 users.
 
 **Goals**:
 
-- Merge the extension into k6 core, and make it available to k6 Cloud users.
-
 - Continue to gather and address feedback from users, thorough testing and polishing.
 
-
-#### Phase 6: deprecate `k6/http`
-
-As the final phase, we should add deprecation warnings when `k6/http` is used, and point users to the new API.
-We'll have to maintain both `k6/http` and `k6/net/http` for likely years to come, though any new development will happen in `k6/net/http`, and `k6/http` would only receive bug and security fixes.
+- As the final step, we should add deprecation warnings when `k6/http` is used, and point users to the new API. We can also consider promoting the API from experimental to a main module under `k6/net/http`.
+  We'll have to maintain both `k6/http` and `k6/net/http` for likely years to come, though any new development will happen in `k6/net/http`, and `k6/http` would only receive bug and security fixes.
 
 
 ## Potential risks

From 5b7e313053ac72747cd0d0f759d0b9ba8770eda7 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ivan=20Miri=C4=87?= <ivan.miric@grafana.com>
Date: Thu, 30 Mar 2023 13:24:58 +0200
Subject: [PATCH 11/16] Add mockup for an Event system

This could be a way to address features like 2667 and 1716.
---
 docs/design/018-new-http-api.md | 84 +++++++++++++++++++++++++++++++++
 1 file changed, 84 insertions(+)

diff --git a/docs/design/018-new-http-api.md b/docs/design/018-new-http-api.md
index 7837588ce46..8a5a86d4ee8 100644
--- a/docs/design/018-new-http-api.md
+++ b/docs/design/018-new-http-api.md
@@ -306,6 +306,90 @@ export default async function () {
 }
 ```
 
+#### Events
+
+The new HTTP API will emit events which scripts can subscribe to, in order to implement advanced functionality.
+
+For example, a `requestToBeSent` event is emitted when the request was processed by k6, and just before it is sent to the server. This allows changing the request body or headers, or introducing artificial delays.
+
+```javascript
+import { sleep } from 'k6';
+import { Client } from 'k6/x/net/http';
+
+export default async function () {
+  const client = new Client();
+  client.on('requestToBeSent', event => {
+    console.log(event.type);    // 'requestToBeSent'
+    const request = event.data;
+    request.headers['Cookie'] = 'somecookie=somevalue;'  // overwrites all previously set cookies
+    request.body += ' world!';  // the final body will be 'Hello world!'
+    sleep(Math.random() * 5);   // random delay up to 5s
+  });
+
+  await client.post('https://httpbin.test.k6.io/post', { body: 'Hello' });
+}
+```
+
+Similarly, a `responseReceived` event is emitted when a response is received from the server, but before it's been fully processed by k6. This can be useful to alter the response in some way, or edit the metrics emitted by k6.
+
+For example:
+
+```javascript
+import { Client } from 'k6/x/net/http';
+
+export default async function () {
+  const client = new Client();
+  let requestID;  // used to correlate a specific response with the request that initiated it
+
+  client.on('requestToBeSent', event => {
+    const request = event.data;
+    if (!requestID && request.url == 'https://httpbin.test.k6.io/get?name=k6'
+        && request.method == 'GET') {
+      // The request ID is a UUIDv4 string that uniquely identifies a single request.
+      // This is a contrived check and example, but you can imagine that in a complex
+      // script there would be many similar requests.
+      requestID = request.id;
+    }
+  });
+
+  client.on('responseReceived', event => {
+    const response = event.data;
+    if (requestID && response.request.id == requestID) {
+      // Change the request duration metric to any value
+      response.metrics['http_req_duration'].value = 3.1415;
+      // Consider the request successful regardless of its response
+      response.metrics['http_req_failed'].value = false;
+      // Or drop a single metric
+      delete response.metrics['http_req_duration'];
+      // Or drop all metrics
+      response.metrics = {};
+    }
+  });
+
+  await client.get('https://httpbin.test.k6.io/get', { query: { name: 'k6' } });
+}
+```
+
+Event handlers can also be attached directly to a single request/response cycle. This avoids having to correlate responses with requests as done above.
+
+```javascript
+import { Client } from 'k6/x/net/http';
+
+export default async function () {
+  const client = new Client();
+  await client.get('https://httpbin.test.k6.io/get', {
+    eventHandlers: {
+      'responseReceived': event => {
+        const response = event.data;
+        // ...
+      }
+    }
+  });
+}
+```
+
+**TODO**: List other possible event types, and their use cases.
+
 
 ### Implementation
 

From 8c97fe07ece1e791e0a4785468bd81e5e372a484 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ivan=20Miri=C4=87?= <ivan.miric@grafana.com>
Date: Fri, 31 Mar 2023 15:02:52 +0200
Subject: [PATCH 12/16] Replace dial* functions with <protocol>.open()

Resolves https://github.com/grafana/k6/pull/2971#discussion_r1153298573
---
 docs/design/018-new-http-api.md | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/docs/design/018-new-http-api.md b/docs/design/018-new-http-api.md
index 8a5a86d4ee8..02960b54d8f 100644
--- a/docs/design/018-new-http-api.md
+++ b/docs/design/018-new-http-api.md
@@ -94,10 +94,10 @@ The Socket state can either be _active_—meaning connected for a TCP socket, bo
 
 - TCP:
 ```javascript
-import { dialTCP } from 'k6/x/net';
+import { TCP } from 'k6/x/net';
 
 export default async function () {
-  const socket = await dialTCP('192.168.1.1:80', {
+  const socket = await TCP.open('192.168.1.1:80', {
                             // default      | possible values
     ipVersion: 0,           // 0            | 4 (IPv4), 6 (IPv6), 0 (both)
     keepAlive: true,        // false        |
@@ -122,10 +122,10 @@ export default async function () {
 
 - UDP:
 ```javascript
-import { dialUDP } from 'k6/x/net';
+import { UDP } from 'k6/x/net';
 
 export default async function () {
-  const socket = new dialUDP('192.168.1.1:9090');
+  const socket = new UDP.open('192.168.1.1:9090');
 
   await socket.write(new TextEncoder().encode('GET / HTTP/1.1\r\n\r\n'));
 }
@@ -133,11 +133,11 @@ export default async function () {
 
 - IPC:
 ```javascript
-import { dialIPC } from 'k6/x/net';
+import { IPC } from 'k6/x/net';
 import { Client } from 'k6/x/net/http';
 
 export default async function () {
-  const socket = await dialIPC('/tmp/unix.sock');
+  const socket = await IPC.open('/tmp/unix.sock');
 
   console.log(socket.file.path); // /tmp/unix.sock
 
@@ -169,11 +169,11 @@ export default async function () {
 
 - Passing a socket with custom transport settings, some HTTP options, and making a POST request:
 ```javascript
-import { dialTCP } from 'k6/x/net';
+import { TCP } from 'k6/x/net';
 import { Client } from 'k6/x/net/http';
 
 export default async function () {
-  const socket = await dialTCP('10.0.0.10:80', { keepAlive: true });
+  const socket = await TCP.open('10.0.0.10:80', { keepAlive: true });
   const client = new Client({
     socket: socket,
     proxy: 'https://myproxy',
@@ -188,11 +188,11 @@ export default async function () {
 
 - A tentative HTTP/3 example:
 ```javascript
-import { dialUDP } from 'k6/x/net';
+import { UDP } from 'k6/x/net';
 import { Client } from 'k6/x/net/http';
 
 export default async function () {
-  const socket = new dialUDP('192.168.1.1:9090');
+  const socket = new UDP.open('192.168.1.1:9090');
 
   const client = new Client({
     socket: socket,
@@ -215,7 +215,7 @@ When connecting to an address using a host name, the resolution can be controlle
 
 For example:
 ```javascript
-import { dialTCP } from 'k6/x/net';
+import { TCP } from 'k6/x/net';
 import dns from 'k6/x/net/dns';
 
 const hosts = {
@@ -224,7 +224,7 @@ const hosts = {
 };
 
 export default async function () {
-  const socket = await dialTCP('myhost', {
+  const socket = await TCP.open('myhost', {
     lookup: async hostname => {
       // Return either the IP from the static map, or do an OS lookup,
       // or fallback to making a DNS query to specific servers.

From 5582425362197b48872941924d1f8d772e6226e6 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ivan=20Miri=C4=87?= <ivan.miric@grafana.com>
Date: Thu, 13 Apr 2023 15:15:26 +0200
Subject: [PATCH 13/16] Remove sleep from example

---
 docs/design/018-new-http-api.md | 2 --
 1 file changed, 2 deletions(-)

diff --git a/docs/design/018-new-http-api.md b/docs/design/018-new-http-api.md
index 02960b54d8f..5fe3824f0d3 100644
--- a/docs/design/018-new-http-api.md
+++ b/docs/design/018-new-http-api.md
@@ -313,7 +313,6 @@ The new HTTP API will emit events which scripts can subscribe to, in order to im
 For example, a `requestToBeSent` event is emitted when the request was processed by k6, and just before it is sent to the server. This allows changing the request body or headers, or introducing artificial delays.
 
 ```javascript
-import { sleep } from 'k6';
 import { Client } from 'k6/x/net/http';
 
 export default async function () {
@@ -323,7 +322,6 @@ export default async function () {
     const request = event.data;
     request.headers['Cookie'] = 'somecookie=somevalue;'  // overwrites all previously set cookies
     request.body += ' world!';  // the final body will be 'Hello world!'
-    sleep(Math.random() * 5);   // random delay up to 5s
   });
 
   await client.post('https://httpbin.test.k6.io/post', { body: 'Hello' });

From ba3383ed1399133c3165ebf2168173e1bff624c2 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ivan=20Miri=C4=87?= <ivan.miric@grafana.com>
Date: Thu, 13 Apr 2023 17:14:30 +0200
Subject: [PATCH 14/16] Add note about the WIP state of the proposal

---
 docs/design/018-new-http-api.md | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/docs/design/018-new-http-api.md b/docs/design/018-new-http-api.md
index 5fe3824f0d3..23aab4e2383 100644
--- a/docs/design/018-new-http-api.md
+++ b/docs/design/018-new-http-api.md
@@ -393,6 +393,8 @@ export default async function () {
 
 Trying to solve all `new-http` issues with a single large and glorious change wouldn't be reasonable, so improvements will undoubtedly need to be done gradually, in several phases, and over several k6 development cycles.
 
+Note that the implementation process described below is not finalized, and will go through several changes during development.
+
 With this in mind, we propose the following phases:
 
 #### Phase 1: create initial k6 extension

From 81034aab4bac5b6378da8363e328ddd474e6e5cd Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ivan=20Miri=C4=87?= <ivan.miric@grafana.com>
Date: Thu, 13 Apr 2023 17:15:41 +0200
Subject: [PATCH 15/16] Rework phase 1 section, prioritize #761

Addresses https://github.com/grafana/k6/pull/2971#discussion_r1165538223
---
 docs/design/018-new-http-api.md | 16 +++++-----------
 1 file changed, 5 insertions(+), 11 deletions(-)

diff --git a/docs/design/018-new-http-api.md b/docs/design/018-new-http-api.md
index 23aab4e2383..6de4a21db28 100644
--- a/docs/design/018-new-http-api.md
+++ b/docs/design/018-new-http-api.md
@@ -402,22 +402,16 @@ With this in mind, we propose the following phases:
 **Goals**:
 
 - Implement a barebones async API that serves as a proof-of-concept for what the final developer experience will look and feel like.
-  The code should be in a state that allows it to be easily extended.
 
-  By barebones, we mean:
+  By barebones, we mean that there must be a `Client` interface with only one method: `request()`, which will work similarly to the current `http.asyncRequest()`. Only `GET` and `POST` methods must be supported.
 
-  - The `Client` interface with only one method: `request()`, which will work similarly to the current `http.asyncRequest()`.
+  The code must be in a state that allows it to be easily extended. Take into account the other design goals of this document, even if they're not ready to be implemented.
 
-    For the initial PoC, it's fine if only `GET` and `POST` methods are supported.
+- This initial API must solve at least one minor, but concrete, issue of the current API. It should fix something that's currently not possible and doesn't have a good workaround.
 
-    It's not required to make `Dialer` and `Transport` fully configurable at this point, but they should use sane defaults, and it should be clear how the configuration will be done.
+  Addressing [#761](https://github.com/grafana/k6/issues/761) would be a good first step.
 
-- This initial API should solve a minor, but concrete, issue of current API. It should fix something that's currently not possible and doesn't have a good workaround.
-
-  Good candidates: [#936](https://github.com/grafana/k6/issues/936), [#970](https://github.com/grafana/k6/issues/970).
-
-- Features like configuring options globally, or per VU or request, should be implemented.
-  Deprecating the `httpx` library should be possible after this phase.
+  As an optional stretch goal, once we settle on the API to configure the transport layer, [#936](https://github.com/grafana/k6/issues/936) and [#970](https://github.com/grafana/k6/issues/970) are good issues to tackle next.
 
 
 **Non-goals**:

From 0fde8246855b6504f778643789e04f04573464cd Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ivan=20Miri=C4=87?= <ivan.miric@grafana.com>
Date: Fri, 14 Apr 2023 12:12:45 +0200
Subject: [PATCH 16/16] Add TLS and HTTP/2 examples, remove HTTP/3 example

This changes the way the HTTP client obtains the socket, following this
discussion[1]. Instead of passing one socket, a dial function can be set
to control how the client creates the socket.

The HTTP/3 example was removed since it's too early to determine that API.

[1]: https://github.com/grafana/k6/pull/2971#discussion_r1161729636
---
 docs/design/018-new-http-api.md | 62 +++++++++++++++++++++++----------
 1 file changed, 43 insertions(+), 19 deletions(-)

diff --git a/docs/design/018-new-http-api.md b/docs/design/018-new-http-api.md
index 6de4a21db28..ca7a2c8f2f6 100644
--- a/docs/design/018-new-http-api.md
+++ b/docs/design/018-new-http-api.md
@@ -111,7 +111,7 @@ export default async function () {
   await socket.write(new TextEncoder().encode('GET / HTTP/1.1\r\n\r\n'));
 
   // And reading...
-  socket.on('data', (data) => {
+  socket.on('data', data => {
     console.log(`received ${data}`);
     socket.close();
   });
@@ -125,9 +125,9 @@ export default async function () {
 import { UDP } from 'k6/x/net';
 
 export default async function () {
-  const socket = new UDP.open('192.168.1.1:9090');
-
+  const socket = await UDP.open('192.168.1.1:9090');
   await socket.write(new TextEncoder().encode('GET / HTTP/1.1\r\n\r\n'));
+  socket.close();
 }
 ```
 
@@ -137,15 +137,17 @@ import { IPC } from 'k6/x/net';
 import { Client } from 'k6/x/net/http';
 
 export default async function () {
-  const socket = await IPC.open('/tmp/unix.sock');
-
-  console.log(socket.file.path); // /tmp/unix.sock
-
   // The HTTP client supports communicating over a Unix socket.
   const client = new Client({
-    socket: socket,
+    dial: async () => {
+      return await IPC.open('/tmp/unix.sock');
+    },
   });
   await client.get('http://unix/get');
+
+  console.log(client.socket.file.path); // /tmp/unix.sock
+
+  client.socket.close();
 }
 ```
 
@@ -167,17 +169,17 @@ export default async function () {
 }
 ```
 
-- Passing a socket with custom transport settings, some HTTP options, and making a POST request:
+- Creating a client with custom transport settings, some HTTP options, and making a POST request:
 ```javascript
 import { TCP } from 'k6/x/net';
 import { Client } from 'k6/x/net/http';
 
 export default async function () {
-  const socket = await TCP.open('10.0.0.10:80', { keepAlive: true });
   const client = new Client({
-    socket: socket,
+    dial: async address => {
+      return await TCP.open(address, { keepAlive: true });
+    },
     proxy: 'https://myproxy',
-    version: 1.1,                     // force a specific HTTP version
     headers: { 'User-Agent': 'k6' },  // set some global headers
   });
   await client.post('http://10.0.0.10/post', {
@@ -186,22 +188,44 @@ export default async function () {
 }
 ```
 
-- A tentative HTTP/3 example:
+- Configuring TLS with a custom CA certificate and forcing HTTP/2:
 ```javascript
-import { UDP } from 'k6/x/net';
+import { TCP } from 'k6/x/net';
 import { Client } from 'k6/x/net/http';
+import { open } from 'k6/x/file';
 
-export default async function () {
-  const socket = new UDP.open('192.168.1.1:9090');
+const caCert = await open('./custom_cacert.pem');
 
+export default async function () {
   const client = new Client({
-    socket: socket,
-    version: 3,      // A UDP socket would imply HTTP/3, but this makes it explicit.
+    dial: async address => {
+      return await TCP.open(address, {
+        tls: {
+          alpn: ['h2'],
+          caCerts: [caCert],
+        }
+      });
+    },
   });
-  await client.get('https://httpbin.test.k6.io/get');
+  await client.get('https://10.0.0.10/');
 }
 ```
 
+- Forcing unencrypted HTTP/2 (h2c):
+```javascript
+import { TCP } from 'k6/x/net';
+import { Client } from 'k6/x/net/http';
+
+export default async function () {
+  const client = new Client({
+    dial: async address => {
+      return await TCP.open(address, { tls: false });
+    },
+    version: [2],
+  });
+  await client.get('http://10.0.0.10/');
+```
+
 
 #### Host name resolution