English__Data Sources__Fetched

Fetched Data Source

wj-config provides the Fetch data source that allows fetching of configuration data using the HTTP protocol (by using the fetch() function).

How To Use the Fetched Data Source

As with all other data sources, there is a specialized addFetched() method in the configuration builder that assists with this job. This function defines four parameters in the form of 2 overloads:

addFetched(url: URL | (() => URL), required: boolean = true, init?: RequestInit, processFn?: ProcessFetchResponse): IBuilder;
addFetched(request: RequestInfo | (() => RequestInfo), required: boolean = true, init?: RequestInit, processFn?: ProcessFetchResponse): IBuilder;

1. The URL, a RequestInfo object, or a function that returns either, pointing to where the configuration data resides.
2. An optional Boolean value that indicates if the data source is considered mandatory or required.
3. An optional RequestInit object that is passed to the fetch() function when called.
4. An optional asynchronous processing function that is responsible for returning the configuration data object.

The first parameter, as with other data sources, can be a function. This function approach is the recommended approach when doing any form of conditional configuration if obtaining the URL or RequestInfo object will incur in potentially-wasted CPU cycles.

As seen in the above list, two of the parameters (first and third) are directly related to the use of fetch() and are not explained here. If you would like to know more about them, feel free to learn about the fetch() function online.

The second parameter is used as a quick way to say "if fetching this data fails, don't worry about it"; the fourth one is an opportunity for the developer to provide custom code that handles the HTTP response obtained by the fetch() function.

Let's see in detail about the second and fourth parameters.

The Required Parameter

Fetching data may or may not succeed, and your application may or may not operate properly if said data cannot be obtained. The job of this parameter is to tell the data source which of the cases apply to the project.

Fetching is a two-part operation: Connect to the HTTP server and make the request, and then read the response.

For the first part to succeed, the specified host must be accessible, known and must accept the specified scheme.

For the second part to succeed, the HTTP server must send a response of some kind back.

The first part is not protected. If the host name is unknown, an error will occur; if the network goes down or the server is down, an error will occur. This will stop your code regardless of the value of the required parameter. This parameter only protects for the second part of the fetching operation.

The second part, as stated, is "protected" by the required parameter. Once the HTTP server sends a response, the response data is extracted and parsed into an object, which will be the object that will contribute to the final configuration object. If, however, an error occurs during parsing, or the server responds with HTTP status code 204 or a non-200 status code, the value of required is consulted:

• If required is true, then an error is thrown.
• If required is false, then the errors are ignored and a blank object (an object with no properties) is returned as the configuration data object.

The Processing Function Parameter

By default, the Fetched data source attempts to parse the HTTP response's body as if it were JSON data using the json() method in the response object. But what if the response is not JSON? What if we need the JSON5 parser instead?

Cases like the ones just mentioned are meant to be covered by providing custom parsing logic in the form of a function.

You can provide a function that receives the HTTP response object as first parameter, and it is the function's job to return the object that will contribute to the final configuration object.

For example, this code snippet reads the HTTP response as text and uses the JSON5 parser (that allows trailing commas, comments, etc.) to obtain the desired object:

const config = await wjConfig()
    .addFetched('https://my.example.com/config?json', true, undefined, async (r) => {
        const data = await r.text();
        return JSON5.parse(data);
    })
    ...
    .build();

This functionality enables the use of any data format as long as you are willing to provide the parsing algorithm.

Data Requirements

This data source is a helper for the fetch() funciton, and all of the data requirements of said function apply, but there are no extra requirements or restrictions to be noted here.