API

The Knowledge Base API lets you access the information that is available on the database. See Postgres for more information on how information is structured and stored.

You can use the API to obtain information about any element present in an API definition. You can also use the API to interrogate the system and obtain answers. A list of the questions we're currently working on is available at the Knowledge Base repo.

Design

This section outlines the Design of the Knowledge Base API. The API Design follows the Strategy/Definition/Validation/Specification process.

flowchart LR
  style Ideation fill:#fdf2d1,stroke:#e8833a,stroke-width:2px,stroke-dasharray: 5 5
  style Strategy fill:#e8833a,color:#fff
  style Definition fill:#ac6363,color:#fff
  style v fill:#ac6363,color:#fff
  style Specification fill:#207868,color:#fff
  style Implementation fill:#cfeeeb,stroke:#207868,stroke-width:2px,stroke-dasharray: 5 5
  Ideation --> Strategy --> Definition --> v{Validation} --> Specification --> Implementation
  v -- not validated --> Definition

Strategy

Interviews

API Designer, on January 30, 2023

This is a summary of the interview:

One of the tasks of an API Designer is to do reviews.
API Design reviews start by understanding context.
Context determines what API Design patterns matter and should be followed.
Understanding if an API follows a generic REST style is important.
Being able to know what elements aren't generic is important. An example is the pagination style.
Knowing what other similar APIs are doing leads to better API Design reviews.
Having valid references helps drive API Design review justification.
A Knowledge Base would enable comparison analysis with APIs from other businesses in the same sector.

Researcher, on January 31, 2023

This is a summary of the interview:

The researcher produces statistics related to large amounts of API-related data.
It's important to know the answers to questions related to API Design and common practices.
Comparing APIs to find similarities between Designs is important.
Looking at particular API definitions is interesting.
The list of used tools include spreadsheets, e.g., Excel, Google Sheets, or Airtable, to retrieve and render the data from the Knowledge Base.
Postman and d3js are also possible ways of retrieving and rendering data.
The retrieved data might be post-processed if needed.
Results could be presented live on Web sites with high traffic.
Being able to execute the same queries over time and compare results is important.
Knowing the trust level of the data is important.

Technical Product Manager, on February 2, 2023

This is a summary of the interview:

The Technical Product Manager makes data-driven decisions to understand what API standards to support in their product.
Questions like "what percentage of APIs use a specific standard," and "how are standards and their versions used across different company segments" are important.
Getting information about trends in the usage of API standards helps drive decisions on what features to build into the product.
The list of used tools include Mixpanel, Segment, and Google Sheets. The goal is to be able to query and analyze data.
The ability to ask direct open questions is interesting. However, knowing what questions other people are asking is also relevant.

Software Architect, on February 8, 2023

This is a summary of the interview:

The Architect's work involves migrating APIs between infrastructure technologies, standards, and organizations.
Being able to reuse APIs is an important challenge.
Understanding where there is an overlap of features or duplication of APIs is a need.
There's a big challenge in understanding the context behind the creation of each API.
Things like patterns and security-related features would be interesting things to know more about.
The list of tools used includes Postman and ABACUS.
The Knowledge Base could provide information to help create learning material.
Having information about API usage would be interesting. Knowing what APIs are "successful" would be helpful.

User Personas

This is a list of the Knowledge Base API personas. The goal of this list is to provide information on how to design the API, so it fulfills the use cases that users care about. By better understanding who the API potential users are, you'll be able to design the API around their specific use cases. The individual personas are built from an API-centric perspective and are to be refined by taking into account more accurate information retrieved from user interviews.

All pictures and names have been generated, and do not represent real people. Pictures have been generated by thispersondoesnotexist, and names have been generated by the fake name generator.

Researcher

Persona name: Marianne Talbert
Persona role: Technical Researcher
Archetype: Explainer

Jobs to be done

Produce statistics related to large amounts of API-related data.
Find answers to questions related to API Design and standard practices.
Find similarities between API Designs.
Enrich statistics by looking at particular API definitions.
Retrieve particular elements of several or a single OpenAPI document.
Present summarized API-related statistics and findings.
Understand how API-related statistical information evolves over time.

Benefits

Obtaining quick access to vast amounts of API-related data.
Getting accurate, up-to-date, and reliable API-related information.
Comparing an API Design with the practice across verticals.
Retrieving reports of API-related statistics and insights.

Behaviors

Populate a spreadsheet with selected attributes of API-related data.
Share a report of API-related statistics.
Get the full definition of one or more APIs.
Get the similarities between two or more API definitions.
Obtain an API definition that is representative of a vertical.

Tools and services used

Spreadsheets, e.g., MS Excel or Google Sheets
The R programming language
VS Code
Postman Visualizer using d3js
HTML and JavaScript to render the data
Google Slides

Challenges

Share technical data with business stakeholders.
Prove hypothesis to business stakeholders based on existing technical data.

API Designer

Persona name: Franklin Gregory
Persona role: Staff API Designer
Archetype: Designer

Jobs to be done

Verify if an API follows a REST style standard across all available APIs.
Search for APIs similar to the one being designed or reviewed.
Understand the context around an API Design based on statistical data.
Incorporate industry best practices into API Design.
Justify the reasons behind API Design decisions based on statistical data.

Benefits

Comparing an API Design with the practice across verticals.
Obtaining reusable API Design patterns.
Retrieving reports of API-related statistics and insights.
Validating API Design options based on accurate data.

Behaviors

Identify and extract API Design patterns.
Share a report of API-related statistics.
Get the similarities between two or more API definitions.
Obtain an API definition that is representative of a vertical.

Tools and services used

VS Code
Postman
Confluence

Challenges

Understand the context behind the creation of an API.
Validate API Design solutions with potential users and stakeholders.
Identify API Design industry best practices.

Product Manager

Persona name: Steven Kelly
Persona role: Technical Product Manager
Archetype: Builder

Jobs to be done

Make data-driven decisions to understand what API standards to support in a product.
Obtain knowledge about API-related trends.
Get information about specific features of API standards.
Understand under which scenarios APIs use specific features.
Know which industries use specific API standards, versions, and features.
Obtain answers to API-related questions asked in a human-friendly way.
Understand how API-related statistical information evolves over time.

Benefits

Obtaining API feature and version usage trends across verticals.
Retrieving reports of API-related statistics and insights.
Backing up product decisions with statistical API information.

Behaviors

Share a report of API-related statistics.

Tools and services used

Aha
Jira
Confluence
Web browser
ChatGPT

Challenges

Validate API Design solutions with potential users and stakeholders.
Decide what API standards and common practices to support in their product.

Technical Architect

Persona name: Thelma Sorenson
Persona role: Software Architect
Archetype: Builder and Explainer

Jobs to be done

Ask questions related to certain features of APIs, e.g., pagination.
Understand the use of API patterns and security-related features.
Understand the context around the Design of an API.
Create learning material from existing API-related statistics.
Obtain usage information of existing APIs.
Obtain answers to API-related questions asked in a human-friendly way.

Benefits

Obtaining API feature information across all existing APIs.
Comparing an API Design with the practice across verticals.
Retrieving reports of API-related statistics and insights.

Behaviors

Share a report of API-related statistics.
Get the similarities between two or more API definitions.
Obtain an API definition that is representative of a vertical.

Tools and services used

Postman
ABACUS
Confluence
ChatGPT

Challenges

Reuse existing APIs.
Know what APIs exist in the organization.
Understand the context behind the creation of an API.

Definition

Architectural style

The definition of the API architectural style is obtained by analyzing the support of the tools used by the personas. The following matrix outlines how each of the identified tools supports different architectural styles with a minimum of customization. Situations that require a specific custom solution are identified.

	REST (GET)	REST (POST)	GraphQL	gRPC	SOAP
MS Excel	✔︎	VBA	add-on		✔︎
Google Sheets	✔︎	add-on	add-on
R	✔︎	✔︎	✔︎	✔︎	✔︎
Postman	✔︎	✔︎	✔︎	✔︎
JavaScript	✔︎	✔︎	✔︎	✔︎	✔︎
Jira	add-on	add-on	add-on
Aha	Zapier	Zapier
Web Browser	✔︎	HTML forms	GraphiQL
URL	✔︎		limited

It's clear that the architectural style that better adapts to all the tools is REST (GET). GraphQL can also be a possible choice. However, sharing API requests via a URL has a length limitation. REST (POST) also seems like a good candidate. However, working with it on a Web Browser requires custom development.

Resources

API Definition

Personas

Researcher
API Designer
Technical Architect

Behaviors

Get the full definition of one or more APIs.
Get the similarities between two or more API definitions.
Obtain an API definition that is representative of a vertical.

JTBD

Find similarities between API Designs.
Enrich statistics by looking at particular API definitions.
Retrieve particular elements of several or a single OpenAPI document.
Verify if an API follows a REST style standard across all available APIs.
Search for APIs similar to the one being designed or reviewed.
Understand the context around the Design of an API.

Benefits

Getting accurate, up-to-date, and reliable API-related information.
Comparing an API Design with the practice across verticals.

Statistic

Personas

Researcher
API Designer
Product Manager
Technical Architect

Behaviors

Share a report of API-related statistics.

JTBD

Present summarized API-related statistics and findings.
Understand how API-related statistical information evolves over time.
Understand the context around an API Design based on statistical data.
Incorporate industry best practices into API Design.
Justify the reasons behind API Design decisions based on statistical data.
Make data-driven decisions to understand what API standards to support in a product.
Obtain knowledge about API-related trends.
Understand how API-related statistical information evolves over time.
Understand the use of API patterns and security-related features.
Create learning material from existing API-related statistics.

Benefits

Retrieving reports of API-related statistics and insights.
Backing up product decisions with statistical API information.

Capabilities

This section describes the capabilities for each one of the endpoints.

API Definition

Return the OpenAPI documents for a list of APIs
Return the OpenAPI document of a given API
Return a JSON path of an OpenAPI document of a given API
Calculate the similarities between two or more OpenAPI documents
Return an OpenAPI document that represents a list of existing APIs

Statistic

Return a list of the available statistics
Return a global report of API-related statistical information
Return a report of API-related statistical information filtered by type of statistic
Return a report of API-related statistical information filtered by sector

For each type of report:

Identify the date and time of the report
Return an HTML version of the report available on a canonical URL
Return a PDF version of the report

Operations

Capability: Return a list of the available types of statistical reports
Path: /reports
Method: GET
Example parameters and body: None
Example return body:

[
  {
    "name": "Unassigned HTTP status code",
    "type": "postman:SpectralRule:http-status-codes:unassigned:v1"
  },
  {
    "name": "Unused or obsolete HTTP status code",
    "type": "postman:SpectralRule:http-status-codes:unused:v1"
  }
]

Capability: Return a global report of API-related statistical information
Path: /statistics
Method: GET
Example parameters and body:
- sector
  - HTTP query parameter
  - The sector to filter the APIs by
Example return body:

{
  "info": {
    "name": "Global API Knowledge Base statistical report",
    "type": "postman:knowledge-base:global",
    "creationDateTime": "2023-03-07T12:17:29+01:00"
  },
  "data": [
    {
      "info": {
        "name": "APIs using unassigned HTTP status codes",
        "type": "postman:SpectralRule:http-status-codes:unassigned:v1",
        "creationDateTime": "2023-02-28T15:41:16+00:00"
      },
      "data": {
        "total": 22756,
        "count": 36
      }
    },
    {
      "info": {
        "name": "APIs specifying license information",
        "type": "info-license",
        "creationDateTime": "2023-03-07T12:16:08+01:00"
      },
      "data": {
        "total": 22756,
        "count": 7428
      }
    }
  ]
}

Capability: Return a statistical report filtered by type
Path: /statistics/{type}
Method: GET
Example parameters and body:
- type
  - Path parameter
  - The type of statistical report to retrieve
- sector
  - HTTP query parameter
  - The sector to filter the APIs by
Example return body:

{
  "info": {
    "name": "APIs using unassigned HTTP status codes",
    "type": "postman:SpectralRule:http-status-codes:unassigned:v1",
    "creationDateTime": "2023-02-28T15:41:16+00:00"
  },
  "data":
  {
    "total": 22756,
    "count": 36
  }
}

Validation

The validation of the designed API capabilities was done with all the internal stakeholders. We have decided to implement the Statistics capability, postponing the implementation of the API Definition capability. Since the stakeholders are interested in obtaining statistical reports for now, investing resources in implementing the "API Definition" capability doesn't make sense.

So, the focus was on validating the response format of the Statistics capability.

High-level validation

A high-level validation was performed involving the internal stakeholders. The goal was to learn from the stakeholders what response formats could be appropriate for the Statistic capability, reach a consensus, and define a response format. The decisions were based on the following assumptions:

Existing options for response formats are specific and generic.
At this point, we aim to have a prototype of the API running.
There are generic formats dedicated to statistical data, such as SDMX and JSONstat.
Implementing and using a generic format takes longer than using a specific format.
Most of our potential user base is not versed in statistical formats.
The statistical data is produced from the results of Spectral rules.

The decision is to implement a specific format such as the one described in the example responses of the Statistic capability. Such a format is easy to define, document, understand, and use.

Low-level validation

The formats that were validated are the ones illustrated by the examples of the GET /statistics and GET /statistics/{type} operations. The response of GET /statistics includes a list of statistical reports described in the GET /statistics/{type} response. Let's review what a single statistical report would look like with an example:

{
  "info": {
    "name": "APIs using unassigned HTTP status codes",
    "type": "postman:SpectralRule:http-status-codes:unassigned:v1",
    "creationDateTime": "2023-02-28T15:41:16+00:00"
  },
  "data":
  {
    "total": 22756,
    "count": 36
  }
}

There's an info section describing the type of information and a data section returning the statistical information. This response format was well accepted by the stakeholders. Each statistical result comprises a total property that represents the total number of APIs analyzed and a count property that represents the number of APIs that passed the Spectral rule. For example, for a Spectral rule that checks if an API uses unassigned HTTP status codes, the count attribute returns the number of APIs that use unassigned HTTP status codes.

This response format can be consumed without complexity from the tools the user personas identified earlier in the Design process. Let's see how potential users would consume an individual statistical report:

Using curl and a jq from the command line:

curl -s <BASE_URL>/statistics/postman:SpectralRule:http-status-codes:unassigned:v1 | jq '(.data.count / .data.total) * 100'`

Using R (Markdown):

```{r, echo=FALSE}

library(jsonlite)

report <- fromJSON("<BASE_URL>/statistics/postman:SpectralRule:http-status-codes:unassigned:v1")

percent <- round(report$data$count / report$data$total * 100, 2)
```

# `r report$info$name`  

**Generated on `r report$info$creationDateTime`**

`r percent`%

Using Postman:

if (pm.response.code == 200) {
  const response = pm.response.json();
  const template = `
  <h1>{{info.name}}</h1>
  <p>Generated on {{info.creationDateTime}}.</p>
  <table style="padding:5px">
    <tr>
      <td style="padding:15px;text-align:right">{{data.percent}}%</td>
    </tr>
  </table>
  `;
  response.info.creationDateTime = new Date(response.info.creationDateTime).toString();
  response.data.percent = ((response.data.count / response.data.total) * 100).toFixed(2);
  pm.visualizer.set(template, response);
}

Using Google Sheets (with importJSON):

=ImportJSON("<BASE_URL>/statistics/postman:SpectralRule:http-status-codes:unassigned:v1")

Using JavaScript:

const requestOptions = {
  method: 'GET',
  redirect: 'follow'
};

fetch("<BASE_URL>/statistics/postman:SpectralRule:http-status-codes:unassigned:v1", requestOptions)
  .then(response => response.json())
  .then(result => {
    console.log(result.info.name);
    const percent = (result.data.count / result.data.total * 100).toFixed(2);
    console.log(`${percent}%`);
  })
  .catch(error => console.log('error', error));

The same approach applies to the response of the GET /statistics operation, which returns a list of statistical reports. It's fair to say from the exercise above that the response format is adequate for most of the tools mentioned before. One thing to note, however, is that we'll have to offer the possibility of consuming the report in HTML, PDF, and possibly CSV formats. The possibility of consuming the report in a pre-rendered fashion makes it even easier for anyone to share statistical information.

API

Design

Strategy

Interviews

API Designer, on January 30, 2023

Researcher, on January 31, 2023

Technical Product Manager, on February 2, 2023

Software Architect, on February 8, 2023

User Personas

Researcher

Jobs to be done

Benefits

Behaviors

Tools and services used

Challenges

API Designer

Jobs to be done

Benefits

Behaviors

Tools and services used

Challenges

Product Manager

Jobs to be done

Benefits

Behaviors

Tools and services used

Challenges

Technical Architect

Jobs to be done

Benefits

Behaviors

Tools and services used

Challenges

Definition

Architectural style

Resources

API Definition

Personas

Behaviors

JTBD

Benefits

Statistic

Personas

Behaviors

JTBD

Benefits

Capabilities

API Definition

Statistic

Operations

Validation

High-level validation

Low-level validation

Clone this wiki locally