Skip to content

Commit

Permalink
chore(src): merge 'refactor-to-pure-functions' into refactor-to-pure-…
Browse files Browse the repository at this point in the history
…functions
  • Loading branch information
narekhovhannisyan committed Feb 28, 2024
2 parents c85c285 + b052d49 commit 9a3ebcb
Show file tree
Hide file tree
Showing 11 changed files with 148 additions and 123 deletions.
6 changes: 4 additions & 2 deletions src/__tests__/unit/lib/co2js/index.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ describe('lib/co2js: ', () => {
]);
});

it('returns a result when provided `options` in the input.', async () => {
it('returns a result when provided `options` in the global config.', async () => {
const config = {type: 'swd', 'green-web-host': false};
const output = Co2js({
options: {
Expand Down Expand Up @@ -161,7 +161,9 @@ describe('lib/co2js: ', () => {
await output.execute(inputs, config);
} catch (error) {
expect(error).toEqual(
new InputValidationError('Co2js: Bytes not provided.')
new InputValidationError(
'Either `network/data/bytes` or `network/data` should be provided in the input.'
)
);
expect(error).toBeInstanceOf(InputValidationError);
}
Expand Down
2 changes: 1 addition & 1 deletion src/lib/azure-importer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ You can create one using [portal.azure.com](https://portal.azure.com). You also

The Azure Importer uses [AzureDefaultCredentials](https://learn.microsoft.com/en-us/dotnet/api/azure.identity.defaultazurecredential?view=azure-dotnet) method which is an abstraction for different scenarios of authentication.

- When hosting the IEF Azure Importer on an Azure service, you can provide a [managed identity](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview).
- When hosting the IF Azure Importer on an Azure service, you can provide a [managed identity](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview).
- When running the Azure Importer outside of Azure, e.g. on your local machine, you can use an [App registration](https://learn.microsoft.com/en-us/entra/identity-platform/quickstart-register-app) (an App registration is a representation of a technical service principal account; you can view it as an identity for your App on Azure).

The following steps in this tutorial use a service principal. You can learn more at https://learn.microsoft.com/en-us/entra/identity-platform/quickstart-register-app
Expand Down
6 changes: 3 additions & 3 deletions src/lib/boavizta/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@
> [!NOTE]
> Boavizta is a community plugin, not part of the IF standard library. This means the IF core team are not closely monitoring these plugins to keep them up to date. You should do your own research before implementing them!
[Boavizta](https://boavizta.org/) is an environmental impact calculator that exposes an API we use in IEF to retrieve energy and embodied carbon estimates.
[Boavizta](https://boavizta.org/) is an environmental impact calculator that exposes an API we use in IF to retrieve energy and embodied carbon estimates.

## Implementation

Boavizta exposes a [REST API](https://doc.api.boavizta.org/). If the `boavizta` plugin is included in an IEF pipeline, IEF sends API requests to Boavizta. The request payload is generated from input data provided to IEF in an `manifest` file.
Boavizta exposes a [REST API](https://doc.api.boavizta.org/). If the `boavizta` plugin is included in an IF pipeline, IF sends API requests to Boavizta. The request payload is generated from input data provided to IF in an `manifest` file.

## Parameters

Expand Down Expand Up @@ -77,7 +77,7 @@ const usage = await output.calculate([

## Example `manifest`

In IEF plugins are expected to be invoked from an `manifest` file. This is a yaml containing the plugin configuration and inputs. The following `manifest` initializes and runs the `boavizta-cpu` plugin:
In IF plugins are expected to be invoked from an `manifest` file. This is a yaml containing the plugin configuration and inputs. The following `manifest` initializes and runs the `boavizta-cpu` plugin:

```yaml
name: boavizta-demo
Expand Down
10 changes: 5 additions & 5 deletions src/lib/ccf/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Cloud Carbon Footprint

> [!NOTE] > `CCF` is a community plugin, not part of the IF standard library. This means the IF core team are not closely monitoring these plugins to keep them up to date. You should do your own research before manifestementing them!
> [!NOTE] > `CCF` is a community plugin, not part of the IF standard library. This means the IF core team are not closely monitoring these plugins to keep them up to date. You should do your own research before implementing them!
"Cloud Carbon Footprint is an open source tool that provides visibility and tooling to measure, monitor and reduce your cloud carbon emissions. We use best practice methodologies to convert cloud utilization into estimated energy usage and carbon emissions, producing metrics and carbon savings estimates that can be shared with employees, investors, and other stakeholders." - [CCF](https://www.cloudcarbonfootprint.org/)

Expand All @@ -22,9 +22,9 @@
- `carbon-embodied`: carbon emitted in manufacturing the device, in gCO2eq
- `energy`: energy used by CPU in kWh

## IEF manifestementation
## IF Implementation

IEF remanifestements the Cloud Carbon Footprint methodology from scratch conforming to the IEF specification. This means the CCF plugins can be run inside IEF without any external API calls and can be invoked as part of a plugin pipeline defined in an `manifest`.
IF reimplements the Cloud Carbon Footprint methodology from scratch conforming to the IF specification. This means the CCF plugins can be run inside IF without any external API calls and can be invoked as part of a plugin pipeline defined in an `manifest`.

Cloud Carbon Footprint includes calculations for three cloud vendors: AWS, Azure and GCP.

Expand All @@ -40,11 +40,11 @@ And:

`Embodied Emissions = estimated metric tons CO2e emissions from the manufacturing of datacenter servers, for compute usage`

You can read a detailed explanation ofn the calculations in the [CCF docs](https://www.cloudcarbonfootprint.org/docs/methodology/) and see the code for our manifestementation in [this repository](../../src/lib/ccf/).
You can read a detailed explanation ofn the calculations in the [CCF docs](https://www.cloudcarbonfootprint.org/docs/methodology/) and see the code for our implementing in [this repository](../../src/lib/ccf/).

## Usage

In IEF, the plugin is called from a `manifest`. A `manifest` is a `.yaml` file that contains configuration metadata and usage inputs. This is interpreted by the command line tool, `if`. The plugin input is expected to contain `duration`,`cpu/utilization`, `cloud/vendor` and `cloud/instance-type` fields.
In IF, the plugin is called from a `manifest`. A `manifest` is a `.yaml` file that contains configuration metadata and usage inputs. This is interpreted by the command line tool, `if`. The plugin input is expected to contain `duration`,`cpu/utilization`, `cloud/vendor` and `cloud/instance-type` fields.

You can see example Typescript invocations for each `cloud/vendor` below:

Expand Down
24 changes: 12 additions & 12 deletions src/lib/co2js/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@
- `dataReloadRatio` - a value between 0 and 1 representing the percentage of data that is downloaded by return visitors. -`firstVisitPercentage` - a value between 0 and 1 representing the percentage of new visitors.
- `returnVisitPercentage` - a value between 0 and 1 representing the percentage of returning visitors.
- `gridIntensity` - an object that can contain the following optional keys:
- `device`
- `dataCenter`
- `networks`
- `device` - a number representing the carbon intensity for the given segment (in grams per kilowatt-hour). Or, an object, which contains a key of country and a value that is an Alpha-3 ISO country code.
- `dataCenter` - a number representing the carbon intensity for the given segment (in grams per kilowatt-hour). Or, an object, which contains a key of country and a value that is an Alpha-3 ISO country code.
- `networks` - A number representing the carbon intensity for the given segment (in grams per kilowatt-hour). Or, an object, which contains a key of country and a value that is an Alpha-3 ISO country code.

The value for `device`, `dataCenter`, or `networks` can be a number representing the carbon intensity for the given segment (in grams per kilowatt-hour). Or, an object, which contains a key of country and a value that is an Alpha-3 ISO country code.

Expand All @@ -39,24 +39,24 @@ The CO2JS Framework is a community plugin, not part of the IF standard library.

## Usage

In IEF the plugin is called from an `impl`. An `impl` is a `.yaml` file that contains configuration metadata and usage inputs. This is interpreted by the command line tool, `if`. There, the plugin's `configure` method is called first.
In IF the plugin is called from an `manifest`. An `manifest` is a `.yaml` file that contains configuration metadata and usage inputs. This is interpreted by the command line tool, `if`.

The plugin config should define a `type` supported by the CO2.JS library (either `swd` or `1byte`). These are different ways to calculate the operational carbon associated with a web application; `swd` is shorthand for 'sustainable web design' plugin and `1byte` refers to the OneByte mdoel. You can read about the details of these plugins and how they differ at the [Green Web Foundation website](https://developers.thegreenwebfoundation.org/co2js/explainer/methodologies-for-calculating-website-carbon/).

Each input is expected to contain `network/data/bytes` or `network/data`, `duration` and `timestamp` fields.

## IMPL
## Manifest

The following is an example of how CO2.JS can be invoked using an `impl`.
The following is an example of how CO2.JS can be invoked using an `manifest`.

```yaml
name: co2js-demo
description: example impl invoking CO2.JS plugin
description: example manifest invoking CO2.JS plugin
tags:
initialize:
plugins:
co2js:
function: Co2js
method: Co2js
path: '@grnsft/if-unofficial-plugins'
global-config:
options:
Expand All @@ -82,19 +82,19 @@ tree:
network/data/bytes: 1000000
```
This impl is run using `if` using the following command, run from the project root:
This manifest is run using `if` using the following command, run from the project root:

```sh
npm i -g @grnsft/if
npm i -g @grnsft/if-unofficial-plugins
if --impl ./examples/impls/test/co2js.yml --ompl ./examples/ompls/co2js.yml
if --manifest ./examples/manifests/test/co2js.yml --output ./examples/outputs/co2js.yml
```

This yields a result that looks like the following (saved to `/ompls/co2js.yml`):
This yields a result that looks like the following (saved to `/outputs/co2js.yml`):

```yaml
name: co2js-demo
description: example impl invoking CO2.JS model
description: example manifest invoking CO2.JS model
tags: null
initialize:
plugins:
Expand Down
115 changes: 77 additions & 38 deletions src/lib/co2js/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,38 +15,24 @@ export const Co2js = (globalConfig?: ConfigParams): PluginInterface => {
const errorBuilder = buildErrorMessage(Co2js.name);

/**
* Executes the model for a list of input parameters.
* Executes the plugin for a list of input parameters.
*/
const execute = async (
inputs: PluginParams[],
config?: ConfigParams
): Promise<PluginParams[]> => {
const mergedConfig = Object.assign({}, config, globalConfig);

validateConfig(mergedConfig);

const model = new co2({model: mergedConfig.type});
const execute = async (inputs: PluginParams[], config?: ConfigParams) => {
const mergedValidatedConfig = Object.assign(
{},
validateConfig(config),
validateGlobalConfig()
);
const model = new co2({model: mergedValidatedConfig.type});

return inputs.map(input => {
const mergedWithConfig = Object.assign({}, input, mergedConfig);

if (!(input['network/data/bytes'] || input['network/data'])) {
throw new InputValidationError(
errorBuilder({
message: 'Bytes not provided',
})
);
}

const mergedWithConfig = Object.assign(
{},
validateInput(input),
mergedValidatedConfig
);
const result = calculateResultByParams(mergedWithConfig, model);

if (result) {
return {
...input,
'carbon-operational': result,
};
}

return result
? {
...input,
Expand Down Expand Up @@ -83,23 +69,76 @@ export const Co2js = (globalConfig?: ConfigParams): PluginInterface => {
};

/**
* Validates static parameters.
* Validates input parameters.
*/
const validateInput = (input: PluginParams) => {
const schema = z
.object({
'network/data/bytes': z.number(),
'network/data': z.number(),
})
.partial()
.refine(data => !!data['network/data/bytes'] || !!data['network/data'], {
message:
'Either `network/data/bytes` or `network/data` should be provided in the input.',
});

return validate<z.infer<typeof schema>>(schema, input);
};

/**
* Validates Global config parameters.
*/
const validateGlobalConfig = () => {
const schema = z.object({
options: z
.object({
dataReloadRatio: z.number().min(0).max(1).optional(),
firstVisitPercentage: z.number().min(0).max(1).optional(),
returnVisitPercentage: z.number().min(0).max(1).optional(),
gridIntensity: z
.object({
device: z
.number()
.or(z.object({country: z.string()}))
.optional(),
dataCenter: z
.number()
.or(z.object({country: z.string()}))
.optional(),
networks: z
.number()
.or(z.object({country: z.string()}))
.optional(),
})
.optional(),
})
.optional(),
});

return validate<z.infer<typeof schema>>(schema, globalConfig || {});
};

/**
* Validates node config parameters.
*/
const validateConfig = (config: ConfigParams) => {
const validateConfig = (config?: ConfigParams) => {
if (!config) {
throw new InputValidationError(
errorBuilder({
message: 'Config is not provided',
})
);
}

const schema = z
.object({
type: z.enum(['1byte', 'swd']),
'green-web-host': z.boolean(),
options: z
.object({
dataReloadRatio: z.number().min(0).max(1).optional(),
firstVisitPercentage: z.number().min(0).max(1).optional(),
returnVisitPercentage: z.number().min(0).max(1).optional(),
gridIntensity: z.object({}).optional(),
})
.optional(),
})
.refine(allDefined);
.refine(allDefined, {
message: '`type` and `green-web-host` are not provided in node config',
});

return validate<z.infer<typeof schema>>(schema, config);
};
Expand Down
2 changes: 1 addition & 1 deletion src/lib/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ export {CloudCarbonFootprint} from './ccf';
export {Co2js} from './co2js';
export {TeadsAWS} from './teads-aws';
export {TeadsCurve} from './teads-curve';
export * from './watt-time';
export {WattTimeGridEmissions} from './watt-time';
16 changes: 6 additions & 10 deletions src/lib/teads-aws/README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,14 @@
# Teads' AWS Estimation Model
# Teads' AWS Estimation Plugin

> [!NOTE] > `Teads-AWS` is a community plugin, not part of the IF standard library. This means the IF core team are not closely monitoring these plugins to keep them up to date. You should do your own research before implementing them!
Teads Engineering Team built a plugin for estimating AWS instances energy usage. This plugin creates a power curve on a correlation to SPEC Power database. This allows the plugin to generate a power curve for any AWS EC2 instance type based on publicly available AWS EC2 Instance CPU data.

The main benefit of this plugin is that it accounts for all the components involved in an instance's compute capacity.

## Model name

IF recognizes the Teads AWS plugin as `teads-aws`

## Parameters

### Model global config
### Plugin global config

- `interpolation`: the interpolation method to apply to the TDP curve

Expand All @@ -30,7 +26,7 @@ IF recognizes the Teads AWS plugin as `teads-aws`

## Implementation

IEF implements this plugin based on the data gathered from the CCF (Cloud Carbon Footprint) dataset.
IF implements this plugin based on the data gathered from the CCF (Cloud Carbon Footprint) dataset.

Spline interpolation is implemented as the default method of estimating the usage using the power curve provided by `IDLE`, `10%`, `50%`, `100%` values in the dataset.

Expand All @@ -55,7 +51,7 @@ const results = teads.execute([
]);
```
## Example `impl`
## Example `manifest`
```yaml
name: teads-aws
Expand All @@ -64,7 +60,7 @@ tags:
initialize:
plugins:
teads-aws:
plugin: TeadsAWS
method: TeadsAWS
path: '@grnsft/if-unofficial-plugins'
global-config:
interpolation: linear
Expand All @@ -88,5 +84,5 @@ You can run this by passing it to `if`. Run impact using the following command r
```sh
npm i -g @grnsft/if
npm i -g @grnsft/if-unofficial-plugins
if --impl ./examples/impls/test/teads-aws.yml --ompl ./examples/ompls/teads-aws.yml
if --manifest ./examples/manifests/test/teads-aws.yml --output ./examples/outputs/teads-aws.yml
```
20 changes: 11 additions & 9 deletions src/lib/teads-aws/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ export const TeadsAWS = (globalConfig: ConfigParams): PluginInterface => {
/**
* Calculate the total emissions for a list of inputs.
*/
const execute = async (inputs: PluginParams[]): Promise<PluginParams[]> => {
const execute = async (inputs: PluginParams[]) => {
standardizeInstanceMetrics();

return inputs.map(input => {
Expand All @@ -40,14 +40,15 @@ export const TeadsAWS = (globalConfig: ConfigParams): PluginInterface => {
const validExpectedLifespan =
input['cpu/expected-lifespan'] ?? expectedLifespan;

input['energy'] = calculateEnergy(safeInput, instanceType);
input['carbon-embodied'] = embodiedEmissions(
safeInput,
instanceType,
validExpectedLifespan
);

return input;
return {
...input,
energy: calculateEnergy(safeInput, instanceType),
'carbon-embodied': embodiedEmissions(
safeInput,
instanceType,
validExpectedLifespan
),
};
});
};

Expand Down Expand Up @@ -209,6 +210,7 @@ export const TeadsAWS = (globalConfig: ConfigParams): PluginInterface => {
validateInstanceType(param['cloud/instance-type']);
return true;
});

return validate(schema, input);
};

Expand Down
Loading

0 comments on commit 9a3ebcb

Please sign in to comment.