From bf7acf0f7d3d66e5880bdda72950e585352c9ba6 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 18 Mar 2024 10:10:42 +0000 Subject: [PATCH] deploy: b41495910eb9d180d730d6b6ae56dfc1d279891e --- 404.html | 4 ++-- FAQ/index.html | 4 ++-- assets/js/{0a19367a.2ca03ad6.js => 0a19367a.0190a3ab.js} | 2 +- .../{runtime~main.5e73509c.js => runtime~main.08341cdf.js} | 2 +- contributions/index.html | 4 ++-- developers/how-to-build-plugins/index.html | 4 ++-- developers/how-to-refine-plugins/index.html | 4 ++-- developers/how-to-visualize-results/index.html | 4 ++-- developers/how-to-write-unit-tests/index.html | 4 ++-- developers/index.html | 4 ++-- index.html | 4 ++-- intro/index.html | 4 ++-- major-concepts/aggregation/index.html | 4 ++-- major-concepts/design-philosophy/index.html | 4 ++-- major-concepts/groupby/index.html | 4 ++-- major-concepts/if/index.html | 4 ++-- major-concepts/index.html | 4 ++-- major-concepts/manifest-file/index.html | 4 ++-- major-concepts/parameters/index.html | 4 ++-- major-concepts/plugins/index.html | 4 ++-- major-concepts/time/index.html | 4 ++-- reference/cli/index.html | 4 ++-- reference/index.html | 4 ++-- reference/plugins/index.html | 4 ++-- users/how-to-export-to-csv/index.html | 4 ++-- users/how-to-import-plugins/index.html | 4 ++-- users/how-to-install-if/index.html | 4 ++-- users/how-to-write-manifests/index.html | 4 ++-- users/index.html | 4 ++-- users/quick-start/index.html | 6 +++--- 30 files changed, 59 insertions(+), 59 deletions(-) rename assets/js/{0a19367a.2ca03ad6.js => 0a19367a.0190a3ab.js} (87%) rename assets/js/{runtime~main.5e73509c.js => runtime~main.08341cdf.js} (98%) diff --git a/404.html b/404.html index 5eb25e50..429340d8 100644 --- a/404.html +++ b/404.html @@ -8,13 +8,13 @@ Page Not Found | Impact Framework - +
Skip to main content
This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

Page Not Found

We could not find what you were looking for.

Please contact the owner of the site that linked you to the original URL and let them know their link is broken.

- + \ No newline at end of file diff --git a/FAQ/index.html b/FAQ/index.html index 292517ba..f9da8fad 100644 --- a/FAQ/index.html +++ b/FAQ/index.html @@ -8,13 +8,13 @@ FAQs | Impact Framework - +
Skip to main content
This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

FAQs

Is there any way to auto-generate manifest files?​

You have to create your basic structure but it's expected that your input data will be fille dusing an importer plugin (we have one for Azure VMs and there will be mroe after the hackathon). For testing and experimenting we also have a MockObservations plugin that can autofill manifests with dummy data

What is your vision for the IF?​

The vision is to be a protocol that enables you to use IF for any and all of those use-cases, and more.

Right now we focus on software applications running in the cloud, but we have people using it on-premise and even for supply chain modeling. The idea is that we provide the minimal infrastructure required for you to build up different use cases using plugins.

Is there a way to generate an audit/report along with a csv/yaml output?​

No - we see the manifest file and associated manifest as a new type of "executable audit". The IF itself does not generate any other form of report, although there is nothing stopping others from building out this functionality on top of IF.

The manifest approach makes things transparent, but you can still manipulate the data you put as inputs in the manifest. How to fight again this?​

We can't really stop people inputting fake data into a manifest file. But we're very interested in ways we can verify that the computation was done correctly and provide public proofs. Don't expect anything imminently, but we are thinking along these lines for the future.

Is it planned to make the IF more user friendly or also more usable for non it people?​

IF is a low level infrastructure project. the core team focuses on building out solid foundations for others to build UIs, apps etc on top. There is no official IF UX tooling.

Does it only calculate emissions for CPU usage or does it also work with other meters like storage/bandwith?​

We do have basic models for memory and network usage in addition to CPU. You can create plugins for anything you can observe and model.

Is the cloud-metadata only for public cloud providers?​

Yes it is - it's based on a database we maintain as part of the IF repository. You could extend it by adding the same data for your on premise resource.

Have you compared the results to the carbon reports that cloud providers are producing for their customers?​

No we haven't - we've been focussing on capturing impacts of individual applications over relatively short time periods so far. It's difficult to find the underlying data to build the comparisons.

Are there plans to include other pieces of cloud metadata beyond servers e.g. for PaaS services?​

We don't have immediate specific plans to build this as the IF core team, but we're enthusiastic about teams building plugins for this to support more use-cases

Is there a cloud-metadata for VMware and/or Openstack ?​

No there isn't! It would be a nice addition.

Will IF be developed beyond the upcoming hackathon?​

yes! we are very passionate about this project and it is only going to grow from here, far beyond the hackathon!

- + \ No newline at end of file diff --git a/assets/js/0a19367a.2ca03ad6.js b/assets/js/0a19367a.0190a3ab.js similarity index 87% rename from assets/js/0a19367a.2ca03ad6.js rename to assets/js/0a19367a.0190a3ab.js index a225e74f..65137c7d 100644 --- a/assets/js/0a19367a.2ca03ad6.js +++ b/assets/js/0a19367a.0190a3ab.js @@ -1 +1 @@ -"use strict";(self.webpackChunkgreen_software_training=self.webpackChunkgreen_software_training||[]).push([[109],{4137:(e,t,n)=>{n.d(t,{Zo:()=>u,kt:()=>d});var a=n(7294);function i(e,t,n){return t in e?Object.defineProperty(e,t,{value:n,enumerable:!0,configurable:!0,writable:!0}):e[t]=n,e}function r(e,t){var n=Object.keys(e);if(Object.getOwnPropertySymbols){var a=Object.getOwnPropertySymbols(e);t&&(a=a.filter((function(t){return Object.getOwnPropertyDescriptor(e,t).enumerable}))),n.push.apply(n,a)}return n}function l(e){for(var t=1;t=0||(i[n]=e[n]);return i}(e,t);if(Object.getOwnPropertySymbols){var r=Object.getOwnPropertySymbols(e);for(a=0;a=0||Object.prototype.propertyIsEnumerable.call(e,n)&&(i[n]=e[n])}return i}var s=a.createContext({}),p=function(e){var t=a.useContext(s),n=t;return e&&(n="function"==typeof e?e(t):l(l({},t),e)),n},u=function(e){var t=p(e.components);return a.createElement(s.Provider,{value:t},e.children)},m={inlineCode:"code",wrapper:function(e){var t=e.children;return a.createElement(a.Fragment,{},t)}},c=a.forwardRef((function(e,t){var n=e.components,i=e.mdxType,r=e.originalType,s=e.parentName,u=o(e,["components","mdxType","originalType","parentName"]),c=p(n),d=i,f=c["".concat(s,".").concat(d)]||c[d]||m[d]||r;return n?a.createElement(f,l(l({ref:t},u),{},{components:n})):a.createElement(f,l({ref:t},u))}));function d(e,t){var n=arguments,i=t&&t.mdxType;if("string"==typeof e||i){var r=n.length,l=new Array(r);l[0]=c;var o={};for(var s in t)hasOwnProperty.call(t,s)&&(o[s]=t[s]);o.originalType=e,o.mdxType="string"==typeof e?e:i,l[1]=o;for(var p=2;p{n.r(t),n.d(t,{assets:()=>s,contentTitle:()=>l,default:()=>m,frontMatter:()=>r,metadata:()=>o,toc:()=>p});var a=n(7462),i=(n(7294),n(4137));const r={sidebar_position:1},l="Quick start",o={unversionedId:"users/quick-start",id:"users/quick-start",title:"Quick start",description:"This page will provide the basic instructions for getting up and running with Impact Framework.",source:"@site/docs/users/quick-start.md",sourceDirName:"users",slug:"/users/quick-start",permalink:"/users/quick-start",draft:!1,editUrl:"https://github.com/Green-Software-Foundation/ief/docs/users/quick-start.md",tags:[],version:"current",sidebarPosition:1,frontMatter:{sidebar_position:1},sidebar:"tutorialSidebar",previous:{title:"Users",permalink:"/users/"},next:{title:"How to install Impact Framework",permalink:"/users/how-to-install-if"}},s={},p=[{value:"1: Install Impact Framework",id:"1-install-impact-framework",level:2},{value:"2: Install some plugins",id:"2-install-some-plugins",level:2},{value:"3: Create a manifest file",id:"3-create-a-manifest-file",level:2},{value:"4: Compute your manifest file",id:"4-compute-your-manifest-file",level:2},{value:"Next steps",id:"next-steps",level:2}],u={toc:p};function m(e){let{components:t,...n}=e;return(0,i.kt)("wrapper",(0,a.Z)({},u,n,{components:t,mdxType:"MDXLayout"}),(0,i.kt)("h1",{id:"quick-start"},"Quick start"),(0,i.kt)("p",null,"This page will provide the basic instructions for getting up and running with Impact Framework."),(0,i.kt)("h2",{id:"1-install-impact-framework"},"1: Install Impact Framework"),(0,i.kt)("p",null,"Install the Impact Framework globally using npm."),(0,i.kt)("pre",null,(0,i.kt)("code",{parentName:"pre",className:"language-sh"},"npm install -g @grnsft/if\n")),(0,i.kt)("p",null,"Read our detailed guide to ",(0,i.kt)("a",{parentName:"p",href:"/users/how-to-install-if"},"installing IF"),"."),(0,i.kt)("h2",{id:"2-install-some-plugins"},"2: Install some plugins"),(0,i.kt)("p",null,"Install some of the plugins you want to include in your pipeline. The following commands will install both the official and unofficial IF model packages."),(0,i.kt)("pre",null,(0,i.kt)("code",{parentName:"pre",className:"language-sh"},"npm install -g @grnsft/if-plugins\nnpm install -g @grnsft/if-unofficial-plugins\n")),(0,i.kt)("p",null,"Read our detailed guide to ",(0,i.kt)("a",{parentName:"p",href:"/users/how-to-import-plugins"},"loading plugins"),"."),(0,i.kt)("h2",{id:"3-create-a-manifest-file"},"3: Create a manifest file"),(0,i.kt)("p",null,"A manifest file contains all the configuration and input data required to measure your application's energy and carbon impacts and should have a ",(0,i.kt)("inlineCode",{parentName:"p"},".yml")," extension. "),(0,i.kt)("p",null,"Open the file, add your data and save the file. The simple example below runs a single snapshot observation through a single plugin."),(0,i.kt)("pre",null,(0,i.kt)("code",{parentName:"pre",className:"language-yaml"},"name: basic-demo\ndescription:\ntags:\ninitialize:\n plugins:\n teads-curve: \n path: '@grnsft/if-unofficial-plugins'\n method: TeadsCurve\n global-config:\n interpolation: spline\ntree:\n children:\n child-0:\n defaults:\n cpu/thermal-design-power: 100\n pipeline:\n - teads-curve\n inputs:\n - timestamp: 2023-07-06T00:00\n duration: 1\n cpu/utilization: 20\n - timestamp: 2023-07-06T00:01\n duration: 1\n cpu/utilization: 80\n - timestamp: 2023-07-06T00:02\n duration: 1\n cpu/utilization: 20\n")),(0,i.kt)("p",null,"Read our detailed guide to ",(0,i.kt)("a",{parentName:"p",href:"/users/how-to-write-manifests"},"writing manifest files"),"."),(0,i.kt)("h2",{id:"4-compute-your-manifest-file"},"4: Compute your manifest file"),(0,i.kt)("p",null,"Run the pipeline by passing the path to your manifest file to the ",(0,i.kt)("inlineCode",{parentName:"p"},"ie")," command line tool:"),(0,i.kt)("pre",null,(0,i.kt)("code",{parentName:"pre",className:"language-sh"},"ie --manifest \n")),(0,i.kt)("p",null,"\ud83c\udf89",(0,i.kt)("strong",{parentName:"p"},"Congratulations")," \ud83c\udf89! You have just used the Impact Framework to compute the energy consumed by an application! "),(0,i.kt)("h2",{id:"next-steps"},"Next steps"),(0,i.kt)("p",null,"Now you know how to use the ",(0,i.kt)("inlineCode",{parentName:"p"},"ie")," you can start building more complex pipelines of plugins and more complicated manifest files. Your overall aim is to create a manifest file that accurately represents a real software application, and a plugin pipeline that yields an environmental metric that's important to you (e.g. ",(0,i.kt)("inlineCode",{parentName:"p"},"carbon"),")."),(0,i.kt)("p",null,"Experiment by adding more plugins to the pipeline, for example add ",(0,i.kt)("inlineCode",{parentName:"p"},"sci-o")," to convert energy into ",(0,i.kt)("inlineCode",{parentName:"p"},"operational-carbon"),". Your output data will be displayed in your console. "),(0,i.kt)("p",null,"You can also configure ",(0,i.kt)("inlineCode",{parentName:"p"},"if")," to save your output data to another ",(0,i.kt)("inlineCode",{parentName:"p"},"yaml")," file. To do this, add the ",(0,i.kt)("inlineCode",{parentName:"p"},"--output")," flag and the path to save the file to. You will also need to add the following config to the ",(0,i.kt)("inlineCode",{parentName:"p"},"initialize")," block in your manifest file:"),(0,i.kt)("pre",null,(0,i.kt)("code",{parentName:"pre",className:"language-yaml"},"outputs:\n - yaml\n")),(0,i.kt)("p",null,"The command is then as follows:"),(0,i.kt)("pre",null,(0,i.kt)("code",{parentName:"pre",className:"language-sh"},"ie --manifest --output \n")),(0,i.kt)("p",null,"Explore our user documentation for walkthrough guides to common Impact Framework tasks:"),(0,i.kt)("ul",null,(0,i.kt)("li",{parentName:"ul"},(0,i.kt)("a",{parentName:"li",href:"/users/how-to-install-if"},"How to install Impact Framework")),(0,i.kt)("li",{parentName:"ul"},(0,i.kt)("a",{parentName:"li",href:"/users/how-to-import-plugins"},"How to load plugins")),(0,i.kt)("li",{parentName:"ul"},(0,i.kt)("a",{parentName:"li",href:"/users/how-to-write-manifests"},"How to write manifest files"))))}m.isMDXComponent=!0}}]); \ No newline at end of file +"use strict";(self.webpackChunkgreen_software_training=self.webpackChunkgreen_software_training||[]).push([[109],{4137:(e,t,n)=>{n.d(t,{Zo:()=>u,kt:()=>d});var a=n(7294);function i(e,t,n){return t in e?Object.defineProperty(e,t,{value:n,enumerable:!0,configurable:!0,writable:!0}):e[t]=n,e}function r(e,t){var n=Object.keys(e);if(Object.getOwnPropertySymbols){var a=Object.getOwnPropertySymbols(e);t&&(a=a.filter((function(t){return Object.getOwnPropertyDescriptor(e,t).enumerable}))),n.push.apply(n,a)}return n}function l(e){for(var t=1;t=0||(i[n]=e[n]);return i}(e,t);if(Object.getOwnPropertySymbols){var r=Object.getOwnPropertySymbols(e);for(a=0;a=0||Object.prototype.propertyIsEnumerable.call(e,n)&&(i[n]=e[n])}return i}var s=a.createContext({}),p=function(e){var t=a.useContext(s),n=t;return e&&(n="function"==typeof e?e(t):l(l({},t),e)),n},u=function(e){var t=p(e.components);return a.createElement(s.Provider,{value:t},e.children)},m={inlineCode:"code",wrapper:function(e){var t=e.children;return a.createElement(a.Fragment,{},t)}},c=a.forwardRef((function(e,t){var n=e.components,i=e.mdxType,r=e.originalType,s=e.parentName,u=o(e,["components","mdxType","originalType","parentName"]),c=p(n),d=i,f=c["".concat(s,".").concat(d)]||c[d]||m[d]||r;return n?a.createElement(f,l(l({ref:t},u),{},{components:n})):a.createElement(f,l({ref:t},u))}));function d(e,t){var n=arguments,i=t&&t.mdxType;if("string"==typeof e||i){var r=n.length,l=new Array(r);l[0]=c;var o={};for(var s in t)hasOwnProperty.call(t,s)&&(o[s]=t[s]);o.originalType=e,o.mdxType="string"==typeof e?e:i,l[1]=o;for(var p=2;p{n.r(t),n.d(t,{assets:()=>s,contentTitle:()=>l,default:()=>m,frontMatter:()=>r,metadata:()=>o,toc:()=>p});var a=n(7462),i=(n(7294),n(4137));const r={sidebar_position:1},l="Quick start",o={unversionedId:"users/quick-start",id:"users/quick-start",title:"Quick start",description:"This page will provide the basic instructions for getting up and running with Impact Framework.",source:"@site/docs/users/quick-start.md",sourceDirName:"users",slug:"/users/quick-start",permalink:"/users/quick-start",draft:!1,editUrl:"https://github.com/Green-Software-Foundation/ief/docs/users/quick-start.md",tags:[],version:"current",sidebarPosition:1,frontMatter:{sidebar_position:1},sidebar:"tutorialSidebar",previous:{title:"Users",permalink:"/users/"},next:{title:"How to install Impact Framework",permalink:"/users/how-to-install-if"}},s={},p=[{value:"1: Install Impact Framework",id:"1-install-impact-framework",level:2},{value:"2: Install some plugins",id:"2-install-some-plugins",level:2},{value:"3: Create a manifest file",id:"3-create-a-manifest-file",level:2},{value:"4: Compute your manifest file",id:"4-compute-your-manifest-file",level:2},{value:"Next steps",id:"next-steps",level:2}],u={toc:p};function m(e){let{components:t,...n}=e;return(0,i.kt)("wrapper",(0,a.Z)({},u,n,{components:t,mdxType:"MDXLayout"}),(0,i.kt)("h1",{id:"quick-start"},"Quick start"),(0,i.kt)("p",null,"This page will provide the basic instructions for getting up and running with Impact Framework."),(0,i.kt)("h2",{id:"1-install-impact-framework"},"1: Install Impact Framework"),(0,i.kt)("p",null,"Install the Impact Framework globally using npm."),(0,i.kt)("pre",null,(0,i.kt)("code",{parentName:"pre",className:"language-sh"},"npm install -g @grnsft/if\n")),(0,i.kt)("p",null,"Read our detailed guide to ",(0,i.kt)("a",{parentName:"p",href:"/users/how-to-install-if"},"installing IF"),"."),(0,i.kt)("h2",{id:"2-install-some-plugins"},"2: Install some plugins"),(0,i.kt)("p",null,"Install some of the plugins you want to include in your pipeline. The following commands will install both the official and unofficial IF model packages."),(0,i.kt)("pre",null,(0,i.kt)("code",{parentName:"pre",className:"language-sh"},"npm install -g @grnsft/if-plugins\nnpm install -g @grnsft/if-unofficial-plugins\n")),(0,i.kt)("p",null,"Read our detailed guide to ",(0,i.kt)("a",{parentName:"p",href:"/users/how-to-import-plugins"},"loading plugins"),"."),(0,i.kt)("h2",{id:"3-create-a-manifest-file"},"3: Create a manifest file"),(0,i.kt)("p",null,"A manifest file contains all the configuration and input data required to measure your application's energy and carbon impacts and should have a ",(0,i.kt)("inlineCode",{parentName:"p"},".yml")," extension. "),(0,i.kt)("p",null,"Open the file, add your data and save the file. The simple example below runs a single snapshot observation through a single plugin."),(0,i.kt)("pre",null,(0,i.kt)("code",{parentName:"pre",className:"language-yaml"},"name: basic-demo\ndescription:\ntags:\ninitialize:\n plugins:\n teads-curve: \n path: '@grnsft/if-unofficial-plugins'\n method: TeadsCurve\n global-config:\n interpolation: spline\ntree:\n children:\n child-0:\n defaults:\n cpu/thermal-design-power: 100\n pipeline:\n - teads-curve\n inputs:\n - timestamp: 2023-07-06T00:00\n duration: 1\n cpu/utilization: 20\n - timestamp: 2023-07-06T00:01\n duration: 1\n cpu/utilization: 80\n - timestamp: 2023-07-06T00:02\n duration: 1\n cpu/utilization: 20\n")),(0,i.kt)("p",null,"Read our detailed guide to ",(0,i.kt)("a",{parentName:"p",href:"/users/how-to-write-manifests"},"writing manifest files"),"."),(0,i.kt)("h2",{id:"4-compute-your-manifest-file"},"4: Compute your manifest file"),(0,i.kt)("p",null,"Run the pipeline by passing the path to your manifest file to the ",(0,i.kt)("inlineCode",{parentName:"p"},"ie")," command line tool:"),(0,i.kt)("pre",null,(0,i.kt)("code",{parentName:"pre",className:"language-sh"},"ie --manifest \n")),(0,i.kt)("p",null,"\ud83c\udf89",(0,i.kt)("strong",{parentName:"p"},"Congratulations")," \ud83c\udf89! You have just used the Impact Framework to compute the energy consumed by an application! "),(0,i.kt)("h2",{id:"next-steps"},"Next steps"),(0,i.kt)("p",null,"Now you know how to use the ",(0,i.kt)("inlineCode",{parentName:"p"},"ie")," you can start building more complex pipelines of plugins and more complicated manifest files. Your overall aim is to create a manifest file that accurately represents a real software application, and a plugin pipeline that yields an environmental metric that's important to you (e.g. ",(0,i.kt)("inlineCode",{parentName:"p"},"carbon"),")."),(0,i.kt)("p",null,"Experiment by adding more plugins to the pipeline, for example add ",(0,i.kt)("inlineCode",{parentName:"p"},"sci-o")," to convert energy into ",(0,i.kt)("inlineCode",{parentName:"p"},"operational-carbon"),". Your output data will be displayed in your console. "),(0,i.kt)("p",null,"You can also configure ",(0,i.kt)("inlineCode",{parentName:"p"},"if")," to save your output data to another ",(0,i.kt)("inlineCode",{parentName:"p"},"yaml")," file. To do this, add the ",(0,i.kt)("inlineCode",{parentName:"p"},"--output")," flag and the path to the output file where the results are saved. You will also need to add the following config to the ",(0,i.kt)("inlineCode",{parentName:"p"},"initialize")," block in your manifest file:"),(0,i.kt)("pre",null,(0,i.kt)("code",{parentName:"pre",className:"language-yaml"},"outputs:\n - yaml\n")),(0,i.kt)("p",null,"The command is then as follows:"),(0,i.kt)("pre",null,(0,i.kt)("code",{parentName:"pre",className:"language-sh"},"ie --manifest --output \n")),(0,i.kt)("p",null,"Explore our user documentation for walkthrough guides to common Impact Framework tasks:"),(0,i.kt)("ul",null,(0,i.kt)("li",{parentName:"ul"},(0,i.kt)("a",{parentName:"li",href:"/users/how-to-install-if"},"How to install Impact Framework")),(0,i.kt)("li",{parentName:"ul"},(0,i.kt)("a",{parentName:"li",href:"/users/how-to-import-plugins"},"How to load plugins")),(0,i.kt)("li",{parentName:"ul"},(0,i.kt)("a",{parentName:"li",href:"/users/how-to-write-manifests"},"How to write manifest files"))))}m.isMDXComponent=!0}}]); \ No newline at end of file diff --git a/assets/js/runtime~main.5e73509c.js b/assets/js/runtime~main.08341cdf.js similarity index 98% rename from assets/js/runtime~main.5e73509c.js rename to assets/js/runtime~main.08341cdf.js index 9974b8a9..797b29c3 100644 --- a/assets/js/runtime~main.5e73509c.js +++ b/assets/js/runtime~main.08341cdf.js @@ -1 +1 @@ -(()=>{"use strict";var e,t,r,a,f,o={},n={};function c(e){var t=n[e];if(void 0!==t)return t.exports;var r=n[e]={id:e,loaded:!1,exports:{}};return o[e].call(r.exports,r,r.exports,c),r.loaded=!0,r.exports}c.m=o,c.c=n,e=[],c.O=(t,r,a,f)=>{if(!r){var o=1/0;for(i=0;i=f)&&Object.keys(c.O).every((e=>c.O[e](r[d])))?r.splice(d--,1):(n=!1,f0&&e[i-1][2]>f;i--)e[i]=e[i-1];e[i]=[r,a,f]},c.n=e=>{var t=e&&e.__esModule?()=>e.default:()=>e;return c.d(t,{a:t}),t},r=Object.getPrototypeOf?e=>Object.getPrototypeOf(e):e=>e.__proto__,c.t=function(e,a){if(1&a&&(e=this(e)),8&a)return e;if("object"==typeof e&&e){if(4&a&&e.__esModule)return e;if(16&a&&"function"==typeof e.then)return e}var f=Object.create(null);c.r(f);var o={};t=t||[null,r({}),r([]),r(r)];for(var n=2&a&&e;"object"==typeof n&&!~t.indexOf(n);n=r(n))Object.getOwnPropertyNames(n).forEach((t=>o[t]=()=>e[t]));return o.default=()=>e,c.d(f,o),f},c.d=(e,t)=>{for(var r in t)c.o(t,r)&&!c.o(e,r)&&Object.defineProperty(e,r,{enumerable:!0,get:t[r]})},c.f={},c.e=e=>Promise.all(Object.keys(c.f).reduce(((t,r)=>(c.f[r](e,t),t)),[])),c.u=e=>"assets/js/"+({53:"935f2afb",99:"449bc0d9",109:"0a19367a",112:"cca8bc57",117:"29966d47",162:"486bc80e",168:"fd565be6",229:"75e434b4",237:"1df93b7f",288:"ad895e75",352:"f744e480",426:"244c9605",468:"5c5a410f",491:"b5ada7f5",514:"1be78505",527:"859a76f9",533:"bdfbfcca",644:"68ffc9f1",671:"0e384e19",672:"a4954f21",681:"425ff8ea",702:"3c29e2ef",713:"93f138be",751:"1a3c9b31",754:"2d14298d",760:"069226b7",806:"28f080da",822:"63925da8",890:"1ead5b6d",918:"17896441",941:"090462b1",960:"91c76d4c"}[e]||e)+"."+{53:"4a3fba5e",99:"4592c902",109:"2ca03ad6",112:"0af78829",117:"72f5fed4",162:"4b12c136",168:"1f7d4f62",229:"6ba4fb88",237:"ac68505e",248:"d6c76d13",288:"6ddd513d",352:"4b66db01",426:"20b5e353",468:"75606c3d",491:"cf500660",514:"a282f924",527:"aa11797a",533:"42b4dcbc",644:"28002050",671:"96f019ec",672:"34a05832",681:"44661c58",702:"85deae6f",713:"dcabecf0",751:"dc917f97",754:"d3baaf66",760:"19270c29",806:"f1939e8e",822:"78d94b7c",890:"25860973",918:"dd24f0b7",941:"96edb2cc",960:"9525df5d"}[e]+".js",c.miniCssF=e=>{},c.g=function(){if("object"==typeof globalThis)return globalThis;try{return this||new Function("return this")()}catch(e){if("object"==typeof window)return window}}(),c.o=(e,t)=>Object.prototype.hasOwnProperty.call(e,t),a={},f="green-software-training:",c.l=(e,t,r,o)=>{if(a[e])a[e].push(t);else{var n,d;if(void 0!==r)for(var b=document.getElementsByTagName("script"),i=0;i{n.onerror=n.onload=null,clearTimeout(s);var f=a[e];if(delete a[e],n.parentNode&&n.parentNode.removeChild(n),f&&f.forEach((e=>e(r))),t)return t(r)},s=setTimeout(l.bind(null,void 0,{type:"timeout",target:n}),12e4);n.onerror=l.bind(null,n.onerror),n.onload=l.bind(null,n.onload),d&&document.head.appendChild(n)}},c.r=e=>{"undefined"!=typeof Symbol&&Symbol.toStringTag&&Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},c.p="/",c.gca=function(e){return e={17896441:"918","935f2afb":"53","449bc0d9":"99","0a19367a":"109",cca8bc57:"112","29966d47":"117","486bc80e":"162",fd565be6:"168","75e434b4":"229","1df93b7f":"237",ad895e75:"288",f744e480:"352","244c9605":"426","5c5a410f":"468",b5ada7f5:"491","1be78505":"514","859a76f9":"527",bdfbfcca:"533","68ffc9f1":"644","0e384e19":"671",a4954f21:"672","425ff8ea":"681","3c29e2ef":"702","93f138be":"713","1a3c9b31":"751","2d14298d":"754","069226b7":"760","28f080da":"806","63925da8":"822","1ead5b6d":"890","090462b1":"941","91c76d4c":"960"}[e]||e,c.p+c.u(e)},(()=>{var e={303:0,532:0};c.f.j=(t,r)=>{var a=c.o(e,t)?e[t]:void 0;if(0!==a)if(a)r.push(a[2]);else if(/^(303|532)$/.test(t))e[t]=0;else{var f=new Promise(((r,f)=>a=e[t]=[r,f]));r.push(a[2]=f);var o=c.p+c.u(t),n=new Error;c.l(o,(r=>{if(c.o(e,t)&&(0!==(a=e[t])&&(e[t]=void 0),a)){var f=r&&("load"===r.type?"missing":r.type),o=r&&r.target&&r.target.src;n.message="Loading chunk "+t+" failed.\n("+f+": "+o+")",n.name="ChunkLoadError",n.type=f,n.request=o,a[1](n)}}),"chunk-"+t,t)}},c.O.j=t=>0===e[t];var t=(t,r)=>{var a,f,o=r[0],n=r[1],d=r[2],b=0;if(o.some((t=>0!==e[t]))){for(a in n)c.o(n,a)&&(c.m[a]=n[a]);if(d)var i=d(c)}for(t&&t(r);b{"use strict";var e,t,r,a,f,o={},n={};function c(e){var t=n[e];if(void 0!==t)return t.exports;var r=n[e]={id:e,loaded:!1,exports:{}};return o[e].call(r.exports,r,r.exports,c),r.loaded=!0,r.exports}c.m=o,c.c=n,e=[],c.O=(t,r,a,f)=>{if(!r){var o=1/0;for(i=0;i=f)&&Object.keys(c.O).every((e=>c.O[e](r[d])))?r.splice(d--,1):(n=!1,f0&&e[i-1][2]>f;i--)e[i]=e[i-1];e[i]=[r,a,f]},c.n=e=>{var t=e&&e.__esModule?()=>e.default:()=>e;return c.d(t,{a:t}),t},r=Object.getPrototypeOf?e=>Object.getPrototypeOf(e):e=>e.__proto__,c.t=function(e,a){if(1&a&&(e=this(e)),8&a)return e;if("object"==typeof e&&e){if(4&a&&e.__esModule)return e;if(16&a&&"function"==typeof e.then)return e}var f=Object.create(null);c.r(f);var o={};t=t||[null,r({}),r([]),r(r)];for(var n=2&a&&e;"object"==typeof n&&!~t.indexOf(n);n=r(n))Object.getOwnPropertyNames(n).forEach((t=>o[t]=()=>e[t]));return o.default=()=>e,c.d(f,o),f},c.d=(e,t)=>{for(var r in t)c.o(t,r)&&!c.o(e,r)&&Object.defineProperty(e,r,{enumerable:!0,get:t[r]})},c.f={},c.e=e=>Promise.all(Object.keys(c.f).reduce(((t,r)=>(c.f[r](e,t),t)),[])),c.u=e=>"assets/js/"+({53:"935f2afb",99:"449bc0d9",109:"0a19367a",112:"cca8bc57",117:"29966d47",162:"486bc80e",168:"fd565be6",229:"75e434b4",237:"1df93b7f",288:"ad895e75",352:"f744e480",426:"244c9605",468:"5c5a410f",491:"b5ada7f5",514:"1be78505",527:"859a76f9",533:"bdfbfcca",644:"68ffc9f1",671:"0e384e19",672:"a4954f21",681:"425ff8ea",702:"3c29e2ef",713:"93f138be",751:"1a3c9b31",754:"2d14298d",760:"069226b7",806:"28f080da",822:"63925da8",890:"1ead5b6d",918:"17896441",941:"090462b1",960:"91c76d4c"}[e]||e)+"."+{53:"4a3fba5e",99:"4592c902",109:"0190a3ab",112:"0af78829",117:"72f5fed4",162:"4b12c136",168:"1f7d4f62",229:"6ba4fb88",237:"ac68505e",248:"d6c76d13",288:"6ddd513d",352:"4b66db01",426:"20b5e353",468:"75606c3d",491:"cf500660",514:"a282f924",527:"aa11797a",533:"42b4dcbc",644:"28002050",671:"96f019ec",672:"34a05832",681:"44661c58",702:"85deae6f",713:"dcabecf0",751:"dc917f97",754:"d3baaf66",760:"19270c29",806:"f1939e8e",822:"78d94b7c",890:"25860973",918:"dd24f0b7",941:"96edb2cc",960:"9525df5d"}[e]+".js",c.miniCssF=e=>{},c.g=function(){if("object"==typeof globalThis)return globalThis;try{return this||new Function("return this")()}catch(e){if("object"==typeof window)return window}}(),c.o=(e,t)=>Object.prototype.hasOwnProperty.call(e,t),a={},f="green-software-training:",c.l=(e,t,r,o)=>{if(a[e])a[e].push(t);else{var n,d;if(void 0!==r)for(var b=document.getElementsByTagName("script"),i=0;i{n.onerror=n.onload=null,clearTimeout(s);var f=a[e];if(delete a[e],n.parentNode&&n.parentNode.removeChild(n),f&&f.forEach((e=>e(r))),t)return t(r)},s=setTimeout(l.bind(null,void 0,{type:"timeout",target:n}),12e4);n.onerror=l.bind(null,n.onerror),n.onload=l.bind(null,n.onload),d&&document.head.appendChild(n)}},c.r=e=>{"undefined"!=typeof Symbol&&Symbol.toStringTag&&Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},c.p="/",c.gca=function(e){return e={17896441:"918","935f2afb":"53","449bc0d9":"99","0a19367a":"109",cca8bc57:"112","29966d47":"117","486bc80e":"162",fd565be6:"168","75e434b4":"229","1df93b7f":"237",ad895e75:"288",f744e480:"352","244c9605":"426","5c5a410f":"468",b5ada7f5:"491","1be78505":"514","859a76f9":"527",bdfbfcca:"533","68ffc9f1":"644","0e384e19":"671",a4954f21:"672","425ff8ea":"681","3c29e2ef":"702","93f138be":"713","1a3c9b31":"751","2d14298d":"754","069226b7":"760","28f080da":"806","63925da8":"822","1ead5b6d":"890","090462b1":"941","91c76d4c":"960"}[e]||e,c.p+c.u(e)},(()=>{var e={303:0,532:0};c.f.j=(t,r)=>{var a=c.o(e,t)?e[t]:void 0;if(0!==a)if(a)r.push(a[2]);else if(/^(303|532)$/.test(t))e[t]=0;else{var f=new Promise(((r,f)=>a=e[t]=[r,f]));r.push(a[2]=f);var o=c.p+c.u(t),n=new Error;c.l(o,(r=>{if(c.o(e,t)&&(0!==(a=e[t])&&(e[t]=void 0),a)){var f=r&&("load"===r.type?"missing":r.type),o=r&&r.target&&r.target.src;n.message="Loading chunk "+t+" failed.\n("+f+": "+o+")",n.name="ChunkLoadError",n.type=f,n.request=o,a[1](n)}}),"chunk-"+t,t)}},c.O.j=t=>0===e[t];var t=(t,r)=>{var a,f,o=r[0],n=r[1],d=r[2],b=0;if(o.some((t=>0!==e[t]))){for(a in n)c.o(n,a)&&(c.m[a]=n[a]);if(d)var i=d(c)}for(t&&t(r);b Contribution Guidelines | Impact Framework - + @@ -19,7 +19,7 @@ like function/variable names

Examples:

 util: add getInitializedPlugin method to plugin.
 deps: add express package to dependencies.
 service: refactor get user.

  • Keep the second line blank.

  • Wrap all other lines at 72 columns:

    • describe each line in logical chunks
    • start each line with: space hyphen space ( - ...)
    • be entirely in lowercase with the exception of proper nouns, acronyms, and the words that refer to code, like function/variable names

    Examples:

      - remove deprecated logger
      - refactor some method
      - add JSDoc on existing function

    • Coding guidelines​

    Code structuring patterns​

    Avoid having functions which are responsible to do multiple things at the same time. Make sure one function/method does one thing, and does it well.

    Object Oriented Programming​

    While following Object Oriented Programming paradigm, it's important to follow SOLID principles.

    Functional Programming​

    When designing module of the application in Functional Programming paradigm, the key to follow basic principles.

    Naming patterns​

    Make sure your class/function/variable describes exactly what it does. Avoid using shortened words like txt, arr while doing naming decision. As a naming pattern camel case should be used.

    ❌ const a = "<MOCK_VALUE_HERE>"
    ✅ const mockValue = "<MOCK_VALUE_HERE>"

    ❌ const a = (txt: string) => console.log(txt)
    ✅ const logMessage = (message: string) => console.log(message)

    Documentation​

    Every logical unit (Class, function, method) should be covered with appropriate documentation. For documenting such, multi-line comment style is used.

    ❌ const a = (message: string) => console.log(message)

    ✅
    /**
     * Logs given `message` to console.
     **/
    const logMessage = (message: string) => console.log(message)

    For documenting variable, expression, single line comments can be used after declaration.

    class MockClass {
         // this is a mock field
      ❌ private mockField: string = "mock-field"
      ✅ private mockField: string = "mock-field" // Single line documenting style is used.
    }

    Writing tests​

    One test file should be responsible for one module. describe blocks should be used for module and function/method description. First describe should follow resource/module: pattern. Second describe title should follow method(): pattern. Test units can use either test or it, title should exactly describe behaviour and input argument. Make sure each test case covers one branch.

    See example: 

    describe('util/args: ', () => {
       describe('parseProcessArgument(): ', () => {
          it('logs help message if property present in env.', () => {
             ...
          })
       })
    })

    ⬅️ back to the root

    - + \ No newline at end of file diff --git a/developers/how-to-build-plugins/index.html b/developers/how-to-build-plugins/index.html index d819782e..770939e8 100644 --- a/developers/how-to-build-plugins/index.html +++ b/developers/how-to-build-plugins/index.html @@ -8,14 +8,14 @@ How to build plugins | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    How to build plugins

    The IF is designed to be as composable as possible. This means you can develop your own plugins and use them in a pipeline. To help developers write Typescript plugins to integrate easily into IF, we provide the PluginInterface interface. Here's an overview of the stages you need to follow to integrate your plugin:

    • create a Typescript file that implements the PluginInterface
    • install the plugin
    • initialize and invoke the plugin in your manifest file

    Step 1: Use our template repository​

    Instead of building up your plugin repository and all the configuration from scratch, you can use our plugin template repository. To use the template, visit the Github repository and click the Use this template button. You will have the option to create a new repository under your own account. Then, you can clone that repository to your local machine.

    use our template repository

    Inside that repository, all you have to do is run npm install typescript in the template folder, rename the project in package.json and write your plugin code inside index.ts. All the configuration and setup is taken care of for you.

    Step 2: Writing your plugin code​

    Now your project is setup, you can focus on your plugin logic. The entry point for your plugin is index.ts. In this guide it is assumed that all your plugin logic is in index.ts but depending on the copmplexity of your plugin you might want to split the code across multiple files. index.ts should always be your entry point, though.

    The following sections describe the rules your plugin code should conform to. We also have an appendix that deep dives a real plugin.

    The plugin interface​

    The PluginInterface is structured as follows:

    export type PluginInterface = {
      execute: (
        inputs: PluginParams[],
        config?: Record<string, any>
      ) => PluginParams[];
      metadata: {
        kind: string;
      };
      [key: string]: any;
    };

    The interface requires an execute function where your plugin logic is implemented. It should also return metadata. This can include any relevant metadata you want to include, with a minimum requirement being kind: execute.

    Global config​

    Global config is passed as an argument to the plugin. In your plugin code you can handle it as follows:

    // Here's the function definition - notice that global config is passed in here!
    export const Plugin = (globalConfig: YourConfig): PluginInterface => {

    // in here you have access to globalConfig[your-params]

    }

    The parameters available to you in globalConfig depends upon the parameters you pass in the manifest file. For example, the Sum plugin has access to input-parameters and output-parameter in its global config, and it is defined in the Initialize block in the manifest file as follows:

    initialize:
      plugins:
        sum:
          method: Sum
          path: '@grnsft/if-plugins'
          global-config:
            input-parameters: ['cpu/energy', 'network/energy']
            output-parameter: 'energy'

    Methods​

    execute​

    execute() is where the main calculation logic of the plugin is implemented. It always takes inputs (an array of PluginParams) as an argument and returns an updated set of inputs.

    Params​

    ParamTypePurpose
    inputsPluginParams[]Array of data provided in the inputs field of a component in a manifest file

    Returns​

    Return valueTypePurpose
    outputsPromise<PluginParams[]>Promise resolving to an array of updated PluginParams[]

    What are PluginParams?​

    What are PluginParams?​

    PluginParams are a fundamental data type in the Impact Framework. The type is defined as follows:

    export type PluginParams = {
      [key: string]: any;
    };

    The PluginParams type therefore defines an array of key-value pairs.

    IF needs to know about all the parameters used in each pipeline. The default behaviour is that it grabs parameters from a local file, params.ts. This file defines the standard set of parameter names, their units, a descriptiona nd the method used to aggregate them across time or across a tree.

    If your new plugin uses new parameters that are not included in params.ts, you can simply add them to your manifest file in a section named params. For example:

    name: params-demo
    description: null
    tags:
    params: 
      - name: new-param-1
        description: dummy
        aggregation: sum
        unit: MT
      - name: new-param-2
        description: dummy
        aggregation: sum
        unit: s

    This will append the new parameter informatrion to the object loaded from params.ts and you can use your plugin as normal. In effect, you have append-only access to params.ts via your manifest file without ever having to change any IF source code.

    However, if you are an advanced user and you want to use something other than out recommended standard set of parameters, you can provide a replacement params.ts file on the command line. This file should be a json or js/ts file with the ame structure as our params.ts. You can rename the file. You then pass the path to the file to the override-params command.

    ie --manifest <path-to-manifest> --override-params <path-to-your-params-file>

    Step 3: Install your plugin​

    Now your plugin code is written, you can install it to make it available to IF.

    npm run build

    Then use npm link to create a package that can be installed into IF:

    npm link

    Step 4: Load your plugin into IF​

    Now your plugin is ready to run in IF. First install your plugin by navigating to the if project folder and running:

    npm link new-plugin

    replacing new-plugin with your plugin name as defined in the plugin's package.json. If you are not sure, the name can be checked by running npm ls -g --depth=0 --link=true.

    Your plugin is now ready to be run in IF. All that remains is to add your plugin to your manifest file. This means adding it to the initialize block and adding it to the component pipelines where you want your plugin to be executed. For example, an initilize block might look as follows:

    initialize:
      plugins:
        new-plugin:
          method: YourFunctionName
          path: 'new-plugin'
          global-config:
            something: true

    Run your manifest uisng

    np run ie -- --manifest <path-to-manifest>

    If you have to link more than one local plugin, for example to test your plugin in a pipeline, you can do so with

    npm link new-plugin --save

    This will create an entry like "new-plugin": "file:path/to/your/plugin" in the package.json which links to your local plugin. This way, multiple plugins can be linked at once. Of course, these changes should not be committed, but they can be helpful for local testing.

    Step 5: Publishing your plugin​

    Now you have run your plugin locally and you are happy with how it works, you can make it public by publishing it to a public Github repository. Now all you have to do to use it in a manifest file is npm install it and pass the path to the Github repository in the plugin initialize block.

    For example, for a plugin saved in github.com/my-repo/new-plugin you can do the following:

    npm install https://github.com/my-repo/new-plugin

    Then, in your manifest file, provide the path in the plugin instantiation. You also need to specify which function the plugin instantiates. Let's say you are using the Sum plugin from the example above:

    name: plugin-demo
    description: loads plugin
    tags: null
    initialize:
      plugins:
        - name: new-plugin
          kind: plugin
          method: FunctionName
          path: https://github.com/my-repo/new-plugin
    tree:
      children:
        child:
          config:
          inputs:

    Now, when you run the manifest file, it will load the plugin automatically.

    You can run this using the globally installed IF as follows:

    ie --manifest <path-to-my-manifest>

    Summary of steps​

    • Copy our template repository and update package.json
    • Add your plugin code to index.ts
    • Build and link the plugin using npm run build && npm link
    • Load your plugin into if using npm link
    • Initialize your plugin and add it to a pipeline in your manifest file.
    • Publish your plugin to Github

    You should also create unit tests for your plugin to demonstrate correct execution and handling of corner cases.

    Next steps​

    You can read our more advanced guide on how to refine your plugins.

    Appendix: Walk-through of the Sum plugin​

    To demonstrate how to build a plugin that conforms to the pluginInterface, let's examine the sum plugin.

    The sum plugin implements the following logic:

    • sum whatever is provided in the input-parameters field from globalConfig.
    • append the result to each element in the output array with the name provided as output-parameter in globalConfig.

    Let's look at how you would implement this from scratch:

    The plugin must be a function conforming to PluginInterface. You can call the function Sum, and inside the body you can add the signature for the execute method:

    export const Sum = (globalConfig: SumConfig): PluginInterface => {
      const errorBuilder = buildErrorMessage(Sum.name);
      const metadata = {
        kind: 'execute',
      };

      /**
       * Calculate the sum of each input.
       */
      const execute = async (inputs: PluginParams[]): Promise<PluginParams[]> => {

      };

      return {
        metadata,
        execute,
      };

    }

    Your plugin now has the basic structure required for IF integration. Your next task is to add code to the body of execute to enable the actual plugin logic to be implemented.

    The execute function should grab the input-parameters (the values to sum) from globalConfig. it should then iterate over the inputs array, get the values for each of the input-parameters and append them to the inputs array, using the name from the output-parameter value in globalConfig. Here's what this can look like, with the actual calculation pushed to a separate function, calculateSum. 

      /**
       * Calculate the sum of each input.
       */
      const execute = async (inputs: PluginParams[]): Promise<PluginParams[]> => {
        const inputParameters = globalConfig['input-parameters'];
        const outputParameter = globalConfig['output-parameter'];

        return inputs.map(input => {
          return {
            ...input,
            [outputParameter]: calculateSum(input, inputParameters),
          };
        });

      return {
        metadata,
        execute,
      };
    }

    Now we just need to define what happens in calculateSum - this can be a simple reduce:

      /**
       * Calculates the sum of the energy components.
       */
      const calculateSum = (input: PluginParams, inputParameters: string[]) =>
        inputParameters.reduce(
          (accumulator, metricToSum) => accumulator + input[metricToSum],
          0
        );

    Note that this example did not include any validation or error handling - you will likely want to add some for a real plugin.

    Finally, if your plugin used any fields in inputs or created new outputs that have not been used in the Impact Framework before, then you should add them to params.ts.

    params.ts can be found in the path src/config.

    Each entry in params.ts looks as follows:

    carbon:
      description: an amount of carbon emitted into the atmosphere
      unit: gCO2e
      aggregation: sum

    This information allows IF to programmatically make decisions about how to handle values in features such as aggregation, time normalization and visualizations, and also acts as a global reference document for understanding IF data. The example above is for carbon.

    You should add your new data, give a name, define a unit and short description. The aggregation field determines how the value is treated when some manipulation has to be done to spread the value over time or aggregate it.

    For absolute metrics like carbon, the right value is sum because you would want to add carbon emissions from each timestep when you aggregate over time.

    For proportional metrics, the right value is avg. For example, you would want to calculate the average cpu/utilization - it would not make sense to sum it when aggregating over multiple timesteps.

    Finally, values that should always be presented identically regardless of any aggregation, such as names or global constants, should be given the aggregation-method value none.

    Now you are ready to run your plugin using the ie CLI tool!

    - + \ No newline at end of file diff --git a/developers/how-to-refine-plugins/index.html b/developers/how-to-refine-plugins/index.html index 02af68f8..94f2cfc1 100644 --- a/developers/how-to-refine-plugins/index.html +++ b/developers/how-to-refine-plugins/index.html @@ -8,7 +8,7 @@ How to make plugins production ready | Impact Framework - + @@ -18,7 +18,7 @@ Validate input parameters against expected types, ranges, or constraints to prevent runtime errors and ensure data consistency. We use zod to validate data. Here's an example from our codebase:

    /**
     * Checks for required fields in input.
     */
    const validateInput = (input: PluginParams) => {
      const schema = z
        .object({
          'cpu/name': z.string(),
        })
        .refine(allDefined, {
          message: '`cpu/name` should be present.',
        });

      return validate<z.infer<typeof schema>>(schema, input);
    }

    Code Modularity​

    Break down complex functionality into smaller, manageable methods with well-defined responsibilities. Encapsulate related functionality into private methods to promote code reusability and maintainability.

    3. Unit tests​

    Your plugin should have unit tests with 100% coverage. We use jest to handle unit testing. We strive to have one describe per function. Each possible outcome from each function is separated using it with a precise and descriptive message.

    Here's an example that covers plugin initialization and the happy path for the execute() function.

    import {Sum} from '../../../../lib';

    import {ERRORS} from '../../../../util/errors';

    const {InputValidationError} = ERRORS;

    describe('lib/sum: ', () => {
      describe('Sum: ', () => {
        const globalConfig = {
          'input-parameters': ['cpu/energy', 'network/energy', 'memory/energy'],
          'output-parameter': 'energy',
        };
        const sum = Sum(globalConfig);

        describe('init: ', () => {
          it('successfully initalized.', () => {
            expect(sum).toHaveProperty('metadata');
            expect(sum).toHaveProperty('execute');
          });
        });

        describe('execute(): ', () => {
          it('successfully applies Sum strategy to given input.', async () => {
            expect.assertions(1);

            const expectedResult = [
              {
                duration: 3600,
                'cpu/energy': 1,
                'network/energy': 1,
                'memory/energy': 1,
                energy: 3,
                timestamp: '2021-01-01T00:00:00Z',
              },
            ];

            const result = await sum.execute([
              {
                duration: 3600,
                'cpu/energy': 1,
                'network/energy': 1,
                'memory/energy': 1,
                timestamp: '2021-01-01T00:00:00Z',
              },
            ]);

            expect(result).toStrictEqual(expectedResult);
          });
        }
      })
    })

    We have a dedicated page explaining in more detail how to write great unit tests for Impact Framework plugins.

    4. Linting​

    We use ESLint to format our code. We use a very simple configuration file (eslintrc.json), as follows:

    {
      "extends": "./node_modules/gts/",
      "rules": {
        "@typescript-eslint/no-explicit-any": ["off"]
      }
    }

    For our repositories we use Github CI to enforce the linting rules for any pull requests.

    Summary​

    On this page, we have outlined best practices for refining your plugins so that they conform to our expected norms. This will help you write clean, efficient, and understandable code!

    - + \ No newline at end of file diff --git a/developers/how-to-visualize-results/index.html b/developers/how-to-visualize-results/index.html index 58ffe4be..81896496 100644 --- a/developers/how-to-visualize-results/index.html +++ b/developers/how-to-visualize-results/index.html @@ -8,14 +8,14 @@ How to visualize results | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    How to visualize results

    (for any questions / help needed on IF visualization please raise an issue here: IF issues)

    There are currently 2 ways to visualize Impact Framework outputs:

    1. Using the Simple HTML Exporter plugin.
    2. Using Grafana.

    Grafana is the more standardized method for visualization. It also provides much more control over what's being visualized and how. Nevertheless, it requires some setting up.

    Using the Simple HTML Exporter plugin is much simpler as it only requires a single configuration in the manifest yml file. However, it only supports visualization of Energy and Carbon values over time. If you need to visualize other output parameters you can either use Grafana or alternatively enhance the Simple HTML Exporter plugin to be a bit less simple...

    Simple HTML Exporter plugin​

    Input​

    energy (kWh), carbon (gCO2) and timestamp should be included in the input array items.

    Output​

    This plugin acts as a relay, returning the input as-is, with the generated HTML acting as a "side effect".

    Example manifest​

    name: simple-html-exporter-demo
    description:
    tags:
    initialize:
      plugins:
        'simple-html-exporter':
          method: SimpleHtmlExporter
          path: "@grnsft/if-plugins"
    tree:
      children:
        child:
          pipeline:
            - simple-html-exporter
          config:
            simple-html-exporter:
              html-path: /usr/local/data/html-export.html
          inputs:
            - timestamp: '2021-01-01T00:00:00Z',
              energy: 0.00001841,
              carbon: 0.0104062,

    Visualization example​

    The resulting HTML file can then be viewed using a web-browser

    img.png

    Grafana​

    (See also https://grafana.com/)

    Grafana is an open source analytics & monitoring solution for every database.

    One of its main features is the ability to create dashboards with various types of data visualizations.

    Please follow these instructions here to set up a Grafana dashboard.

    This method requires converting the resulting output yml into a CSV. The standard way to do so would be to use the CSV export plugin.

    Visualization example​

    img.png

    - + \ No newline at end of file diff --git a/developers/how-to-write-unit-tests/index.html b/developers/how-to-write-unit-tests/index.html index 562874ec..86427881 100644 --- a/developers/how-to-write-unit-tests/index.html +++ b/developers/how-to-write-unit-tests/index.html @@ -8,14 +8,14 @@ How to write unit tests | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    How to write unit tests

    Impact Framework unit tests follow a standard format. We use the jest testing library. You can run all our existing tests by opening the project directory and running npm test. This page explains how you can add new unit tests for your plugins (or add some for our plugins if you notice a gap).

    Test files​

    Both the IF and the project repositories include a __test__ directory. Inside, you will find subdirectory unit/lib containing directories for each plugin. Your plugin repository should also follow this structure. Inside the plugin directory you can add index.test.ts. This is where you write your unit tests. For example, here's the directory tree for our teads-curve test file:


    if-unofficial-plugins
     |
     |- src
        |
        |-__tests__
            |
            |-unit
              |
              |-lib
                 |
                 teads-curve
                     |
                     |- index.test.ts

    Setting up your test file​

    You will need to import your plugin so that it can be instantiated and tested. You will also need some elements from jest/globals: For example, these are the imports for our Sum plugin.

    import {Sum} from '../../../../lib';
    import {ERRORS} from '../../../../util/errors';
    const {InputValidationError} = ERRORS;

    You may require other imports for your specific set of tests.

    Describe​

    Each method should have its own dedicated describe block.

    Your unit tests should have at least two describe blocks, one to test the plugin initialization and one for execute.

    describe("init", ()=> {})
    describe("execute", ()=> {})

    For example, here is a describe block checking that the Sum plugin initializes correctly:

    describe('lib/sum: ', () => {
      describe('Sum: ', () => {
        const globalConfig = {
          'input-parameters': ['cpu/energy', 'network/energy', 'memory/energy'],
          'output-parameter': 'energy',
        };
        const sum = Sum(globalConfig);

        describe('init: ', () => {
          it('successfully initalized.', () => {
            expect(sum).toHaveProperty('metadata');
            expect(sum).toHaveProperty('execute');
          });
        });
      })
    })

    It​

    Within each describe block, each effect to be tested should have a dedicated it block.

    Here's an example of a new describe block for the execute() method on the Sum plugin. The describe block indicates that we are testing effects of the execute() method. it is specific to a single outcome - in this case there are two it blocks that test that the plugin returns a specific result in the happy path and throws an exception if the user has provided invalid config data, specifically that the user-provided cpu/energy parameter is missing:

        describe('execute(): ', () => {
          it('successfully applies Sum strategy to given input.', async () => {
            expect.assertions(1);

            const expectedResult = [
              {
                duration: 3600,
                'cpu/energy': 1,
                'network/energy': 1,
                'memory/energy': 1,
                energy: 3,
                timestamp: '2021-01-01T00:00:00Z',
              },
            ];

            const result = await sum.execute([
              {
                duration: 3600,
                'cpu/energy': 1,
                'network/energy': 1,
                'memory/energy': 1,
                timestamp: '2021-01-01T00:00:00Z',
              },
            ]);

            expect(result).toStrictEqual(expectedResult);
          });

          it('throws an error on missing params in input.', async () => {
            const expectedMessage =
              'Sum: cpu/energy is missing from the input array.';

            expect.assertions(1);

            try {
              await sum.execute([
                {
                  duration: 3600,
                  timestamp: '2021-01-01T00:00:00Z',
                },
              ]);
            } catch (error) {
              expect(error).toStrictEqual(
                new InputValidationError(expectedMessage)
              );
            }
          });
        })

    Errors​

    We prefer to use expect to check the errors returned from a test. We do this by writing expect in a catch block. Here's an example from our sci plugin tests:

    it('throws an exception on missing functional unit data.', async () => {
      const inputs = [
        {
          timestamp: '2021-01-01T00:00:00Z',
          'operational-carbon': 0.002,
          'embodied-carbon': 0.0005,
          'functional-unit': 'requests',
          duration: 1,
        },
      ];
      expect.assertions(1);

      try {
        await sciModel.execute(inputs);
      } catch (error) {
        expect(error).toBeInstanceOf(InputValidationError);
      }
    });

    It is also necessary to include expect.assertions(n) for testing asynchronous code, where n is the number of assertiosn that should be tested before the test completes.

    Mocks​

    Please try to avoid mocking data if possible. However, if it is necessary to mock (e.g. if your plugin relies on a third party credentialed API) then please make your mock data as realistic as possible (no foo, bar, baz style mock data, please).

    We do have mock backends in several of our tests, and we also have a mock data generator plugin that can create realistic dummy data to your specific requirements.

    Coverage​

    Please use jest --coverage to see a coverage report for your plugin - your unit tests should yield 100% coverage. The snippet below shows what to expect from the coverage report:

    -------------------------------|---------|----------|---------|---------|-------------------
    | File                        | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s |
    | --------------------------- | ------- | -------- | ------- | ------- | ----------------- |
    | All files                   | 100     | 100      | 100     | 100     |
    | lib                         | 100     | 100      | 100     | 100     |
    | index.ts                    | 100     | 100      | 100     | 100     |
    | lib/cloud-metadata          | 100     | 100      | 100     | 100     |
    | index.ts                    | 100     | 100      | 100     | 100     |
    | lib/e-mem                   | 100     | 100      | 100     | 100     |
    | index.ts                    | 100     | 100      | 100     | 100     |
    | lib/e-net                   | 100     | 100      | 100     | 100     |
    | index.ts                    | 100     | 100      | 100     | 100     |
    - + \ No newline at end of file diff --git a/developers/index.html b/developers/index.html index 414c4151..3ac2d533 100644 --- a/developers/index.html +++ b/developers/index.html @@ -8,13 +8,13 @@ Developers | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    Developers

    This section contains information for Impact Framework developers. You are a developer if you want to change or update the Impact Framework by adding new features, fixing bugs or building new plugins.

    The developer documentation includes:

    If you are looking for guidance for how to use IF to measure the environmental impact of your apps, you should go to our user documentation instead.

    - + \ No newline at end of file diff --git a/index.html b/index.html index 6ad02fe6..47024eb6 100644 --- a/index.html +++ b/index.html @@ -8,13 +8,13 @@ Welcome to Impact Framework | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    Impact Framework

    Impact Framework is a way to compute and report the environmental impacts of software applications accurately.

    Read The DocsJoin the IF Hackathon

    Join an upcoming
    1hr live training for Impact Framework

    Sign up for a 1-hour Impact Framework demo to improve how you measure software to reduce its environmental impact.

    Wednesday, March 6 at 9:30 AM UTC

    Wednesday, March 6 at 4:30 PM UTC

    Wednesday, March 13 at 9:30 AM UTC

    Wednesday, March 13 at 4:30 PM UTC

    Transform Observations into Impacts

    Take easily observable metrics like CPU utilization, page views and installs and convert them into environmental impacts such as carbon emissions, water usage, energy consumption, and air quality.

    Buildable Plugin Ecosystem

    Engage an existing library of plugins or create new plugins within Impact Framework based on your assumptions and observations.

    Explore What-If Scenarios

    Uncover the environmental impact of your software changes in real-time by examining how your software’s environmental performance changes if your application moves to the cloud or experiences shifts in its runtime.

    Decentralize Data

    Record your observations, chosen plugins, configurations, and computed environmental impacts in a manifest file. Open the door for others to understand, verify, and challenge the entire process.

    Democratize Measurement

    Empower others to rerun your manifest file, validate your findings, or question your assumptions. They can tweak configurations, select different plugins, and run the analysis themselves.

    - + \ No newline at end of file diff --git a/intro/index.html b/intro/index.html index aec4e04c..a720998c 100644 --- a/intro/index.html +++ b/intro/index.html @@ -8,14 +8,14 @@ Introduction | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    Introduction

    Hackathon!​

    From March 18 to April 8, 2024, participants will compete to showcase their best application of IF in measuring the environmental impacts of software.

    Carbon Hack is a dynamic competition that combines healthy rivalry with collaborative innovation. Hackers will push the limits of the framework, uncover potential weaknesses, and create innovations to enhance the tool.

    CarbonHack is open to all, including software practitioners and those with a passion for Green Software. Find out more about CarbonHack 2024 on the CarbonHack website

    Registration opens 22nd January!


    Impact Framework​

    Impact Framework (IF) aims to make the environmental impacts of software easier to calculate and share.

    IF allows you to calculate the environmental impacts, such as carbon, of your software applications without writing any code. All you have to do is write a simple manifest file and IF handles the rest.

    The project is entirely open source and composability is a core design principle - we want you to be able to create your own plugins and plug them in to our framework, or pick from a broad universe of open source plugins created by others.

    Motivation​

    If you can't measure, you can't improve. Software has many negative environmental impacts which we need to optimize, carbon, water, and energy, to name just a few.

    Unfortunately, measuring software impact metrics like carbon, water, and energy is complex and nuanced.

    Modern applications are composed of many smaller pieces of software (components) running on different environments, for example, private cloud, public cloud, bare-metal, virtualized, containerized, mobile, laptops, desktops, embedded, and IoT. Many components that make up a typical software application are run on something other than resources you own or control, which makes including the impact of managed services in your measurement especially hard.

    The impacts of software components also vary over time, so as well as understanding which components contribute most to the overall impacts, there is also a question of when they contribute the most.

    Only through a granular analysis of the impacts of your software system can investments in reducing its impact be prioritized and verified. Measurement is the first and most crucial step in greening a software system, and the first step in that process with the Impact Framework is to create a tree.

    Background​

    This project has evolved over the two years of the GSF's existence.

    During the development of the SCI, we acknowledged that the biggest blocker to adoption was data regarding the emissions of software components on different platforms and runtimes.

    We then launched the sci-data project to help create the data sets required to calculate an SCI score.

    After some investigation, the original sci-data team quickly realized that there were several existing data sources, and many more were in development, free open source or private commercial. The future challenge wouldn't be to source them, it would be knowing which data set to use for which use case, how data sets differed in their methodology and interface and when to use one over the other, the pros/cons, and trade-offs.

    The project evolved into the sci-guide to document existing data sets, providing guidance for when to use one over another and how to use it to create your own software measurement reports.

    Finally, we had enough information, and SCI case studies started to be written. This was a milestone moment.

    But now we are in the next evolution, to have software measurement be a mainstream activity. For this to be an industry with thousands of professionals working to decarbonize software, for businesses to grow and thrive in a commercial software measurement ecosystem, we need to formalize software measurement into a discipline with standards and tooling. The SCI Specification is the standard, and the Impact Framework is the tooling.

    Project Structure​

    The IF source code can be found in the IF Github repository. The code there covers the framework, which includes all the infrastructure for reading and writing input and output yamls, invoking plugins, running the command line tool and associated helper functions. However, it does not include the actual plugins themselves. Part of the IF design philosophy is that all plugins should be plugins, so that the IF is as composable and configurable as possible. Therefore, to use IF, you have to either create your own plugins or find some prebuilt ones and install them yourself. This also implies that you take responsibility for the plugins you choose to install.

    We do provide a standard library of plugins built and maintained by the IF core team. These can be found in the if-plugins Github repository. You can install these into if by running npm install https://github.com/Green-Software-Foundation/if-plugins from the if project directory.

    There is also a second repository for plugins we expect community members to maintain. These can be found in the if-unofficial-plugins Github repository. You can install these into if by running npm install https://github.com/Green-Software-Foundation/if-unofficial-plugins from the if project directory.

    Finally, the source code for this documentation website is available at the if-docs Github repository.

    The lefthand sidebar contains links to all the information you need to understand Impact Framework.

    You can explore the key ideas underpinning Impact Framework in the Major Concepts section.

    Users can read our guides explaining how to use IF, including installation, using the CLI and loading plugins.

    We also have developer documentation to help you get started building with IF.

    You will find documentation for the individual built-in plugin implementations in plugins.

    - + \ No newline at end of file diff --git a/major-concepts/aggregation/index.html b/major-concepts/aggregation/index.html index 473f56df..36071d94 100644 --- a/major-concepts/aggregation/index.html +++ b/major-concepts/aggregation/index.html @@ -8,13 +8,13 @@ Aggregation | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    Aggregation

    Aggregation is the process of summarizing a set of metrics.

    Two types of aggregation can be executed in IF. The first takes a time series and condenses it down into a single number representing an entire observation period, for example, if your time series contains three timesteps, the value of the metric being aggregated is 1 in each timestep and the aggregate is being computed as a sum, the aggregated value for the whole observation period is 3. We refer to this as "time series aggregation" or "horizontal aggregation".

    The second type of aggregation happens across components in a tree. Where time series exist for multiple child nodes under a parent, their time series are aggregated together into one summary time series that is pushed to the parent node. For example, in the following tree, each child has a time series with three timesteps. At each timestep, the metric value is 1 for both children:

             parent
            /      \
           /        \
    child-1          child-2
    data: [1,1,1]    data: [1,1,1]

    Assuming the aggregation method is sum, the parent would receive an aggregated time series [2,2,2], representing the aggregated values from both children. We refer to this as "tree aggregation" or "vertical aggregation".

    Configuration​

    Aggregate is a built-in feature of the IF. This means you do not need to initialize it along with the plugins you are using in your pipeline. All you need to do is to add a small piece of config to your manifest file.

    The aggregate config looks as follows:

    aggregation:
      metrics:
        - "carbon"
        - "energy"
      type: "both"

    There are two fields: metrics and type.

    metrics is an array of metrics that you want to aggregate. You can provide any value here, but they must match a key that exists in your output data (i.e. if you tell IF to aggregate carbon but carbon is not in your outputs you will receive an error message and aggregation will fail). You can provide any number of metrics. In the example above, the aggregation feature will operate on the carbon and energy values.

    type determines which kind of aggregation you want to perform. The choices are horizontal (time-series aggregation only), vertical (tree aggregation only) or both (both kinds of aggregation will be performed). In the example above, both types of aggregation will be performed over the two selected metrics.

    Aggregation methods​

    Aggregation can happen in several ways. In the example above, both of the selected metrics are absolute values measured in gCO2eq (carbon) and kWh (energy). To aggregate, it makes sense to add the values in each timestep for time-series aggregation and element-wise across components for tree aggregation. However, summing the values is not always the right way to aggregate.

    For example, some values are constants that apply to every timestep - these should simply be copied into the aggregated metric and not summed. For example, if you are using a hard-coded value for grid carbon intensity that applies globally, then you simply want to persist that value into the aggregate - adding them together would provide a misleading result.

    Similarly, some values are proportions or percentages. In these cases, the right way to aggregate is usually to take an average rather than summing over a set of values.

    The decisions about how to aggregate are made on a case-by-case basis for each individual parameter. We track the aggregation method for each parameter in our params.ts file, which contains the canonical set of parameters, their units and aggregation methods. You can append to the parameters in this file by providing params data in your manifest file, or you can override our recommended set of parameters entirely by providing a new file to the --override-params command in the CLI. In either case, you need to provide an aggregation method with your parameters so that IF can look up the right way to aggregate the values.

    Aggregation outputs​

    The aggregation process adds new output data to your manifest file. The two types of aggregation add different outputs. The horizontal (time-series) aggregation adds a new field called aggregated to each node whose time series has been aggregated. In the aggregated block, you will find the aggregated value for each of the aggregation metrics defined in the aggregation config.

    The vertical aggregation adds a new array of output observations. These are simply named outputs and they always contain a timestamp and duration along with the aggregated metrics for each timestep.

    The example below shows the result of running both kinds of aggregation for a single component:

       "outputs": [
          {
            "carbon": 0.04846481793320214,
            "energy": 0.00030285447154471535,
            "timestamp": "2023-12-12T00:00:00.000Z"
          },
          {
            "carbon": 0.037777724630840566,
            "energy": 0.00023606013840495548,
            "timestamp": "2023-12-12T00:00:05.000Z"
          },
          {
            "carbon": 0.03630388278027921,
            "energy": 0.000226848626838947,
            "timestamp": "2023-12-12T00:00:10.000Z"
          },
          {
            "carbon": 0.0360935970659935,
            "energy": 0.0002255343411246613,
            "timestamp": "2023-12-12T00:00:15.000Z"
          },
          {
            "carbon": 0.0360935970659935,
            "energy": 0.0002255343411246613,
            "timestamp": "2023-12-12T00:00:20.000Z"
          },
          {
            "carbon": 0.0360935970659935,
            "energy": 0.0002255343411246613,
            "timestamp": "2023-12-12T00:00:25.000Z"
          },
          {
            "carbon": 0.0360935970659935,
            "energy": 0.0002255343411246613,
            "timestamp": "2023-12-12T00:00:30.000Z"
          },
          {
            "carbon": 0.0360935970659935,
            "energy": 0.0002255343411246613,
            "timestamp": "2023-12-12T00:00:35.000Z"
          },
        ],
        "aggregated": {
          "carbon": 0.3246705689138855,
          "energy": 0.0020287555470867216
        }
    - + \ No newline at end of file diff --git a/major-concepts/design-philosophy/index.html b/major-concepts/design-philosophy/index.html index 18443b09..97ef7213 100644 --- a/major-concepts/design-philosophy/index.html +++ b/major-concepts/design-philosophy/index.html @@ -8,13 +8,13 @@ Design philosophy | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    Design philosophy

    Transparency​

    The manifest file is the lifeblood of Impact Framework. It defines all the context for an environmental impact calculation, defining the architecture of an application, the observation period, the pipeline of calculations and transformations to execute, and the environmental impacts to track. This document can then be executed to generate impact values. This gives unparalleled transparency to an environmental audit, all in a standard, easy to read format that anyone can read or re-execute.

    Verifiability​

    An Impact Framework manifest file is powerful because anyone can re-execute it and verify an organization's impact calculation. You can even experiment by swapping out different plugins. The critical concept is that everything you need to calculate an impact is provided in the manifest file and anyone can re-run a calculation with the manifest file and the lightweight Impact Framework command line tool.

    Flexibility

    We aim to bake the minimum of constraints into the Imapct Framework, balancing the helpers and standards that make plugins interoperable and consistent against freedom of expression and creativity. The real power of Impact Framework comes from the community. This includes the community of experts contributing to the design decisions and standards baked into the protocol and the community of plugin developers experimenting at the margins, and the organizations using Impact Framework to measure, report and mitigate their environmental imapct.

    Impact Framework can be a tool for transparent, verifiable environmental impact audits, but it can also be a platform for experimentation. Your manifest file is a foundation for forecasting into the future or exploring where you can tweak your stack to most effectively minimize your impact. Impact Framework can be a tool for research, hypothesis testing, R&D and business decision making as well as environemntal reporting. To realize this vision, we know we have to make Impact Framework as flexible as possible, imposing the absolute minimum of constraints in the underlying protocol, while also providing the necessary functionality and safeguards our users require.

    Modularity​

    Impact Framework is the minimal set of features that enable a manifest file to be processed according to some agreed principles. We provide a tool for processing manifest files and a set of standards and norms. This allows builders to create plugins that do some specific task, such as grabbing data from a particular cloud provider, or applying some calculation over some particular data.

    Anyone can build a plugin and share them with the world, meaning Impact Framework development can be bottom-up and community driven. It also means that if you are not satisfied with how some calculation was done, you can easily fork it and replace it with your own.

    What we provide is a minimal set of rules and guardrails for plugin builders to conform to to ensure compatibility with Impact Framework.

    Neutrality​

    Impact Framework aims to support maximally decentralized plugin development. We want anyone to be able to build plugins and use them to calculate their environmental impacts. We do not want to gatekeep what people can measure and monitor - we want to encourage people to build freely and experiment on Impact Framework rails!

    At the same time, we want to provide the helpers and guardrails that make impact calculations as friction free as possible. This means we focus on providing the minimal protocol required to support community plugin development and make it as safe as possible from unit errors and other footguns.

    We want to see the universe of Impact Framework plugins grow organically and permissionlessly in ways we can't even imagine today!

    To this end, what we are really building is a protocol. Impact Framework is just a Typescript implementation of the protocol. The protocol itself is a set of fundamental principles that define how a manifest file should be processed, such that any implementation in any language will yield the same result from a given manifest file.

    The Impact Protocol is the result of countless discussions, experiments, conversations with industry partners, academics, researchers and developers, and represents a community consensus for how certain actions should be executed, for example, how should a series of observations be aggregated, what standard units should be used, how should a manifest file be structured, etc.

    This means we can be neutral about what can be built with IF while also providing a set of canonical processes and standards.

    Where to go next​

    This page has outlined the design philosophies that guide Impact Framework development.

    Explore the other pages in this section to see how these principles have been applied to specific Impact Framework features, or head to our user documentation to get started running Impact Framework for yourself.

    - + \ No newline at end of file diff --git a/major-concepts/groupby/index.html b/major-concepts/groupby/index.html index d8df7876..ecf742d1 100644 --- a/major-concepts/groupby/index.html +++ b/major-concepts/groupby/index.html @@ -8,14 +8,14 @@ Group-by | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    Group-by

    Groupby is an IF plugin that reorganizes a tree according to keys provided by the user. This allows users to regroup their observations according to various properties of their application. For example, the following manifest file contains a flat array of observations. This is how you might expect data to arrive from an importer plugin, maybe one that hits a metrics API for a cloud service.

    name: if-demo
    description: demo pipeline
    graph:
      children:
        my-app:
          pipeline:     
            - group-by
            - teads-curve
          config:
            group-by:
              - cloud-region
              - instance-type
          inputs:
            - timestamp: 2023-07-06T00:00
              duration: 300 
              instance-type: A1 
              region: uk-west
              cpu-util: 99
            - timestamp: 2023-07-06T05:00 
              duration: 300 
              instance-type: A1 
              region: uk-west
              cpu-util: 23    
            - timestamp: 2023-07-06T10:00
              duration: 300
              instance-type: A1 
              region: uk-west
              cpu-util: 12
            - timestamp: 2023-07-06T00:00 # note this time restarts at the start timstamp
              duration: 300 
              instance-type: B1
              region: uk-west
              cpu-util: 11
            - timestamp: 2023-07-06T05:00 
              duration: 300 
              instance-type: B1
              region: uk-west
              cpu-util: 67
            - timestamp: 2023-07-06T10:00
              duration: 300 
              instance-type: B1
              region: uk-west
              cpu-util: 1

    However, each observation contains an instance-type field that varies between observations. There are two instance types being represented in this array of observations. This means there are duplicate entries for the same timestamp in this array. This is the problem that group-by solves. You provide instance-type as a key to the group-by plugin and it extracts the data belonging to the different instances and separates them into independent arrays. The above example would be restructured so that instance types A1 and B1 have their own data, as follows:

    graph:
      children:
        my-app:
          pipeline:
            # - group-by
            - teads-curve
          config:
            group-by:
              groups:
                - cloud-region
                - instance-type
          children:
            A1:
                inputs:
                - timestamp: 2023-07-06T00:00
                    duration: 300
                    instance-type: A1
                    region: uk-west
                    cpu-util: 99
                - timestamp: 2023-07-06T05:00
                    duration: 300
                    instance-type: A1
                    region: uk-west
                    cpu-util: 23
                - timestamp: 2023-07-06T10:00
                    duration: 300
                    instance-type: A1
                    region: uk-west
                    cpu-util: 12
            B1:
                inputs:
                - timestamp: 2023-07-06T00:00
                    duration: 300
                    instance-type: B1
                    region: uk-east
                    cpu-util: 11
                - timestamp: 2023-07-06T05:00
                    duration: 300
                    instance-type: B1
                    region: uk-east
                    cpu-util: 67
                - timestamp: 2023-07-06T10:00
                    duration: 300
                    instance-type: B1
                    region: uk-east
                    cpu-util: 1

    Using group-by​

    To use group-by, you have to initialize it as a plugin and invoke it in a pipeline.

    The initialization looks as follows:

    initialize:
    plugins:
    group-by: 
        path: 'builtin'
        method: GroupBy

    You then have to provide config defining which keys to group by in each component. This is done at the component level (i.e. not global config). For example:

    tree:
      children:
        my-app:
          pipeline:     
            - group-by
          config:
            group-by:
              group:
                - region
                - instance-type

    In the example above, the plugin would regroup the input data for the specific component by region and by instance-type.

    Assuming the values A1 and B1 are found for instance-type and the values uk-east and uk-west are found for region, the result of group-by would look similar to the following:

    tree:
      children:
        my-app:
          pipeline:
            - group-by
          config:
            group-by:
              groups:
                - region
                - instance-type
          children:
            uk-west:
              children:
                A1:
                  inputs:
                    - timestamp: 2023-07-06T00:00
                      duration: 300
                      instance-type: A1
                      region: uk-west
                      cpu-util: 99
                    - timestamp: 2023-07-06T05:00
                      duration: 300
                      instance-type: A1
                      region: uk-west
                      cpu-util: 23
                    - timestamp: 2023-07-06T10:00
                      duration: 300
                      instance-type: A1
                      region: uk-west
                      cpu-util: 12
            uk-east:
              children:
                B1:
                  inputs:
                    - timestamp: 2023-07-06T00:00
                      duration: 300
                      instance-type: B1
                      region: uk-east
                      cpu-util: 11
                    - timestamp: 2023-07-06T05:00
                      duration: 300
                      instance-type: B1
                      region: uk-east
                      cpu-util: 67
                    - timestamp: 2023-07-06T10:00
                      duration: 300
                      instance-type: B1
                      region: uk-east
                      cpu-util: 1

    This reorganized data can then be used to feed the rest of a computation pipeline.

    - + \ No newline at end of file diff --git a/major-concepts/if/index.html b/major-concepts/if/index.html index c29d7052..7e1eb7dc 100644 --- a/major-concepts/if/index.html +++ b/major-concepts/if/index.html @@ -8,14 +8,14 @@ Impact Engine (CLI) | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    Impact Engine (CLI)

    Introduction​

    ie is a command line tool that computes Manifest files. It is the portal allowing users to interact with the Impact Framework.

    The available options are:

    • --manifest: path to an input manifest file
    • --output (optional): path to the output file where the results as saved, if none is provided it prints to stdout.
    • --override-params (optional): if you are an advanced user and you want to override our standard set of parameters and their definitions, you can provide the path to an alternative file as an argument to this command.

    The only required command is --manifest. Without a valid path to a manifest file, ie has nothing to execute.

    To use ie, you must first write a manifest file. Then, you can simply pass the path to the manifest file to ie on the command line. 

    ie --manifest /my-manifest.yml

    You can also pass a path where you would like to save the output file to. For example:

    ie --manifest ./my-manifest.yml --output ./my-results.yml

    Note that you also need to add some config to your manifest file to enable exporting to a file. The config is as follows:

    initialize:
      outputs:
        - yaml

    If you omit the --output command, your results will be displayed in the console.

    For more information on the ie commands see the CLI reference documentation.

    - + \ No newline at end of file diff --git a/major-concepts/index.html b/major-concepts/index.html index b72fa567..3a328c7d 100644 --- a/major-concepts/index.html +++ b/major-concepts/index.html @@ -8,13 +8,13 @@ Major Concepts | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.
    - + \ No newline at end of file diff --git a/major-concepts/manifest-file/index.html b/major-concepts/manifest-file/index.html index 36792192..c5c510f2 100644 --- a/major-concepts/manifest-file/index.html +++ b/major-concepts/manifest-file/index.html @@ -8,14 +8,14 @@ Manifest File | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    Manifest File

    Manifest files are fundamental to Impact Framework and they serve multiple important purposes, including:

    • They contain all the necessary configurations for Impact Framework
    • They define your application architecture
    • They hold your input data
    • They are shareable, portable and human-readable
    • They can be used as verifiable audits form your application

    The manifest is a yaml file with a particular structure. It can be thought of as an executable audit because the file itself can be shared with others and re-executed to verify your environmental impact calculations.

    It is a formal report detailing not just the end impact but all the assumptions, inputs, and plugins used in calculating the impact.

    This is possible because all the configuration and data required to run Impact Framework is contained in the manifest file.

    Anyone can download Impact Framework and execute a manifest file to verify the results.

    Structure of a manifest file​

    Overview​

    Manifest files can be simple or very intricate, depending on the plugin pipeline you want to use and the complexity of your application. However, all manifest files conform to a basic structure that looks as follows:

    name:
    description:
    tags:
    initialize:
      plugins:
        <PLUGIN-NAME-HERE>: 
          method: 
          path: 
      outputs:
    tree:
      children:
        child:
          pipeline:
          config:
          defaults:
          inputs:
            - timestamp: 2023-08-06T00:00
              duration: 3600

    Global metadata​

    The global metadata includes the name, description, and tags that can be used to describe the nature of the manifest file. For example, you might name the file Carbon Jan 2024 or similar. A short description might briefly outline the scope of the manifest file, e.g. company x's carbon emissions due to web serves from Jab 24 - July 24. Tags can be used to group manifest files (we do not explicitly use this field for anything currently).

    Plugin initialization​

    The initialize section is where you define which plugins will be used in your manifest file and provide the global configuration for them. Below is sample for initialization: 

    initialize:
      plugins:
        <PLUGIN-NAME-HERE>:
          method: <METHOD-NAME-HERE>
      outputs: ['csv', 'yaml', 'log']

    Where required values are:

    • method: the name of the function exported by the plugin.
    • path: the path to the plugin code. For example, for a plugin from our standard library installed from npm, this value would be @grnsft/if-plugins

    There is also an optional global-config field that can be used to set global configuration that is common to a plugin wherever it is invoked across the entire manifest file.

    Impact Framework uses the initialize section to instantiate each plugin. A plugin cannot be invoked elsewhere in the manifest file unless it is included in this section.

    outputs is a list of possible export types (currently csv, yaml, and log are supported).

    Tree​

    The tree section of a manifest file defines the topology of all the components being measured. The shape of the tree defines the grouping of components. It describes the architecture of the application being studied and contains all the usage observations for each component. The tree has individual components such as leaves, intermediate nodes representing groupings, and the top level is the root.

    For example, a web application could be organized as follows:

    tree:
      children:
        front-end:
          children:
            build-pipeline:
              children:
                vercel:
                github-pages:
        backend-database:
          children:
            server1:
            server2:
            server3:
        front-end:
        networking:

    This example has a relatively straightforward structure with a maximum of 3 levels of nesting. You can continue to nest components to any depth.

    Each component has some configuration, some input data, and a plugin pipeline.

    • pipeline: a list of plugins that should be executed for a specific component
    • config: contains configuration for each plugin that applies just inside this specific component.
    • defaults: fallback values that IF defaults to if they are not present in an input observation.
    • inputs: an array of observation data, with each observation containing usage data for a given timestep.

    If a component does not include its own pipeline, config, or inputs values, they are inherited from the closest parent.

    Here's an example of a moderately complex tree:

    tree:
      children:
        child-0:
          pipeline:
            - sci-e
          children:
            child-0-1:
              pipeline:
                - sci-e
              config: null
              defaults: null
              inputs:
                - timestamp: 2023-07-06T00:00
                  duration: 10
                  cpu-util: 50
                  energy-network: 0.000811
              outputs:
                - timestamp: 2023-07-06T00:00
                  duration: 10
                  cpu-util: 50
                  energy-network: 0.000811
                  energy: 0.000811
            child-0-2:
              children:
                child-0-2-1:
                  pipeline:
                    - sci-e
                  config: null
                  defaults: null
                  inputs:
                    - timestamp: 2023-07-06T00:00
                      duration: 10
                      cpu-util: 50
                      energy-network: 0.000811
                  outputs:
                    - timestamp: 2023-07-06T00:00
                      duration: 10
                      cpu-util: 50
                      energy-network: 0.000811
                      energy: 0.000811

    Inputs​

    Every component includes an inputs field that gets read into plugins as an array. inputs are divided into observations, each having a timestamp and a duration. Every observation refers to an element in inputs representing some snapshot in time.

    Each plugin takes the inputs array and applies some calculation or transformation to each observation in the array.

    Observations can include any type of data, including human judgment, assumptions, other plugins, APIs, survey data or telemetry.

    The separation of timestamps in the inputs array determines the temporal granularity of your impact calculations. The more frequent your observations, the more accurate your impact assessment.

    Computing a manifest file​

    Impact Framework computes manifest files. For each component in the tree, the inputs array is passed to each plugin in the pipeline in sequence.

    Each plugin enriches the inputs array in some specific way, typically by adding a new key-value pair to each observation in the array. For example, the teads-curve plugin takes in CPU utilization expressed as a percentage as an input and appends cpu/energy expressed in kWh. cpu/energy is then available to be passed as an input to, for example, the sci-e plugin.

    This implies a sequence of plugins where the inputs for some plugins must either be present in the original manifest file or be outputs of the preceding plugins in the pipeline.

    There are also plugins and built-in features that can synchronize time series of observations across an entire tree and aggregate data across time or across components.

    Outputs​

    When Impact Framework computes a manifest file, it appends new data to the manifest file and the final result is an enriched manifest that includes all the configuration and contextual data, the input data, and the results of executing each plugin. This means the output file is completely auditable - the manifest file can be recovered simply by deleting the outputs section of the output file.

    Here's an example output file:

    name: e-mem
    description: null
    tags: null
    initialize:
      plugins:
        e-mem:
          path: "@grnsft/if-plugins"
          method: EMem
    tree:
      children:
        child:
          pipeline:
            - e-mem
          config: null
          defaults:
          inputs:
            - timestamp: 2023-08-06T00:00
              duration: 3600
              memory/utilization: 40
              memory/capacity: 1
          outputs:
            - timestamp: 2023-08-06T00:00
              duration: 3600
              mem-util: 40
              memory/capacity: 1
              memory/energy: 0.15200000000000002
    - + \ No newline at end of file diff --git a/major-concepts/parameters/index.html b/major-concepts/parameters/index.html index ed5e5b6a..a4c352f6 100644 --- a/major-concepts/parameters/index.html +++ b/major-concepts/parameters/index.html @@ -8,13 +8,13 @@ Parameters | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    Parameters

    Parameters are handled carefully in IF because of the risk of errors propagating through plugin pipelines. We have a set of canonical parameters that are stored in a file called params.ts in the IF repository.

    Each entry in params.ts has the following structure:

    name: {
    description: 
    unit: 
    aggregation:
    }

    The name is the accepted name for a particular parameter. For example, IF has a parameter, carbon that represents carbon in gCO2eq. Anywhere carbon exists as a parameter anywhere across the IF and its plugins, the name carbon is reserved for carbon with this specific unit.

    The description is a short sentence describing the parameter.

    The unit is the unit the parameter is expected to be expressed in. Prescribing this here is important because it allows everyone to know what units to expect from a plugin that returns this parameter, protecting against unit errors in pipelines.

    The aggregation parameter can accept sum, avg or none. This is the method IF should use to aggregate values over time or across a tree. Some values should be summed to aggregate them over time, such as carbon or energy - it makes sense to add these up as they are absolute values that can be added up over time or across components. However, it does not make sense to aggregate metrics expressed as percentages or proportions - these cases are better aggregated by averaging their values. Some other values should not be manipulated when they are aggregated and instead they are treated as constants that should simply be copied into the aggregate, such as names.

    Adding parameters​

    The canonical set of parameters protects users against introducing unit errors into plugin pipelines by asserting the units and aggregation methods associated with a specific named parameter. However, this also means that if you create a new plugin that deals with a parameter not currently available in params.ts, IF with throw an error. To handle this, you can add your plugin's new parameters to a block of config in the manifest file, and they will be added to the canonical set of parameters at runtime. This effectively gives you read-only access to params.ts via the manifest file.

    Here's an example of the config block that could be added to a manifest to enable a plugin that uses the dummy parameter:

    params:
      - name: 
        description:
        unit:
        aggregaton:

    Later, you could raise a pull request to IF to add your parameters to the canonical set in params.ts, where it would be discussed and agreed among the community. Eventually, we would like to move the governance of params.ts to a separate repository with independent custodians and a dedicated discussion forum.

    Overriding parameters​

    If you want to, you can also reject our canonical set of parameters and use your own instead. We do not recommend this, but we do make it possible via a CLI command, override-params. In this case, you can provide the path to a new file, structured the same way as params.ts that will be used instead of ours. For example:

    ie --manifest <manifest> --override-params my-params.ts

    The file must parse as JSON or a Javascript object.

    - + \ No newline at end of file diff --git a/major-concepts/plugins/index.html b/major-concepts/plugins/index.html index 07367813..7fdffa6e 100644 --- a/major-concepts/plugins/index.html +++ b/major-concepts/plugins/index.html @@ -8,13 +8,13 @@ Plugins | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    Plugins

    Plugins are self-contained units of code that do one thing. They can be loaded into IF and chained together in a pipeline so that simple individual plugins can form a complicated procedure for computing a manifest file.

    What are plugins?​

    A plugin converts an input into some output, for example, some plugins convert an input of CPU utilization into an output of energy.

    There are many different plugins: Boavizta, Cloud Carbon Footprint, Climatiq are some great examples of open-source IMs, there are many other closed source, commercial and private plugins being built in-house inside organizations.

    The set of plugins is increasing; however, no single plugin can cover all impacts, scenarios, environments, contexts, and use cases. To calculate the end-to-end impact of a software application, you need to stitch together many different plugins. Plugins differ in fundamental ways in the inputs inputs they accept, their interface, their calculation methodology, their outputs, their granularity, and their coverage.

    We expect the choice of which plugin to use for which software component to come down to an expert decision by a green software professional.

    Why do we need to standardize the interface to plugins?​

    Currently, suppose you want to consume a plugin in your measurement application. In that case, you must craft custom code to interact with a custom interface since every plugin has its unique interface. Swapping one plugin for another requires code changes, and comparing plugins or validating their accuracy/precision is challenging.

    If every plugin exposes the same interface, then those plugins can easily be plugged into different applications, swapped in and out, upgraded, and compared.

    Our thesis is simple: if we want a large, vibrant ecosystem of people and tooling around measurement, we need a standard, common interface for all plugins.

    Ecosystems grow and thrive through standards.

    You can explore more in the plugins reference docs or our plugin building tutorial.

    - + \ No newline at end of file diff --git a/major-concepts/time/index.html b/major-concepts/time/index.html index 19a312d2..8b1fd8d9 100644 --- a/major-concepts/time/index.html +++ b/major-concepts/time/index.html @@ -8,13 +8,13 @@ Time | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    Time

    Every observation in an array of inputs represents a snapshot with a known start time and a known duration. For example, the following observation shows that the CPU utilization for a resource was 20% for the 10 second period starting at 1500 on the 22nd January 2024:

    inputs:
        - timestamp: 2024-01-15T00:00:00.000Z
          duration: 10
          cpu-util: 20

    The total time covered by an inputs array is determined by the timestamp of the first observation in the array and the timestamp and duration in the last observation in the array. Since every observation needs both a timestamp and a duration, an inputs array is always a time series.

    Synchronizing time series'

    The time series for each component is defined by its inputs array. However, a manifest file can contain many separate components, each with their own time series. There is no guarantee that an individual time series is continuous, or that all the components in a manifest file have the same start time, end time and resolution. This makes it difficult to aggregate, visualize or do like-for-like comparisons between components.

    To solve this problem, we provide a built-in time-sync feature that synchronizes the time series' across all the components in a tree. The time-sync feature takes a global start time, end time and interval, then forces every individual time series to conform to this global configuration.

    • This works by first upsampling each time series to a common base resolution (typically 1s).
    • Any gaps in the time series are filled in with "zero objects", which have an identical structure to the real observations but with usage metrics set to zero (we assume that when there is no data, there is no usage).
    • Next, we check to see whether the first timestamp in each time series is before, after or identical to the global start time.
    • If a component's time series starts after the global start time, we pad the start of the time series with "zero objects" so that the start times are identical.
    • If the component's time series starts *before* the global start time, we trim the time series down, discarding observations from before the global start time. The same trimming logic is applied to the end times.
    • After synchronizing the start and end times and padding any discontinuities, we have a set of continuous time series' of identical length.
    • Next, we batch observations together into time bins whose size is define by the global interval value. This means that the resolution of the final time series' are identical and equal to interval.

    This process yields synchronized time series for all components across a tree, enabling easy visualization and intercomparison. This synchronization is also a prerequisite for our aggregation function.

    Toggling off time sync​

    Some applications will not want to pad with zero values, and may be strict about continuous time series' being provided in the raw manifest file. In these cases, simply toggle the padding off in the manifest file.

    - + \ No newline at end of file diff --git a/reference/cli/index.html b/reference/cli/index.html index b7155f5e..e09ace6a 100644 --- a/reference/cli/index.html +++ b/reference/cli/index.html @@ -8,13 +8,13 @@ Command line tool | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    Command line tool

    A core feature of the Impact Framework is the ie command line tool (CLI). This is how you trigger Impact Framework to execute a certain manifest file.

    Let's take a look at the various commands exposed by ie.

    ie​

    If you have globally installed our if npm package, you can invoke the CLI using the ie command directly in your terminal. The ie command is an alias to npx ts-node src/index.ts, which executes the Impact Framework's src/index.ts script and acts as the entry point for Impact Framework.

    ie <args>

    --manifest​

    The --manifest flag is the only required flag and tells ie where to find the manifest file that you want to execute. This command expects to receive the path where your manifest file is saved, as shown in the following example:

    ie --manifest examples/manifests/my-manifest.yml

    --output​

    The --output flag is optional and is used for defining a path to save your output data. If you provide the --output command with a path, your output data will be saved as a .yml file to disk. If you omit this command, your output data will be displayed in the terminal.

    Here is an example of --output being used to define a path:

    ie --manifest examples/manifests/my-manifest.yml --output examples/outputs/my-outdata.yml

    CSV export identifiers​

    If you want to save data to CSV, you have to select a specific metric to export. You do this by adding a hashtag and the metric name after the savepath provided to the output command. For example, you could save the carbon data to a CSV file called demo.csv as follows:

    ie --manifest demo.yml --output demo#carbon

    --override-params​

    The override-params command is used when you want to discard our recommended set of parameters and associated units and aggregation methods and instead provide your own. We do not recommend this, and if you use this feature you take full responsibility for any errors you introduce downstream, including unit or aggregation errors. This is why we hide the ability to override the parameters behind a CLI command - it is an advanced feature that you should only use if you really know what you are doing.

    You pass the path to your new parameter file as an argument. The file is expected to conform to the same structure as our src/config/params.ts file.

    For example:

    ie --manifest <your manifest> --override-params <path-to-your-params-file>
    - + \ No newline at end of file diff --git a/reference/index.html b/reference/index.html index 9b9dfc7e..504585cc 100644 --- a/reference/index.html +++ b/reference/index.html @@ -8,13 +8,13 @@ Reference | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    Reference

    In this section you will find reference documentation for the core data structures and features used in the Impact Framework.

    This includes:

    These are developer focused reference docs. If you are not a developer and looking for usage guides, please head over to the Using IF section.

    - + \ No newline at end of file diff --git a/reference/plugins/index.html b/reference/plugins/index.html index 329a2c6c..6c609161 100644 --- a/reference/plugins/index.html +++ b/reference/plugins/index.html @@ -8,14 +8,14 @@ Plugins | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    Plugins

    The Impact Framework includes several builtin plugins (e.g. time-sync, groupby and csv-exporter). All other plugins are external plugins that have to be installed before they can be run in an IF pipeline. Anyone can create and install a new plugin. As long as the plugin conforms to the expected interface, IF can run it.

    There are two repositories that were created by the IF core team. The if-plugins repository contains the "core" set of plugins that IF developers will maintain and support. We also provide a second repository of if-unofficial plugins that are mainly re-implementations of existing third party plugins or code that we expect community members to maintain independently of the IF core team.

    Documentation for the specific individual plugins are available in the plugin READMEs. Below you will find a brief description of each plugin and a link to its documentation:

    built-in​

    • Time Sync: Takes a heterogeneous set of time series data that might be offset, discontinuous or irregularly spaces and returns time series conforming to a user defined time grid. E.g. a user can define that all sets of observations should start at sopme global start time, end at some global end time and have a specific temporal resolution.

    • CSV Exporter: Exports data for a given metric to a CSV file.

    • Groupby: Allows a user to regroup their output data according to given keys.

    if-plugins​

    • Cloud metadata: Looks up detailed metadata about a given cloud instance type and region, including the physical processor being used.
    • E-MEM: Calculate the energy expended due to memroy usage, by multiplying the energy used in GB by a coefficient.
    • SCI-E: Calculates the sum of all energy components.
    • SCI-M - Calculates the embodied carbon for a component.
    • SCI-O - Calculates the operational carbon from the total energy and grid carbon intensity.
    • SCI: Calculates the software carbon intensity.
    • SHELL - A plugin that enables external code in any language to be run in a child process
    • TDP-FINDER: Looks up the thermnal desig power for a given processor in a local database.
    • Sum: a generic arithmetic plugin that allows you to sum any set of input parameters.
    • Multiply: a generic arithmetic plugin that allows you to multiply any set of input parameters.
    • Coefficient: a generic arithmetic plugin that allows you to multiply any input value by a coefficient.
    • E-NET: simply multiplies the amount of data transferred (GB) by a coefficient (kWh/GB) to yield network/energy.
    • Mock Observations: A plugin for mocking observations (inputs) for testing and demo purposes.
    • CSV-Export: a generic CSV exporter plugin.
    • Divide: A generic plugin for doing arithmetic division of two values.
    • Regex: A generic plugin to match part of one string and extract it into another.

    if-unofficial-plugins​

    • Azure importer: Grabs usage metrics from an Azure virtual machine, given user credentials and virtual machine details.
    • Cloud Carbon Footprint: Calculates usage metrics using the Cloud Carbon Footprint APIs.
    • WattTime: WattTime is an external service for looking up grid emissions based on location.
    • TEADS-CURVE: Calculates the energy in kWh used by the CPU
    • TEADS-AWS: Calculates the energy in kWh used by the CPU using a model specific to AWS instances.
    • Boavizta: Calculates energy and embodied carbon using the Boavizta APIs.
    • co2js: Calculates the carbon emissions of a website.

    Exhaust plugins (outputs)​

    Export plugins designed to implement custom ways of exporting output file. Currenlty supported ones are csv, yaml and log plugins. These are currently built in to the IF, but migrating to dynamically loading export functions as plugins is part of our near-term roadmap.

    - + \ No newline at end of file diff --git a/users/how-to-export-to-csv/index.html b/users/how-to-export-to-csv/index.html index d41fa986..36968391 100644 --- a/users/how-to-export-to-csv/index.html +++ b/users/how-to-export-to-csv/index.html @@ -8,13 +8,13 @@ How to export to CSV | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    How to export to CSV

    IF supports exporting data to CSV files. This provides users with a data format that enables visualization and data analysis using standard data analysis tools.

    Manifest config​

    To export your data to a CSV file, you have to provide a small piece of config data to your manifest file:

    initialize:
      outputs:
        - csv

    You can also add - yaml if you want to export to both yaml and csv simultaneously.

    CLI command​

    Then, you must select the metric you want to export to CSV. The name of that metric must be added to the savepath provided to the --output command in the CLI, after a hashtag.

    For example, to export the carbon data from your tree to a CSV file:

    ie --manifest example.yml --output example#carbon

    This will save a CSV file called example.csv. The contents will look similar to the following:

    PathAggregated2024-03-05T00:00:00.000Z2024-03-05T00:05:00.000Z2024-03-05T00:10:00.000Z
    tree.carbon425.28923200872517.92698771575438.902438878301845.6021901509012
    tree.children.westus3.carbon104.6968367228783.599738031978873.474381490323726.91318436533634
    tree.children.westus3.children.server-1.carbon104.6968367228783.599738031978873.474381490323726.91318436533634
    tree.children.france.carbon320.59239528584714.32724968377545.4280573879780838.6890057855649
    tree.children.france.children.server-2.carbon320.59239528584714.32724968377545.4280573879780838.6890057855649

    Comparing CSV to Yaml​

    The CSV above is generated from the following yaml. The carbon metric is extracted and added to the CSV. Otherwise,the CSV is an exact representation of the following yaml tree. You can see that the CSV reepresentation is much easier to understand than the full yaml tree:

    tree:
      pipeline:
        - mock-observations
        - group-by
        - cloud-metadata
        - time-sync
        - watttime
        - teads-curve
        - operational-carbon
      defaults:
        grid/carbon-intensity: 500
      config:
        group-by:
          group:
            - cloud/region
            - name
      children:
        westus3:
          children:
            server-1:
              inputs:
                - timestamp: '2024-03-05T00:00:00.000Z'
                  duration: 300
                  name: server-1
                  cloud/instance-type: Standard_E64_v3
                  cloud/region: westus3
                  cloud/vendor: azure
                  cpu/utilization: 66
                  grid/carbon-intensity: 500
                - timestamp: '2024-03-05T00:05:00.000Z'
                  duration: 300
                  name: server-1
                  cloud/instance-type: Standard_E64_v3
                  cloud/region: westus3
                  cloud/vendor: azure
                  cpu/utilization: 4
                  grid/carbon-intensity: 500
                - timestamp: '2024-03-05T00:10:00.000Z'
                  duration: 300
                  name: server-1
                  cloud/instance-type: Standard_E64_v3
                  cloud/region: westus3
                  cloud/vendor: azure
                  cpu/utilization: 54
                  grid/carbon-intensity: 500
                - timestamp: '2024-03-05T00:15:00.000Z'
                  duration: 300
                  name: server-1
                  cloud/instance-type: Standard_E64_v3
                  cloud/region: westus3
                  cloud/vendor: azure
                  cpu/utilization: 19
                  grid/carbon-intensity: 500
              outputs:
                - timestamp: '2024-03-05T00:00:00.000Z'
                  duration: 300
                  name: server-1
                  cloud/instance-type: Standard_E64_v3
                  cloud/region: westus3
                  cloud/vendor: azure
                  cpu/utilization: 65.78
                  grid/carbon-intensity: 369.4947514218548
                  vcpus-allocated: 64
                  vcpus-total: 64
                  memory-available: 432
                  physical-processor: >-
                    Intel® Xeon® Platinum 8370C,Intel® Xeon® Platinum 8272CL,Intel®
                    Xeon® 8171M 2.1 GHz,Intel® Xeon® E5-2673 v4 2.3 GHz
                  cpu/thermal-design-power: 269.1
                  cloud/region-cfe: CAISO
                  cloud/region-em-zone-id: US-CAL-CISO
                  cloud/region-wt-id: CAISO_NORTH
                  cloud/region-location: US West (N. California)
                  cloud/region-geolocation: 34.0497,-118.1326
                  geolocation: 34.0497,-118.1326
                  cpu/energy: 0.018934842060004835
                  carbon: 6.996324760173567
                - timestamp: '2024-03-05T00:05:00.000Z'
                  duration: 300
                  name: server-1
                  cloud/instance-type: Standard_E64_v3
                  cloud/region: westus3
                  cloud/vendor: azure
                  cpu/utilization: 3.986666666666667
                  grid/carbon-intensity: 369.38452029076234
                  vcpus-allocated: 64
                  vcpus-total: 64
                  memory-available: 432
                  physical-processor: >-
                    Intel® Xeon® Platinum 8370C,Intel® Xeon® Platinum 8272CL,Intel®
                    Xeon® 8171M 2.1 GHz,Intel® Xeon® E5-2673 v4 2.3 GHz
                  cpu/thermal-design-power: 269.1
                  cloud/region-cfe: CAISO
                  cloud/region-em-zone-id: US-CAL-CISO
                  cloud/region-wt-id: CAISO_NORTH
                  cloud/region-location: US West (N. California)
                  cloud/region-geolocation: 34.0497,-118.1326
                  geolocation: 34.0497,-118.1326
                  cpu/energy: 0.004545546617763956
                  carbon: 1.6790545568620359
                - timestamp: '2024-03-05T00:10:00.000Z'
                  duration: 300
                  name: server-1
                  cloud/instance-type: Standard_E64_v3
                  cloud/region: westus3
                  cloud/vendor: azure
                  cpu/utilization: 53.82
                  grid/carbon-intensity: 372.58122309244305
                  vcpus-allocated: 64
                  vcpus-total: 64
                  memory-available: 432
                  physical-processor: >-
                    Intel® Xeon® Platinum 8370C,Intel® Xeon® Platinum 8272CL,Intel®
                    Xeon® 8171M 2.1 GHz,Intel® Xeon® E5-2673 v4 2.3 GHz
                  cpu/thermal-design-power: 269.1
                  cloud/region-cfe: CAISO
                  cloud/region-em-zone-id: US-CAL-CISO
                  cloud/region-wt-id: CAISO_NORTH
                  cloud/region-location: US West (N. California)
                  cloud/region-geolocation: 34.0497,-118.1326
                  geolocation: 34.0497,-118.1326
                  cpu/energy: 0.017357893372978016
                  carbon: 6.467225143212361
                - timestamp: '2024-03-05T00:15:00.000Z'
                  duration: 300
                  name: server-1
                  cloud/instance-type: Standard_E64_v3
                  cloud/region: westus3
                  cloud/vendor: azure
                  cpu/utilization: 18.936666666666667
                  grid/carbon-intensity: 434.20042537311633
                  vcpus-allocated: 64
                  vcpus-total: 64
                  memory-available: 432
                  physical-processor: >-
                    Intel® Xeon® Platinum 8370C,Intel® Xeon® Platinum 8272CL,Intel®
                    Xeon® 8171M 2.1 GHz,Intel® Xeon® E5-2673 v4 2.3 GHz
                  cpu/thermal-design-power: 269.1
                  cloud/region-cfe: CAISO
                  cloud/region-em-zone-id: US-CAL-CISO
                  cloud/region-wt-id: CAISO_NORTH
                  cloud/region-location: US West (N. California)
                  cloud/region-geolocation: 34.0497,-118.1326
                  geolocation: 34.0497,-118.1326
                  cpu/energy: 0.010385485956624245
                  carbon: 4.5093824200727735
              aggregated:
                carbon: 19.651986880320734
          outputs:
            - carbon: 6.996324760173567
              timestamp: '2024-03-05T00:00:00.000Z'
              duration: 300
            - carbon: 1.6790545568620359
              timestamp: '2024-03-05T00:05:00.000Z'
              duration: 300
            - carbon: 6.467225143212361
              timestamp: '2024-03-05T00:10:00.000Z'
              duration: 300
            - carbon: 4.5093824200727735
              timestamp: '2024-03-05T00:15:00.000Z'
              duration: 300
          aggregated:
            carbon: 19.651986880320734
        france:
          children:
            server-2:
              inputs:
                - timestamp: '2024-03-05T00:00:00.000Z'
                  duration: 300
                  name: server-2
                  cloud/instance-type: Standard_E64_v3
                  cloud/region: france
                  cloud/vendor: azure
                  cpu/utilization: 15
                  grid/carbon-intensity: 500
                - timestamp: '2024-03-05T00:05:00.000Z'
                  duration: 300
                  name: server-2
                  cloud/instance-type: Standard_E64_v3
                  cloud/region: france
                  cloud/vendor: azure
                  cpu/utilization: 78
                  grid/carbon-intensity: 500
                - timestamp: '2024-03-05T00:10:00.000Z'
                  duration: 300
                  name: server-2
                  cloud/instance-type: Standard_E64_v3
                  cloud/region: france
                  cloud/vendor: azure
                  cpu/utilization: 16
                  grid/carbon-intensity: 500
                - timestamp: '2024-03-05T00:15:00.000Z'
                  duration: 300
                  name: server-2
                  cloud/instance-type: Standard_E64_v3
                  cloud/region: france
                  cloud/vendor: azure
                  cpu/utilization: 6
                  grid/carbon-intensity: 500
              outputs:
                - timestamp: '2024-03-05T00:00:00.000Z'
                  duration: 300
                  name: server-2
                  cloud/instance-type: Standard_E64_v3
                  cloud/region: france
                  cloud/vendor: azure
                  cpu/utilization: 14.95
                  grid/carbon-intensity: 1719.1647205176753
                  vcpus-allocated: 64
                  vcpus-total: 64
                  memory-available: 432
                  physical-processor: >-
                    Intel® Xeon® Platinum 8370C,Intel® Xeon® Platinum 8272CL,Intel®
                    Xeon® 8171M 2.1 GHz,Intel® Xeon® E5-2673 v4 2.3 GHz
                  cpu/thermal-design-power: 269.1
                  cloud/region-cfe: France
                  cloud/region-em-zone-id: FR
                  cloud/region-wt-id: FR
                  cloud/region-location: Paris
                  cloud/region-geolocation: 48.8567,2.3522
                  geolocation: 48.8567,2.3522
                  cpu/energy: 0.00905914075141129
                  carbon: 15.574155178030272
                - timestamp: '2024-03-05T00:05:00.000Z'
                  duration: 300
                  name: server-2
                  cloud/instance-type: Standard_E64_v3
                  cloud/region: france
                  cloud/vendor: azure
                  cpu/utilization: 77.74
                  grid/carbon-intensity: 1719.0544893865829
                  vcpus-allocated: 64
                  vcpus-total: 64
                  memory-available: 432
                  physical-processor: >-
                    Intel® Xeon® Platinum 8370C,Intel® Xeon® Platinum 8272CL,Intel®
                    Xeon® 8171M 2.1 GHz,Intel® Xeon® E5-2673 v4 2.3 GHz
                  cpu/thermal-design-power: 269.1
                  cloud/region-cfe: France
                  cloud/region-em-zone-id: FR
                  cloud/region-wt-id: FR
                  cloud/region-location: Paris
                  cloud/region-geolocation: 48.8567,2.3522
                  geolocation: 48.8567,2.3522
                  cpu/energy: 0.020379266251888902
                  carbon: 35.0330691407141
                - timestamp: '2024-03-05T00:10:00.000Z'
                  duration: 300
                  name: server-2
                  cloud/instance-type: Standard_E64_v3
                  cloud/region: france
                  cloud/vendor: azure
                  cpu/utilization: 15.946666666666667
                  grid/carbon-intensity: 1718.8707708347622
                  vcpus-allocated: 64
                  vcpus-total: 64
                  memory-available: 432
                  physical-processor: >-
                    Intel® Xeon® Platinum 8370C,Intel® Xeon® Platinum 8272CL,Intel®
                    Xeon® 8171M 2.1 GHz,Intel® Xeon® E5-2673 v4 2.3 GHz
                  cpu/thermal-design-power: 269.1
                  cloud/region-cfe: France
                  cloud/region-em-zone-id: FR
                  cloud/region-wt-id: FR
                  cloud/region-location: Paris
                  cloud/region-geolocation: 48.8567,2.3522
                  geolocation: 48.8567,2.3522
                  cpu/energy: 0.009405866514354337
                  carbon: 16.16746902589712
                - timestamp: '2024-03-05T00:15:00.000Z'
                  duration: 300
                  name: server-2
                  cloud/instance-type: Standard_E64_v3
                  cloud/region: france
                  cloud/vendor: azure
                  cpu/utilization: 5.98
                  grid/carbon-intensity: 1718.6686804277592
                  vcpus-allocated: 64
                  vcpus-total: 64
                  memory-available: 432
                  physical-processor: >-
                    Intel® Xeon® Platinum 8370C,Intel® Xeon® Platinum 8272CL,Intel®
                    Xeon® 8171M 2.1 GHz,Intel® Xeon® E5-2673 v4 2.3 GHz
                  cpu/thermal-design-power: 269.1
                  cloud/region-cfe: France
                  cloud/region-em-zone-id: FR
                  cloud/region-wt-id: FR
                  cloud/region-location: Paris
                  cloud/region-geolocation: 48.8567,2.3522
                  geolocation: 48.8567,2.3522
                  cpu/energy: 0.0054492484351820105
                  carbon: 9.365452617417297
              aggregated:
                carbon: 76.1401459620588
          outputs:
            - carbon: 15.574155178030272
              timestamp: '2024-03-05T00:00:00.000Z'
              duration: 300
            - carbon: 35.0330691407141
              timestamp: '2024-03-05T00:05:00.000Z'
              duration: 300
            - carbon: 16.16746902589712
              timestamp: '2024-03-05T00:10:00.000Z'
              duration: 300
            - carbon: 9.365452617417297
              timestamp: '2024-03-05T00:15:00.000Z'
              duration: 300
          aggregated:
            carbon: 76.1401459620588
      outputs:
        - carbon: 22.57047993820384
          timestamp: '2024-03-05T00:00:00.000Z'
          duration: 300
        - carbon: 36.71212369757613
          timestamp: '2024-03-05T00:05:00.000Z'
          duration: 300
        - carbon: 22.63469416910948
          timestamp: '2024-03-05T00:10:00.000Z'
          duration: 300
        - carbon: 13.87483503749007
          timestamp: '2024-03-05T00:15:00.000Z'
          duration: 300
      aggregated:
        carbon: 95.79213284237952

    CSV and aggregation​

    The CSV representation of the output data is helpful for intuiting how the aggregation procedure works. What we refer to as "horizontal" aggregation is really an aggregation of the rows of the CSV. You can replicate the IF aggregation function by summing the cells in each row of the CSV. Similarly, what we refer to as "vertical" aggregation can be replicatd by summing the columns in the CSV representation (this is not exactly accurate because you have to skip summing both parent nodes and their children, both of which are represented in the CSV, but it is true conceptually).

    Walkthrough​

    You can start with a demo yaml, such as the following (save this as pipeline-demo.yml):

    name: pipeline-demo
    description:
    tags:
    initialize:
      outputs:
        - csv
      plugins:
        boavizta-cpu:
          method: BoaviztaCpuOutput
          path: "@grnsft/if-unofficial-plugins"
          global-config:
            allocation: LINEAR
            verbose: true
        "sum":
          path: "@grnsft/if-plugins"
          method: Sum
          global-config:
            input-parameters:
              - cpu/energy
              - network/energy
            output-parameter: energy
        "sci-m":
          path: "@grnsft/if-plugins"
          method: SciM
        "sci-o":
          path: "@grnsft/if-plugins"
          method: SciO
    tree:
      children:
        child-1:
          pipeline:
            - boavizta-cpu
            - sum
            - sci-m
            - sci-o
          config:
          defaults:
            cpu/thermal-design-power: 100
            grid/carbon-intensity: 800
            device/emissions-embodied: 1533.120 # gCO2eq
            time-reserved: 3600 # 1hr in seconds
            device/expected-lifespan: 94608000 # 3 years in seconds
            resources-reserved: 1
            resources-total: 8
            cpu/number-cores: 24
            cpu/name: Intel® Core™ i7-1185G7
          inputs:
            - timestamp: "2023-12-12T00:00:00.000Z"
              cloud/instance-type: A1
              region: uk-west
              duration: 1
              cpu/utilization: 50
              network/energy: 0.000001
            - timestamp: "2023-12-12T00:00:01.000Z"
              duration: 5
              cpu/utilization: 20
              cloud/instance-type: A1
              region: uk-west
              network/energy: 0.000001
            - timestamp: "2023-12-12T00:00:06.000Z"
              duration: 7
              cpu/utilization: 15
              cloud/instance-type: A1
              region: uk-west
              network/energy: 0.000001
            - timestamp: "2023-12-12T00:00:13.000Z"
              duration: 30
              cloud/instance-type: A1
              region: uk-west
              cpu/utilization: 15
              network/energy: 0.000001

    Run this using:

    ie --manifest pipeline-demo.yml --output pipeline-demo#carbon

    This will save a csv file called pipeline-demo.csv. Inside, the data will look as follows:

    Path 2023-12-12T00:00:00.000Z2023-12-12T00:00:01.000Z2023-12-12T00:00:06.000Zb
    tree.children.child-1.energy0.000028777777777777780.000112111111111111110.00027877777777777780.0005565555555555556
    - + \ No newline at end of file diff --git a/users/how-to-import-plugins/index.html b/users/how-to-import-plugins/index.html index 29b8038d..257e5b6b 100644 --- a/users/how-to-import-plugins/index.html +++ b/users/how-to-import-plugins/index.html @@ -8,13 +8,13 @@ How to load plugins | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    How to load plugins

    Plugins are developed separately to the Impact Framework core. However, the IF core developers maintain a standard library of plugins that can be found in this Github repository. We also provide some implementations of popular community plugins, although we rely on other members of the community to maintain them.

    Use the following commands to install the if-plugins and if-unofficial-plugins repositories:

    npm -i -g @grnsft/if-plugins
    npm -i -g @grnsft/if-unofficial-plugins

    Plugins in these packages can then be invoked in an manifest by providing their path in the plugin initialization, as shown in the following example:

    name: if-demo
    description: demo pipeline
    tags:
    initialize:
      plugins:
        azure-importer:
          method: AzureImporter
          path: "@grnsft/if-unofficial-plugins"
        cloud-metadata:
          method: CloudMetadata
          path: "@grnsft/if-plugins"

    Load your plugin directly from your Github repository, or from npm if you have published your plugin there. First, you'll need to install it by providing the path to the repository to npm install as follows:

    npm install https://github.com/Green-Software-Foundation/if-plugins

    You'll need to provide the following fields:

    • YOUR-PLUGIN-HERE: the same name has to be used to refer to this plugin everywhere across the manifest
    • method: the class name for your plugin, e.g. AzureImporter
    • path: the path to the plugin

    And, if your plugin requires it, add global-config too.

    Then, in your manifest, initialize the plugin as follows:

    name: plugin-demo
    description: loads plugin
    tags: null
    initialize:
      plugins:
        <YOUR-PLUGIN-HERE:
          method: OutputPlugin
          path: https://github.com/my-repo/my-plugin

    Anyone can develop plugins. As long as you conform to our plugin specification, you can load your plugin into the Impact Framework and run it as part of a pipeline. We provide a guide to building plugins and a template to help you structure them correctly.

    - + \ No newline at end of file diff --git a/users/how-to-install-if/index.html b/users/how-to-install-if/index.html index b552219f..c79b0437 100644 --- a/users/how-to-install-if/index.html +++ b/users/how-to-install-if/index.html @@ -8,13 +8,13 @@ How to install Impact Framework | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    How to install Impact Framework

    You can install Impact Framework either globally or locally. For most users, we recommend installing our official releases globally using npm. You can do this using the following command:

    npm install -g @grnsft/if

    Then, run the package using the ie command:

    ie --manifest <path to manifest file>

    There is only one plugin that is built in to the core Impact Framework codebase (time-sync). If you only want to use Impact Framework with your own, locally-developed plugins or plugins loaded from remote Github repositories, then you have already installed everything you need to get started. However, most likely you will want to install some of our plugins too.

    There are two plugin packages you will probably want to install: if-plugins and if-unofficial-plugins.

    • if-plugins is our standard library of plugins that we maintain. Generally, these are used for calculating a Software Carbon Intensity score.
    • if-unofficial-plugins is a collection of plugins that rely on some third party API or that reimplement a pre-existing plugin originally developed by another group. For example, our Azure data importer and Co2js.

    You can install the latest releases of these plugin packages using npm:

    npm install -g @grnsft/if-plugins
    npm install -g @grnsft/if-unofficial-plugins

    Now you have globally installed the framework and a set of plugins and can start using IF for calculating the impact of your applications.

    Installing locally​

    You can also clone the Impact Framework repositories and install them locally, useful for developers who want to make changes or build new plugins. Use the following command for local installation:

    git clone https://github.com/Green-Software-Foundation/if && cd if
    npm install

    Then, use the following command to run Impact Framework:

    npm run ie -- --manifest <path to your manifest file>

    Next, install local plugin repositories using npm link. You can do this by entering the plugin folder and running the following command:

    npm link

    This creates a global package with the same name as your project root directory which you can then load by passing the path in your manifest file.

    Read our detailed guide to installing plugins.

    - + \ No newline at end of file diff --git a/users/how-to-write-manifests/index.html b/users/how-to-write-manifests/index.html index 2a549dd5..17cd354c 100644 --- a/users/how-to-write-manifests/index.html +++ b/users/how-to-write-manifests/index.html @@ -8,13 +8,13 @@ How to write a manifest file | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    How to write a manifest file

    The Impact Framework receives all its configuration and input data in the form of a manifest file known as an manifest. To use the framework, you will need to write a manifest file and pass its path to the command line tool. This guide will help you to understand how to construct one of these files and use it to measure the energy and carbon usage of your app.

    Structure of a manifest​

    The basic structure of a manifest is as follows: 

    name: 
    description: 
    tags: 
    initialize:
      plugins:
        <PLUGIN-NAME-HERE>: 
          method:
          path: 
    tree:
      children:
        child:
          pipeline:
            - 
          config:
          defaults:
          inputs:

    Project metadata​

    The file starts with some metadata about the project. There are no strict specfications for what to put in these fields, they are for you to keep track of your manifest files and to help other users to understand your use case.

    name:
    description:
    tags:

    Initialize​

    The initialize fields are where you specify each individual plugin that will be initialized in your pipeline. The plugins can be initialized in any order, but can only be invoked elsewhere in the manifest if they have been initialized first here. In each case, you will need to provide the name, path and method (and global-config if your plugin requires it):

    initialize:
      plugins:
        sci-m:
          path: ''
          method:
    • The name is the name you want this plugin instance to be recognized as by Impact Framework.
    • The path defines where IF should look for the installed plugin. For example, for our standard library of plugins you would specify "@grnsft/if-plugins", as this is the name of the directory they are installed into in node_modules.
    • For the method field, you should provide the name of the function exported by your plugin. For example, for the sci-e plugin, the correct value is SciE.

    Tree​

    The tree fields are where you define the various components of your application. Each component is defined as children, where each child's output is summed to give the overall impact. Each child can have its own plugin pipeline and its own configuration, but when none is provided, it is inherited from the tree-level configuration.

    In the following example, there is only one component but the plugin pipeline contains two plugins; teads-curve and sci-m. Neither requires any config data, but certain information is required in inputs.

    tree:
      children:
        child:
          pipeline:
            - teads-curve
            - sci-m
          config:
          defaults:
          inputs:
            - timestamp: '2023-11-02T10:35:31.820Z'
              duration: 3600
              total-embodied-emissions: 1533.12
              time-reserved: 1
              expected-lifespan: 3
              resources-reserved: 1
              total-resources: 8

    Inputs​

    The most granular level of the manifest file are the inputs. This is where you can add specific data for each child. Inputs must always include a timestamp and a duration.

    inputs:
      - timestamp: 2023-07-06T00:00
        duration: 3600
        cpu-util: 45

    You now have a simple manifest file that will use the plugin config and input data to run the teads-curve and sci-m plugins. The output data will be appended to the manifest under a new outputs field and saved as an output file.

    More complex manifests​

    Complex pipelines​

    Whilst the manifest file we looked at above works perfectly well, it will only return the most basic output data. Most users will want to calculate an SCI score, which implies a number of additional steps:

    • operational-carbon and embodied-carbon must appear as inputs.
    • This means that sci will need to be preceded by sci-m and sci-o in the plugin pipeline.
    • In most cases, sci-o will have to be preceded by sci-e to ensure energy is available to be piped to sci-o.
    • The inputs to sci-e will most likely be coming from a plugin such as teads-curve or boavizta.
    • The sci plugin also requires functional-unit information so it can convert the estimated carbon into a useful unit.
    • You may also wish to grab your input data by querying a metrics API on a virtual machine.

    The example below gives you the full pipeline implemented in an manifest.There are also several other executable example manifests in if/examples/manifests that you can run for yourself.

    name: pipeline-demo
    description:
    tags:
    aggregation:
      metrics:
        - 'carbon'
      type: 'both'
    initialize:
      plugins:
        "teads-curve":
          path: "@grnsft/if-unofficial-plugins"
          method: TeadsCurve
          global-config:
            interpolation: spline
        "sci-e":
          path: "@grnsft/if-plugins"
          method: SciE
        "sci-m":
          path: "@grnsft/if-plugins"
          method: SciM
        "sci-o":
          path: "@grnsft/if-plugins"
          method: SciO
        "sci":
          path: "@grnsft/if-plugins"
          method: Sci
          global-config:
            functional-unit: "requests"
            functional-unit-time: "1 minute"
        "time-sync":
          method: TimeSync
          path: "builtin"
          global-config:
            start-time: "2023-12-12T00:00:00.000Z"
            end-time: "2023-12-12T00:01:00.000Z"
            interval: 5
            allow-padding: true
        'group-by':
          path: builtin
          method: GroupBy
    tree:
      children:
        child-1:
          pipeline:
            - teads-curve
            - sci-e
            - sci-m
            - sci-o
            - time-sync
            - sci
          config:
            group-by:
              group:
                - region
                - instance-type
          defaults:
            cpu/thermal-design-power: 100
            grid/carbon-intensity: 800
            device/emissions-embodied: 1533.120 # gCO2eq
            time-reserved: 3600 # 1hr in seconds
            device/expected-lifespan: 94608000 # 3 years in seconds
            resources-reserved: 1
            resources-total: 8
            functional-unit-time: "1 min"
          inputs:
            - timestamp: "2023-12-12T00:00:00.000Z"
              instance-type: A1 
              region: uk-west
              duration: 1
              cpu/utilization: 10
            - timestamp: "2023-12-12T00:00:01.000Z"
              duration: 5
              cpu/utilization: 20
              instance-type: A1 
              region: uk-west
            - timestamp: "2023-12-12T00:00:06.000Z"
              duration: 7
              cpu/utilization: 15
              instance-type: A1 
              region: uk-west
            - timestamp: "2023-12-12T00:00:13.000Z"
              duration: 30
              instance-type: A1 
              region: uk-west
              cpu/utilization: 15
        child-2:
          pipeline:
            - teads-curve
            - sci-e
            - sci-m
            - sci-o
            - time-sync
            - sci
          config:
            group-by:
              group:
                - region
                - instance-type
          defaults:
            cpu/thermal-design-power: 100
            grid/carbon-intensity: 800
            device/emissions-embodied: 1533.120 # gCO2eq
            time-reserved: 3600 # 1hr in seconds
            device/expected-lifespan: 94608000 # 3 years in seconds
            resources-reserved: 1
            resources-total: 8
            functional-unit-time: "1 min"
          inputs:
            - timestamp: "2023-12-12T00:00:00.000Z"
              duration: 1
              cpu/utilization: 30
              instance-type: A1 
              region: uk-west
            - timestamp: "2023-12-12T00:00:01.000Z"
              duration: 5
              cpu/utilization: 28
              instance-type: A1 
              region: uk-west
            - timestamp: "2023-12-12T00:00:06.000Z"
              duration: 7
              cpu/utilization: 40
              instance-type: A1 
              region: uk-west
            - timestamp: "2023-12-12T00:00:13.000Z"
              duration: 30
              cpu/utilization: 33
              instance-type: A1 
              region: uk-west

    Complex applications​

    The manifest examples provided so far have only had a single component. However, Impact Framework can handle any number of nested children.

    In this way, you can combine complex plugin pipelines and application architectures to calculate the energy and carbon outputs of complicated systems.

    Choosing which plugins to run​

    The plugins are designed to be composable, but they each have specific input requirements that must be met in order for the plugins to run correctly. For example, the teads-curve plugin requires cpu/thermal-design-power to be available in the manifest. If it is not there, the plugin cannot use it to calculate cpu/energy.

    It is also possible to leapfrog some plugins if you have access to high-level data. For example, perhaps you already know the energy being used by your CPU. In this case, there is no need to run teads-curve, you can simply provide cpu/energy as an input and omit teads-curve from the plugin pipeline.

    We have deliberately made the plugins modular and composable so that you can be creative in developing new plugins to replace those provided as part of IF.

    Running a manifest​

    You run a manifest by providing its path to our command line tool and a path to save the results file to. You can run a manifest named my-manifest.yml using the following command:

    ie --manifest my-manifest.yml
    - + \ No newline at end of file diff --git a/users/index.html b/users/index.html index 089cb210..4e2d8dd9 100644 --- a/users/index.html +++ b/users/index.html @@ -8,13 +8,13 @@ Users | Impact Framework - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    Users

    This section contains information for Impact Framework users. You are a user if you want to apply the Impact Framework to your own use-case, such as using it to measure the environmental impact of your own apps running on some cloud platform.

    The user documentation includes:

    If you are looking for guidance for how to change or update the Impact Framework by adding new features, fixing bugs or building new plugins, you should go to our developers documentation instead.

    - + \ No newline at end of file diff --git a/users/quick-start/index.html b/users/quick-start/index.html index 9efe6b1e..391971d0 100644 --- a/users/quick-start/index.html +++ b/users/quick-start/index.html @@ -8,13 +8,13 @@ Quick start | Impact Framework - +
    -
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    Quick start

    This page will provide the basic instructions for getting up and running with Impact Framework.

    1: Install Impact Framework​

    Install the Impact Framework globally using npm.

    npm install -g @grnsft/if

    Read our detailed guide to installing IF.

    2: Install some plugins​

    Install some of the plugins you want to include in your pipeline. The following commands will install both the official and unofficial IF model packages.

    npm install -g @grnsft/if-plugins
    npm install -g @grnsft/if-unofficial-plugins

    Read our detailed guide to loading plugins.

    3: Create a manifest file​

    A manifest file contains all the configuration and input data required to measure your application's energy and carbon impacts and should have a .yml extension.

    Open the file, add your data and save the file. The simple example below runs a single snapshot observation through a single plugin.

    name: basic-demo
    description:
    tags:
    initialize:
      plugins:
        teads-curve: 
          path: '@grnsft/if-unofficial-plugins'
          method: TeadsCurve
          global-config:
            interpolation: spline
    tree:
      children:
        child-0:
          defaults:
            cpu/thermal-design-power: 100
          pipeline:
            - teads-curve
          inputs:
            - timestamp: 2023-07-06T00:00
              duration: 1
              cpu/utilization: 20
            - timestamp: 2023-07-06T00:01
              duration: 1
              cpu/utilization: 80
            - timestamp: 2023-07-06T00:02
              duration: 1
              cpu/utilization: 20

    Read our detailed guide to writing manifest files.

    4: Compute your manifest file​

    Run the pipeline by passing the path to your manifest file to the ie command line tool:

    ie --manifest <path-to-your-manifest>

    🎉Congratulations 🎉! You have just used the Impact Framework to compute the energy consumed by an application!

    Next steps​

    Now you know how to use the ie you can start building more complex pipelines of plugins and more complicated manifest files. Your overall aim is to create a manifest file that accurately represents a real software application, and a plugin pipeline that yields an environmental metric that's important to you (e.g. carbon).

    Experiment by adding more plugins to the pipeline, for example add sci-o to convert energy into operational-carbon. Your output data will be displayed in your console.

    You can also configure if to save your output data to another yaml file. To do this, add the --output flag and the path to save the file to. You will also need to add the following config to the initialize block in your manifest file:

    outputs:
      - yaml

    The command is then as follows:

    ie --manifest <path-to-your-impl> --output <save-path>

    Explore our user documentation for walkthrough guides to common Impact Framework tasks:

    - +
    This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.

    Quick start

    This page will provide the basic instructions for getting up and running with Impact Framework.

    1: Install Impact Framework​

    Install the Impact Framework globally using npm.

    npm install -g @grnsft/if

    Read our detailed guide to installing IF.

    2: Install some plugins​

    Install some of the plugins you want to include in your pipeline. The following commands will install both the official and unofficial IF model packages.

    npm install -g @grnsft/if-plugins
    npm install -g @grnsft/if-unofficial-plugins

    Read our detailed guide to loading plugins.

    3: Create a manifest file​

    A manifest file contains all the configuration and input data required to measure your application's energy and carbon impacts and should have a .yml extension.

    Open the file, add your data and save the file. The simple example below runs a single snapshot observation through a single plugin.

    name: basic-demo
    description:
    tags:
    initialize:
      plugins:
        teads-curve: 
          path: '@grnsft/if-unofficial-plugins'
          method: TeadsCurve
          global-config:
            interpolation: spline
    tree:
      children:
        child-0:
          defaults:
            cpu/thermal-design-power: 100
          pipeline:
            - teads-curve
          inputs:
            - timestamp: 2023-07-06T00:00
              duration: 1
              cpu/utilization: 20
            - timestamp: 2023-07-06T00:01
              duration: 1
              cpu/utilization: 80
            - timestamp: 2023-07-06T00:02
              duration: 1
              cpu/utilization: 20

    Read our detailed guide to writing manifest files.

    4: Compute your manifest file​

    Run the pipeline by passing the path to your manifest file to the ie command line tool:

    ie --manifest <path-to-your-manifest>

    🎉Congratulations 🎉! You have just used the Impact Framework to compute the energy consumed by an application!

    Next steps​

    Now you know how to use the ie you can start building more complex pipelines of plugins and more complicated manifest files. Your overall aim is to create a manifest file that accurately represents a real software application, and a plugin pipeline that yields an environmental metric that's important to you (e.g. carbon).

    Experiment by adding more plugins to the pipeline, for example add sci-o to convert energy into operational-carbon. Your output data will be displayed in your console.

    You can also configure if to save your output data to another yaml file. To do this, add the --output flag and the path to the output file where the results are saved. You will also need to add the following config to the initialize block in your manifest file:

    outputs:
      - yaml

    The command is then as follows:

    ie --manifest <path-to-your-impl> --output <save-path>

    Explore our user documentation for walkthrough guides to common Impact Framework tasks:

    + \ No newline at end of file