diff --git a/docs/converters/converter.md b/docs/converters/converter.md index e1c7611f..af827285 100644 --- a/docs/converters/converter.md +++ b/docs/converters/converter.md @@ -6,28 +6,28 @@ SPDX-License-Identifier: MPL-2.0 # Converters -There are 4 types of converters present as of now. -Here, we shall discuss their basic structure and guidelines for building a custom converter. +We currently provide 4 types of converters. +Here, we discuss their basic structure and guidelines for building a custom converter. Use the examples notebooks to understand how to convert data from the respective formats. - **PGM JSON Converter:** Refer to the [PGM JSON Example](../examples/pgm_json_example.ipynb) - **VisonExcelConverter** Refer to the [Vision Example](../examples/vision_example.ipynb) -- **Pandapower Converter:** Converts [pandapower network](https://pandapower.readthedocs.io/en/stable/elements.html), which is a dictionary of dataframes, to power-grid-model data. +- **Pandapower Converter:** Converts [pandapower network](https://pandapower.readthedocs.io/en/stable/elements.html) (a dictionary of dataframes) to power-grid-model data. Refer to [converters](../power_grid_model_io.md#converters) in API documentation for more details ## Structure -The `VisonExcelConverter` extends the [tabular converters](tabular_converter.md) for Excel exports of Vision. All converters are derived from the base {py:class}`power_grid_model_io.converters.base_converter`. The usable functions for loading, saving and converting the data are located in the base class. -The private functions (`_load_data`, `_parse_data` and `_serialize_data`) are overloaded based on the specific type of converter (ie. excel, json or pandapower). -It is recommended to create any custom converter in a similar way. +The private functions (`_load_data`, `_parse_data` and `_serialize_data`) are overloaded based on the specific type of converter (i.e., excel, json or pandapower). +The `VisonExcelConverter` extends the [tabular converters](tabular_converter.md) for Excel exports of Vision. +It is recommended that custom converters be created in a similar way. ## Instantiation -A converter object can be instantiated in the following way. For eg, for a `PgmJsonConverter`, +Converter objects can be instantiated in the following way. For instance, for a `PgmJsonConverter`: ```python from power_grid_model_io.converters.pgm_json_converter import PgmJsonConverter @@ -35,20 +35,23 @@ from power_grid_model_io.converters.pgm_json_converter import PgmJsonConverter converter = PgmJsonConverter(source_file=source, destination_file=destination) ``` -The usable methods of converters for loading and saving the data are described below. +## Common methods for converters + +The common methods across different converters, including data IO and log level configurations, are described below. +Converter specific methods are presented in corresponding sections. -## Loading data +### Loading data -Use the methods load_input_data(), load_update_data(), load_sym_output_data() or load_asym_output_data() to load the relevant data to the converter. -The Converter can be initialised with `source_file` containing path to source data. Or alternatively, the data can be provided as an argument to the load function. +Methods load_input_data(), load_update_data(), load_sym_output_data() and load_asym_output_data() are for loading the relevant data to the converter. +Input data can be configured at converter initialisation by passing the path to source data to parameter `source_file`. Alternatively, the data can also be provided as an argument to the load function after initialisation. -In addition to the power-grid-model input data, other miscellaneous information in the source file not used in calculations by power-grid-model gets stored under `extra_info` +In addition to the input data that will be used in calculations by power-grid-model, other miscellaneous information in the source file is stored in `extra_info`: ```python input_data, extra_info = converter.load_input_data(data=example_data) ``` -## Saving Data +### Saving data It is possible to save the data in the format of the converter. The Converter can be instantiated with a path given to `destination_file`. @@ -59,9 +62,10 @@ You can also add additional information about each component in the form of `ext converter.save(example_data, extra_info=example_extra_info, destination=destination_path) ``` -## Configuring The Log Output +### Configuring the log output -We have provided an interface to configure the log level of the converter. +An interface is provided to configure the log level of converters. +This configuration can be done at converter initialisation or after. Notice that this log level only belongs to the logger within the converter. Users need to set their basic configuration of the `logging` module to a level that is below what is configuired for the converters. @@ -78,5 +82,6 @@ converter_info = VisionExcelConverter(input_file) # Uses default INFO level logging.basicConfig(level=logging.WARNING) # Only levels WARNING and above will be logged assert converter_info.get_log_level() == logging.INFO # Previously created converters will retain their original log level, regardless of system wide configuration -converter_debug = VisionExcelConverter(input_file, log_level=logging.DEBUG) # Any logs on DEBUG and INFO level will not be logged +converter_ = VisionExcelConverter(input_file, log_level=logging.DEBUG) # Any logs on DEBUG and INFO level will not be logged +converter_.set_log_level(logging.WARNING) # Now the converter's log level is set to WARNING ``` \ No newline at end of file diff --git a/docs/converters/pandapower_converter.md b/docs/converters/pandapower_converter.md index 58a95b23..5bd2ddd5 100644 --- a/docs/converters/pandapower_converter.md +++ b/docs/converters/pandapower_converter.md @@ -22,14 +22,14 @@ If a `vector_group` is not available for transformer then a default is set in co Similarly for three winding transformer, `YNynyn` is set for even clocks of `shift_mv_degree` and `shift_lv_degree`. If the clocks are odd, then the vector group is converted as `YNynd`, `YNdyn` or `YNdd`. -## Modelling differences +## Modeling differences The user must be aware of following unsupported features or differences in conversion. * Currently, the conversions only support powerflow calculations and their relevant attributes. -* Any feature involving a PV bus, ie. generator, DC line are unsupported as of now. +* Any feature involving a PV bus, i.e., generator, DC line are unsupported as of now. -In addition, the following component-specific modelling differences exist. +In addition, the following component-specific modeling differences exist. ### Load @@ -41,7 +41,7 @@ The features regarding `in_ka` and `z_ohm` attributes are currently unsupported. ### External grid -The external grid is modelled with a source impedance all sequence components in power-grid-model. +The external grid is modeled with a source impedance all sequence components in power-grid-model. Whereas it is only present in negative and zero sequence networks in pandapower. Hence, this impedance value has to be mentioned in the power-flow calculation when using the power-grid-model power flow calculation. @@ -57,4 +57,4 @@ The default for transformer model to be used in pandapower is `t` model but powe ### Three winding transformer The differences defined in [Transformer](#transformer) are applicable here as well. -Additionally, tap connection at star point, ie. `tap_at_star_point` is not supported. \ No newline at end of file +Additionally, tap connection at star point, i.e., `tap_at_star_point` is not supported. \ No newline at end of file diff --git a/docs/converters/tabular_converter.md b/docs/converters/tabular_converter.md index 916aebb7..3f374835 100644 --- a/docs/converters/tabular_converter.md +++ b/docs/converters/tabular_converter.md @@ -4,14 +4,13 @@ SPDX-FileCopyrightText: Contributors to the Power Grid Model project # Tabular converter -Tabular data can come from Excel files, a set of CSV files, GNF files, databases, pandas DataFrames, etc etc. -The similarity between all tabular data is that it contains multiple `tables`, -each with multiple `columns`, possibly with a specific `unit`. -Others may have a categorical value which needs to be mapped (i.e. open: 0, closed: 1); in general we'll call these -`substitutions`. +Tabular data is commonly stored in spreadsheet files: Excel files, CSV files, GNF files, databases, pandas DataFrames, etc. +The similarity between all tabular data is that it contains multiple `tables`, each with multiple `columns`, possibly with a specific `unit` row. +Others may have categorical values that need to be further mapped (i.e., open: 0, closed: 1). +These attributes are referred to as `substitutions`. ## Mapping file -A mapping file is a yaml with three main sections `grid`, `units` and `substitutions`: +A mapping file is a yaml file with three main sections `grid`, `units` and `substitutions`: ```yaml grid: @@ -53,13 +52,12 @@ substitutions: ``` ## Grid -For each `table`, the target PGM `component` is listed (e.g. Nodes: node, Cables: line). -The for each PGM `column` the source column is supplied (e.g. u_rated: Unom, from_status: From.SwitchStatus). +For each `table`, the target PGM `component` is listed (e.g., Nodes: node, Cables: line). +The for each PGM `column` the source column is supplied (e.g., u_rated: Unom, from_status: From.SwitchStatus). ## Field Definitions -If the `column` definition is a one on one mapping, -the value is simply the name of the source column (e.g.u_rated: Unom), -but in many cases the mapping is a bit more complex. +If the `column` definition is a one-on-one mapping, the value is simply the name of the source column (e.g., u_rated: Unom). +In many other cases, however, mappings can be a bit more complex. You can use the following `column` definitions: * Column name `str` @@ -93,7 +91,7 @@ You can use the following `column` definitions: tan1: 0.0 ``` * Pandas DataFrame functions `Dict[str, List[Any]]` - (`prod`, `sum`, `min`, `max`, etc and the alias `multiply` which translates to `prod`) + (`prod`, `sum`, `min`, `max`, etc. and the alias `multiply` which translates to `prod`) ```yaml p_specified: min: @@ -130,15 +128,12 @@ You can use the following `column` definitions: ) ``` ## Units -Power Grid Model uses SI units (e.g. "W" for Watts), -but source data may be supplied in different units (e.g. "MW" for Mega Watts). -If units are supplied in the tabular data, -the pandas DataFrame storing the data is expected to have `MultiIndexes` for the columns. -For our application, a `MultiIndex` can be interpreted as a tuple; the first element is the column name, the second -element is the column unit. For example: `("C0", "µF")`. +Power Grid Model uses SI units (e.g., "W" for Watts), but source data may be supplied in different units (e.g., "MW" for Mega Watts). +If units are supplied in the tabular data, the data stored using pandas DataFrame is expected to have `MultiIndexes` for columns. +For our application, a `MultiIndex` can be interpreted as a tuple; the first element is the column name, the second element is the column unit. For example: `("C0", "µF")`. -If a unit is supplied, it should be defined in the units section of the mapping. -Undefined units are not allowed to prevent errors. +If an optional unit is supplied, it should be defined in the unit section of the mapping. +Undefined units that do not exist in mapping are not allowed. ```yaml units: @@ -156,7 +151,7 @@ The definitions above can be interpreted as: * 1 **ohm/km** = 0.001 **ohm/m** ## Substitutions -Some columns may contain categorical values (enums) which should be replaced. The column names can be defined as +Some columns may contain categorical values (enums) that should be mapped. The column names can be defined as regular expressions. ```yaml substitutions: @@ -169,16 +164,16 @@ substitutions: own: true ``` The definitions above can be interpreted as: - * In all columns that end with `SwitchState` (e.g. `From.Switch State`, `To.Switch State` or just `Switch State`), - the word "off" should be replaced with the integer 0 and the word "in" should be replaced with the value 1. + * In all columns that end with `SwitchState` (e.g., `From.Switch State`, `To.Switch State` or just `Switch State`), + the word "off" should be replaced with integer value 0, and the word "in" integer value 1. * In all columns called "N1", - the word "none" should be replaced with the boolean `false` - and the word "own" should be replaced with the value boolean `true`. + the word "none" should be replaced with the boolean value `false`, + and the word "own" boolean value `true`. ## AutoID The `id` field is special in the sense that each object should have a unique numerical id in power grid model. -Therefore, each id definition is mapped to a numerical ID. -Also each field name that ends with `node` is converted into the matching numerical ID. +Therefore, each id definition is mapped to a numerical (integer) ID. +Field names that end with `node` are also mapped to corresponding numerical IDs. ```python from power_grid_model_io.utils.auto_id import AutoID @@ -193,8 +188,8 @@ item = auto_id[1] # item = "Bravo" See also {py:class}`power_grid_model_io.utils.AutoID` ## AutoID Mapping -Let's consider a very common example of the usage of `auto_id` in a mapping file. -(Note that we're focussing on the ids and references, so the other attributes have been disregarded.) +Let's consider a very common example of the usage of `auto_id` in a mapping file +(note that we're focussing on the ids and references, other attributes are therefore omitted). ```yaml Nodes: node: @@ -217,7 +212,7 @@ Let's consider a very common example of the usage of `auto_id` in a mapping file key: Number: To_Number ``` -This basically reads as: +This basically reads: * For each row in the Nodes table, a PGM node instance is created. * For each node instance, a numerical id is generated, which is unique for each value in the Number column. This assumes that the Number column is unique in the source table. Let's say tha values of the Number column in that @@ -247,7 +242,7 @@ This basically reads as: ## Advanced AutoID Mapping In some cases, multiple components have to be created for each row in a source table. -In such cases, the `name` attribute may be necessary to create multiple PGM IDs for a single row. Let's consider +In such cases, the `name` attribute may be necessary to create multiple PGM IDs for a single row. Consider this example: ```yaml Transformer loads: @@ -291,7 +286,7 @@ this example: - Subnumber ``` -Let's say we have one Transformer Load connected to the Node Number 103 and it's Subnumber is 1. +Suppose we have one Transformer Load connected to the Node Number 103 and its Subnumber is 1. Then the following IDs will be generated / retrieved: * `transformer.id`: `{"table": "Transformer loads", "name": "transformer", "key" {"Node_Number": 103, "Subnumber": 1} -> 5` diff --git a/docs/converters/vision_converter.md b/docs/converters/vision_converter.md index b6fad62b..97b9aaae 100644 --- a/docs/converters/vision_converter.md +++ b/docs/converters/vision_converter.md @@ -5,42 +5,42 @@ SPDX-License-Identifier: MPL-2.0 # Vision converter -The vision converter converts the excel exports of vision to PGM data. As mentioned in [Converters](converters/converter.md), vision converter is an implementation of the tabular converter. -The mapping of all attributes is stored in the `vision_en.yaml` and `vision_nl.yaml` files in [config](https://github.com/PowerGridModel/power-grid-model-io/tree/main/src/power_grid_model_io/config) directory. +The Vision Excel converter converts the Excel exports from Vision to PGM data format. As mentioned in [Converters](converters/converter.md), Vision Excel converter is an implementation of the tabular converter. +The default mapping of all attributes is stored in the `vision_en.yaml` and `vision_nl.yaml` files in [config](https://github.com/PowerGridModel/power-grid-model-io/tree/main/src/power_grid_model_io/config) directory. +Custom mapping files are supported via passing the file directory to the constructor of the converter. ## Load rate of elements -Certain `elements` in vision, ie. appliances like transformer loads and induction motor have a result parameter of load rate. -In vision the load rate is calculated without considering the simultaneity factor of the connected node. -So we may observe a variation in power inflow/outflow result (ie. P,Q and S) due to different simultaneity factors. But the load rate always corresponds to `simultaneity of loads=1`. +Certain `elements` in Vision, i.e., appliances like transformer loads and induction motor have a result parameter of load rate. +In Vision, load rates are calculated without considering the simultaneity factors of connected nodes. +So we may observe a variation in power inflow/outflow result (i.e., P, Q and S) due to different simultaneity factors. But the load rate always corresponds to `simultaneity of loads=1`. When we make conversion to PGM, the input data attributes of PGM for loads like `p_specified` and `q_specified` are modified as per simultaneity. The resulting loading then takes simultaneity into account. -**Hence, the loading of such elements may not correspond to the load rate obtained in vision** +**Hence, the loading of such elements may not correspond to the load rate obtained in Vision** -## Transformer load modelling +## Transformer load modeling power-grid-model-io converts the transformer load into a individual transformer and a load for usage in power-grid-model. -In vision, the modelling of a transformer load seems to be different from an individual transformer and load. -There is a minor difference in both in the reactive power consumed/generated. -This can correspond to a minor voltage deviation too in the results. +To the best of our knowledge, Vision modeles a transformer load differently than an individual transformer plus load. +There is a minor difference in both the reactive power consumed and generated. +This might correspond to a minor voltage deviation in the results. ```{tip} -It is recommended to split the transformer load into a individual components in vision beforehand to avoid this issue. -This can be done by first selecting the transformer loads: (Start | Select | Object -> Element -> Check Transformer load, Ok) -Then split it into individual components: (Start | Edit | Topological | Split) +To avoid this issue, it is recommended to split the transformer load into individual components in Vision beforehand. +This can be done by first selecting the transformer loads: [Start | Select | Object -> Element -> Check Transformer load -> Ok], then split it into individual components: [Start | Edit | Topological | Split] ``` ## Voltage angle of buses in symmetric power-flow -Note that vision does not include clock angles of transformer for symmetrical calculations in the result of voltage angles. power-grid-model however does consider them so a direct comparison of angle results needs to be done with this knowledge. +Note that in symmetrical calculations, Vision does not include clock angles of transformers in the result of voltage angles. power-grid-model, however, does consider them. Therefore when doing a direct comparison of angle results, this needs to be taken into consideration. -## Modelling differences or unsupported attributes +## Modeling differences or unsupported attributes -Some components are yet to be modelled for conversions because they might not have a straightforward mapping in power-grid-model. Those are listed here. +Some components are yet to be modeled for conversions because they might not have a straightforward mapping in power-grid-model. Those are listed here. - power-grid-model currently does not support PV(Active Power-Voltage) bus and related corresponding features. -- Currently, the efficiency type of PVs(Photovoltaics) element is also unsupported for all types except the `100%` type. For the efficiency type: `0,1 pu: 93 %; 1 pu: 97 %`, the generation power is multiplied by 97% as a closest approximation. -- The conversions for load behaviors of `industry`, `residential`, `business` are not yet modelled. The load behaviors usually do not create a significant difference in power-flow results for most grids when the voltage at bus is close to 1 p.u. Hence, the conversion of the mentioned load behaviors is approximated to be of `Constant Power` type for now. -- The source bus in PGM is mapped with a source impedance. `Sk"nom`, `R/X` and `Z0/Z1` are the attributes used in modelling source impedance. In vision, these attributes are used only for short circuit calculations -- The load rate for transformer is calculated in vision by current ie. `load_rate = max(u1 * I1, u2 * I2) * sqrt(3) / Snom * 100`. Whereas in power-grid-model, loading is calculated by power. ie. `loading = max(s1,s2)/sn`. (Note: The attribute names are as per relevant notation in vision and PGM respectively). This gives a slight difference in load rate of transformer. +- Currently, the efficiency type of PVs(Photovoltaics) element is also unsupported for all types except the `100%` type. For the efficiency type: `0, 1 pu: 93 %; 1 pu: 97 %`, the generation power is multiplied by 97% as a closest approximation. +- The conversions for load behaviors of `industry`, `residential` and `business` are not yet modeled. The load behaviors usually do not create a significant difference in power-flow results for most grids when the voltage at bus is close to `1 pu`. Hence, the conversion of the mentioned load behaviors is approximated to be of `Constant Power` type for the time being. +- The source bus in power-grid-model is mapped with a source impedance. `Sk"nom`, `R/X` and `Z0/Z1` are the attributes used in modeling source impedance. In Vision, these attributes are used only for short circuit calculations +- The load rate for transformer is calculated in Vision by current i.e., `load_rate = max(u1 * I1, u2 * I2) * sqrt(3) / Snom * 100`. Whereas in power-grid-model, loading is calculated by power, i.e., `loading = max(s1,s2)/sn`. (Note: The attribute names are as per relevant notation in Vision and PGM respectively). This gives a slight difference in load rate of transformer. - A minor difference in results is expected since Vision uses a power mismatch in p.u. as convergence criteria whereas power-grid-model uses voltage mismatch. \ No newline at end of file diff --git a/docs/examples/pandapower_example.ipynb b/docs/examples/pandapower_example.ipynb index 3b721730..c169af30 100644 --- a/docs/examples/pandapower_example.ipynb +++ b/docs/examples/pandapower_example.ipynb @@ -472,7 +472,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "To get the results of powerflow in the pandapower net, convert the result from power-grid-model powerflow ie. `output_data` from previous section to the pandapower `res_*` dataframes." + "To get the results of powerflow in the pandapower net, convert the result from power-grid-model powerflow i.e., `output_data` from previous section to the pandapower `res_*` dataframes." ] }, {