Merge branch 'main' into feature/csv-dir-store

docs/converters/vision_converter.md

@@ -0,0 +1,45 @@
+<!--
+SPDX-FileCopyrightText: 2022 Contributors to the Power Grid Model IO project <[email protected]>
+SPDX-License-Identifier: MPL-2.0
+-->
+
+# Vision converter
+
+The vision converter converts the excel exports of vision to PGM data. As mentioned in [Converters](converters/converter.md), vision converter is an implementation of the tabular converter.
+The mapping of all attributes is stored in the `vision_en.yaml` and `vision_nl.yaml` files in [config](https://github.com/alliander-opensource/power-grid-model-io/tree/main/src/power_grid_model_io/config) directory.
+
+## Load rate of elements 
+
+Certain `elements` in vision, ie. appliances like transformer loads and induction motor have a result parameter of load rate.
+In vision the load rate is calculated without considering the simultaneity factor of the connected node.
+So we may observe a variation in power inflow/outflow result (ie. P,Q and S) due to different simultaneity factors. But the load rate always corresponds to `simultaneity of loads=1`.
+
+When we make conversion to PGM, the input data attributes of PGM for loads like `p_specified` and `q_specified` are modified as per simultaneity. The resulting loading then takes simultaneity into account. 
+**Hence, the loading of such elements may not correspond to the load rate obtained in vision**
+
+## Transformer load modelling
+
+power-grid-model-io converts the transformer load into a individual transformer and a load for usage in power-grid-model. 
+In vision, the modelling of a transformer load seems to be different from an individual transformer and load.
+There is a minor difference in both in the reactive power consumed/generated. 
+This can correspond to a minor voltage deviation too in the results.
+
+```{tip}
+It is recommended to split the transformer load into a individual components in vision beforehand to avoid this issue.
+This can be done by first selecting the transformer loads: (Start | Select | Object -> Element -> Check Transformer load, Ok)
+Then split it into individual components: (Start | Edit | Topological | Split)
+```
+
+## Voltage angle of buses in symmetric power-flow
+
+Note that vision does not include clock angles of transformer for symmetrical calculations in the result of voltage angles. power-grid-model however does consider them so a direct comparison of angle results needs to be done with this knowledge.
+
+## Modelling differences or unsupported attributes
+
+Some components are yet to be modelled for conversions because they might not have a straightforward mapping in power-grid-model. Those are listed here.
+
+- power-grid-model currently does not support PV(Active Power-Voltage) bus and related corresponding features. 
+- Currently, the efficiency type of PVs(Photovoltaics) element is also unsupported for all types except the `100%` type.
+- The conversions for load behaviors of `industry`, `residential`, `business` are not yet modelled. The load behaviors usually do not create a significant difference in power-flow results for most grids when the voltage at bus is close to 1 p.u. Hence, the conversion of the mentioned load behaviors is approximated to be of `Constant Power` type for now. 
+- The source bus in PGM is mapped with a source impedance. `Sk"nom`, `R/X` and `Z0/Z1` are the attributes used in modelling source impedance. In vision, these attributes are used only for short circuit calculations
+- A minor difference in results is expected since Vision uses a power mismatch in p.u. as convergence criteria whereas power-grid-model uses voltage mismatch.

docs/index.md

@@ -33,6 +33,7 @@ quickstart.md
 :maxdepth: 2
 converters/converter.md
 converters/tabular_converter.md
+converters/vision_converter.md
 ```
 
 ```{toctree}

src/power_grid_model_io/config/excel/vision_en.yaml

@@ -428,7 +428,7 @@ grid:
           key:
             Number: Node.Number
       status: Switch state
-      type: 1
+      type: 0
       p_specified:
         power_grid_model_io.functions.value_or_default:
           value: Pref

src/power_grid_model_io/functions/phase_to_phase.py

@@ -6,21 +6,19 @@
 """
 
 import math
-import re
 from typing import Tuple
 
 from power_grid_model import WindingType
 
 from power_grid_model_io.functions import get_winding
-
-CONNECTION_PATTERN = re.compile(r"(Y|YN|D|Z|ZN)(y|yn|d|z|zn)(\d|1[0-2])")
+from power_grid_model_io.utils.regex import TRAFO_CONNECTION_RE
 
 
 def relative_no_load_current(i_0: float, p_0: float, s_nom: float, u_nom: float) -> float:
     """
     Calculate the relative no load current.
     """
-    i_rel = max(i_0 / (s_nom / (u_nom / math.sqrt(3))), p_0 / s_nom)
+    i_rel = max(i_0 / (s_nom / (u_nom * math.sqrt(3))), p_0 / s_nom)
     if i_rel > 1.0:
         raise ValueError(f"Relative current can't be more than 100% (got {i_rel * 100.0:.2f}%)")
     return i_rel
@@ -40,13 +38,17 @@ def power_wind_speed(  # pylint: disable=too-many-arguments
     nominal_wind_speed: float = 14.0,
     cutting_out_wind_speed: float = 25.0,
     cut_out_wind_speed: float = 30.0,
+    axis_height: float = 30.0,
 ) -> float:
     """
     Estimate p_ref based on p_nom and wind_speed.
 
     See section "Wind turbine" in https://phasetophase.nl/pdf/VisionEN.pdf
     """
 
+    # Calculate wind speed at the axis height
+    wind_speed *= (axis_height / 10) ** 0.143
+
     # At a wind speed below cut-in, the power is zero.
     if wind_speed < cut_in_wind_speed:
         return 0.0
@@ -109,7 +111,7 @@ def _split_connection_string(conn_str: str) -> Tuple[str, str, int]:
      * winding_to
      * clock
     """
-    match = CONNECTION_PATTERN.fullmatch(conn_str)
+    match = TRAFO_CONNECTION_RE.fullmatch(conn_str)
     if not match:
         raise ValueError(f"Invalid transformer connection string: '{conn_str}'")
     return match.group(1), match.group(2), int(match.group(3))

src/power_grid_model_io/utils/regex.py

@@ -0,0 +1,42 @@
+# SPDX-FileCopyrightText: 2022 Contributors to the Power Grid Model project <[email protected]>
+#
+# SPDX-License-Identifier: MPL-2.0
+"""
+General regular expressions
+"""
+
+import re
+
+TRAFO_CONNECTION_RE = re.compile(r"^(Y|YN|D|Z|ZN)(y|yn|d|z|zn)(\d|1[0-2])?$")
+r"""
+Regular expressions to the winding_from and winding_to codes and optionally the clock number:
+^               Start of the string
+(Y|YN|D|Z|ZN)   From winding type
+(y|yn|d|z|zn)   To winding type
+(\d|1[0-2])?    Optional clock number (0-12)
+$               End of the string
+"""
+
+TRAFO3_CONNECTION_RE = re.compile(r"^(Y|YN|D|Z|ZN)(y|yn|d|z|zn)(y|yn|d|z|zn)(\d|1[0-2])?$")
+r"""
+Regular expressions to the winding_1, winding_2 and winding_3 codes and optionally the clock number:
+^               Start of the string
+(Y|YN|D|Z|ZN)   First winding type
+(y|yn|d|z|zn)   Second winding type
+(y|yn|d|z|zn)   Third winding type
+(\d|1[0-2])?    Optional clock number (0-12)
+$               End of the string
+"""
+
+NODE_REF_RE = re.compile(r"^(.+_)?node(_.+)?$")
+r"""
+Regular expressions to match the word node with an optional prefix or suffix, e.g.:
+    - node
+    - from_node
+    - node_1
+^               Start of the string
+(.+_)?          Optional prefix, ending with an underscore
+node            The word 'node'
+(_.+)?          Optional suffix, starting with in an underscore
+$               End of the string
+"""

tests/unit/functions/test_phase_to_phase.py

@@ -3,7 +3,7 @@
 # SPDX-License-Identifier: MPL-2.0
 import numpy as np
 from power_grid_model.enum import WindingType
-from pytest import approx, mark, raises
+from pytest import approx, mark, param, raises
 
 from power_grid_model_io.functions.phase_to_phase import (
     get_clock,
@@ -20,8 +20,8 @@
     ("i_0", "p_0", "s_nom", "u_nom", "expected"),
     [
         (float("nan"), float("nan"), float("nan"), float("nan"), float("nan")),
-        (5.0, 1000.0, 100000.0, 400.0, 0.011547005383792516),
-        (5.0, 2000.0, 100000.0, 400.0, 0.02),
+        (5.0, 1000.0, 100000.0, 400.0, 0.0346410161513775),
+        (5.0, 4000.0, 100000.0, 400.0, 0.04),
     ],
 )
 def test_relative_no_load_current(i_0: float, p_0: float, s_nom: float, u_nom: float, expected: float):
@@ -30,7 +30,7 @@ def test_relative_no_load_current(i_0: float, p_0: float, s_nom: float, u_nom: f
 
 
 def test_relative_no_load_current__exception():
-    with raises(ValueError, match="can't be more than 100% .* 115.47%"):
+    with raises(ValueError, match="can't be more than 100% .* 346.41%"):
         relative_no_load_current(500.0, 1000.0, 100000.0, 400.0)
 
 
@@ -47,22 +47,27 @@ def test_reactive_power(p: float, cos_phi: float, expected: float):
 
 
 @mark.parametrize(
-    ("wind_speed", "expected"),
+    ("kwargs", "expected"),
     [
-        (0.0, 0.0),
-        (1.5, 0.0),
-        (3.0, 0.0),  # cut-in
-        (8.5, 125000.0),
-        (14.0, 1000000.0),  # nominal
-        (19.5, 1000000.0),
-        (25.0, 1000000.0),  # cutting-out
-        (27.5, 500000.0),
-        (30.0, 0.0),  # cut-out
-        (100.0, 0.0),
+        param({"wind_speed": 0.0, "axis_height": 10.0}, 0.0, id="no-wind"),
+        param({"wind_speed": 1.5, "axis_height": 10.0}, 0.0, id="half-way-cut-in"),
+        param({"wind_speed": 3.0, "axis_height": 10.0}, 0.0, id="cut-in"),
+        param({"wind_speed": 8.5, "axis_height": 10.0}, 125000.0, id="cut-in-to-nominal"),
+        param({"wind_speed": 14.0, "axis_height": 10.0}, 1000000.0, id="nominal"),  # nominal
+        param({"wind_speed": 19.5, "axis_height": 10.0}, 1000000.0, id="nominal-to-cutting-out"),
+        param({"wind_speed": 25.0, "axis_height": 10.0}, 1000000.0, id="cutting-out"),
+        param({"wind_speed": 27.5, "axis_height": 10.0}, 500000.0, id="cutting-out-to-cut-out"),
+        param({"wind_speed": 30.0, "axis_height": 10.0}, 0.0, id="cut-out"),
+        param({"wind_speed": 50.0, "axis_height": 10.0}, 0.0, id="more-than-cut-out"),
+        # 30 meters high
+        param({"wind_speed": 3.0, "axis_height": 30.0}, 99.86406950142123, id="cut-in-at-30m"),
+        param({"wind_speed": 20.0, "axis_height": 30.0}, 1000000.0, id="nominal-at-30m"),
+        param({"wind_speed": 25.0, "axis_height": 30.0}, 149427.79246831674, id="cutting-out-at-30m"),
+        param({"wind_speed": 25.63851786, "axis_height": 30.0}, 0.0, id="cut-out-at-30m"),
     ],
 )
-def test_power_wind_speed(wind_speed, expected):
-    assert power_wind_speed(1e6, wind_speed) == approx(expected)
+def test_power_wind_speed(kwargs, expected):
+    assert power_wind_speed(p_nom=1e6, **kwargs) == approx(expected)
 
 
 @mark.parametrize(

tests/unit/utils/test_regex.py

@@ -0,0 +1,59 @@
+# SPDX-FileCopyrightText: 2022 Contributors to the Power Grid Model project <[email protected]>
+#
+# SPDX-License-Identifier: MPL-2.0
+
+import pytest
+
+from power_grid_model_io.utils.regex import NODE_REF_RE, TRAFO3_CONNECTION_RE, TRAFO_CONNECTION_RE
+
+
+def test_trafo_connection__pos():
+    assert TRAFO_CONNECTION_RE.fullmatch("Dyn").groups() == ("D", "yn", None)
+    assert TRAFO_CONNECTION_RE.fullmatch("Yyn").groups() == ("Y", "yn", None)
+    assert TRAFO_CONNECTION_RE.fullmatch("Yzn").groups() == ("Y", "zn", None)
+    assert TRAFO_CONNECTION_RE.fullmatch("YNy").groups() == ("YN", "y", None)
+    assert TRAFO_CONNECTION_RE.fullmatch("Dy5").groups() == ("D", "y", "5")
+    assert TRAFO_CONNECTION_RE.fullmatch("Dy11").groups() == ("D", "y", "11")
+
+
+def test_trafo_connection__neg():
+    assert not TRAFO_CONNECTION_RE.fullmatch("Xyn")
+    assert not TRAFO_CONNECTION_RE.fullmatch("yyn")
+    assert not TRAFO_CONNECTION_RE.fullmatch("YZN")
+    assert not TRAFO_CONNECTION_RE.fullmatch("YNx")
+    assert not TRAFO_CONNECTION_RE.fullmatch("Dy13")
+    assert not TRAFO_CONNECTION_RE.fullmatch("Dy-1")
+
+
+def test_trafo3_connection__pos():
+    assert TRAFO3_CONNECTION_RE.fullmatch("Dynyn").groups() == ("D", "yn", "yn", None)
+    assert TRAFO3_CONNECTION_RE.fullmatch("Yynd").groups() == ("Y", "yn", "d", None)
+    assert TRAFO3_CONNECTION_RE.fullmatch("Yzny").groups() == ("Y", "zn", "y", None)
+    assert TRAFO3_CONNECTION_RE.fullmatch("YNdz").groups() == ("YN", "d", "z", None)
+    assert TRAFO3_CONNECTION_RE.fullmatch("Dyy5").groups() == ("D", "y", "y", "5")
+    assert TRAFO3_CONNECTION_RE.fullmatch("Dyd11").groups() == ("D", "y", "d", "11")
+
+
+def test_trafo3_connection__neg():
+    assert not TRAFO3_CONNECTION_RE.fullmatch("Xynd")
+    assert not TRAFO3_CONNECTION_RE.fullmatch("ydyn")
+    assert not TRAFO3_CONNECTION_RE.fullmatch("DYZN")
+    assert not TRAFO3_CONNECTION_RE.fullmatch("YNxd")
+    assert not TRAFO3_CONNECTION_RE.fullmatch("Dyd13")
+    assert not TRAFO3_CONNECTION_RE.fullmatch("DyD13")
+    assert not TRAFO3_CONNECTION_RE.fullmatch("Dynd-1")
+
+
+def test_node_ref__pos():
+    assert NODE_REF_RE.fullmatch("node")
+    assert NODE_REF_RE.fullmatch("from_node")
+    assert NODE_REF_RE.fullmatch("to_node")
+    assert NODE_REF_RE.fullmatch("node_1")
+    assert NODE_REF_RE.fullmatch("node_2")
+    assert NODE_REF_RE.fullmatch("node_3")
+
+
+def test_node_ref__neg():
+    assert not NODE_REF_RE.fullmatch("nodes")
+    assert not NODE_REF_RE.fullmatch("anode")
+    assert not NODE_REF_RE.fullmatch("immunodeficient")