We are still at the early stage of the implementation. There will be more functionalities and flexible I/Os coming in the future. Please watch us progress to have the latest update.
This repo is developed and maintained by Wangkun Xu at Control and Power research group, Imperial College London.
The main purpose of this repository is to
- Provide an efficient I/O for generating the power system testbeds for optimization problem.
- Host a set of basic power system operation formulations for the future research and teaching purposes. This repo contains some basic power system operations written in Python and formulated by
cvxpy, such as:- Network Constrained Unit Commitment (with/out integer variables, and various tim e steps) (finished)
- Economic Dispatch (finished)
- Stochastic Unit Commitment (ongoing)
- The package also comes with an efficient modifications of the load, solar, and wind data that are suitable for the proposed power system case study. Therefore, it can be used to train machine and deep learning models with large training dataset.
cvxpy: is an open source Python-embedded modeling language for convex optimization problems. It lets you express your problem in a natural and matrix way that follows the math, rather than in the restrictive standard form required by different solvers.
Note: you may also need to have Gurobi, Mosek or other optimization software to efficiently solve the optimization problems, especially if integers are included. Please refer here for details.
PyPower: is a power flow and Optimal Power Flow (OPF) solver. It also hosts a set of commonly used power system testbeds. It is a part of MATPOWER to the Python programming language.
Other packages inlcudes
openpyxl, XlsxWriter
for efficient I/O of the .xlsx files.
The implementation of this repo follows the online cource here and the textbook Power System Operations here, both by Prof. Antonio Conejo. We also write a series of blog posts to explain the formulation used in the code, including:
| Case Name | Data Generation | Grid Formulation | Continuous UC and ED | Binary UC and ED | Stochastic UC and ED |
|---|---|---|---|---|---|
| case14 | ✔️ | ✔️ | ✔️ | ❓ | ❌ |
| case39 | ✔️ | ✔️ | ✔️ | ❓ | ❌ |
| case118 | ✔️ | ✔️ | ✔️ | ❓ | ❌ |
Note: the data generation is not necessary for power system operation. You can consider use this part to generate dataset for training machine learning models, such as load, solar, and wind forecasting.
We use open source dataset from A Synthetic Texas Backbone Power System with Climate-Dependent Spatio-Temporal Correlated Profiles. You can download/read the descriptions of the dataset from the official webpage.
Please cite/recogenize the original authors if you use the dataset.
After download the the .zip file into the data/ and change the name into raw_data.zip, unzip the file by
unzip data/raw_data.zip -d data/This will give a new folder data/Data_public/.
Then we need to group the feature-label pairs for each data (bus). There are 123 buses in the Texas backbone power system. To group the data, run
from data import group_data
group_data()This will result in folder data/data_grouped/ which contains .csv files for all 123 buses dataset.
Note: This may take several minutes but you only need to run this script once.
The optimization formulation relies on reading system configuration from a .xlsx file. There are several ways to construct the configuration file, either from scratch or build it from existing configurations via the PyPower package. An example file can be found here.
We recommend to construct the .xlsx file from the basic PyPower file to avoid errors. The PyPower contains useful grid topology and parameters that can be directly read by the package. However, you must include several necessary extra configs (that are not covered by the PyPower) to support the full functionality of power system operation. For example, you need to specifiy the up and down cost and limit of the generators to formulate the UC. An example can be found here. The detailed description on how to construct the extra config file can be found here.
For instance, to generate the case14, run
from utils.loading import from_pypower
from_pypower('case14', 'configs/case14_default.json')The function from_pypower will generate the case14.xlsx file in the configs/ folder. It will also rescale the default load, solar, and wind data to the defined power_ratio in the config file.
The next step is to assign the grouped data to the buses defined in the configuration file. First, the grouped data will be first assigned to the solar or wind buses. Then, the remaining bus will be assigned with the remaining grouped data. The solar, wind, and load will be rescaled into their default values.
For example, to assign the data to the case14, run
from utils.modufy_data import assign_data
assign_data(
xlsx_dir = f'configs/case14.xlsx',
save_dir = f"data/case14/",
seed = 0,
force_new = True
)When force_new is set to True, the function will overwrite the existing files.
A new folder will be constructed with the modified grouped data in the sequence of the load.
The thermal limit pf_max can be automatically altered based on the previously defined grid configuration, load level, generator level, and renewable levels so that the operation cannot be trivially solved.
To achieve this for bus14 system, run
from utils.modufy_data import modify_pfmax
modify_pfmax(grid_op, with_int, T,
f"data/case14/",
min_pfmax = 0.1,
xlsx_dir = f"configs/case14.xlsx")This will overwrite the pf_max column in the generated .xlsx file.
The main.py function is an end-to-end approach to generate all the configurations and data mentioend above. You can run the script by
python main.py -n case14 -c configs/case14_default.json -ffor the case14 example. -n is short for --pypower_case_name, -c is short for --extra_config_path, and -f is short for --force_new to generate new data. The -f flag will overwrite the existing files.
The functions in utils/standard_form.py are developed to reformulate the UC/ED in cvxpy form into the correspinding standard form. This conversion is general in addition to the UC/ED. Therefore it can be used outside power system operation. In this sense, you can "standardize" your problem by leveraging the descriptive power of cvxpy, which can be useful for developing your own optimization algorithm or for physics-informed machine learning.
For a genenal QP/LP without integer variable, it transforms into:
$$
\begin{array}{rl}
\min & (1/2) x^TPx + q^Tx \
\text{s.t.} & Ax = \sum_i B_i z_i + b \
& Gx \leq \sum_i H_i z_i + h \
\end{array}
$$
where
For a MIQP/MILP, it transforms into:
$$
\begin{array}{rl}
\min & (1/2) x^TPx + q^Tx \
\text{s.t.} & Ax = \sum_i B_i z_i + b \
& Gx \leq \sum_i H_i z_i + h \
& x[\text{bool_idx}] \in {0, 1}
\end{array}
$$
where bool_idx is the index of the binary (or integer) variables.
The package comes with several ready-to-use test files in test/. You can learn most of the operations by reading the test files.
test/data.py: test if the data generation is correct. E.g., if the assigned load and renewable data have correct maximum values.
test/grid_formulation.py: test if the grid matrices are the same to the PyPower package.
Part of the dataset and power system optimization toolbox has been used in our previous paper:
- Xu, Wangkun, Jianhong Wang, and Fei Teng. "E2E-AT: A Unified Framework for Tackling Uncertainty in Task-Aware End-to-End Learning." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 38. No. 14. 2024.
- Xu, Wangkun, and Fei Teng. "Task-aware machine unlearning and its application in load forecasting." IEEE Transactions on Power Systems (2024).