Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Watt-time plugin #65

Merged
merged 30 commits into from
Apr 9, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
b623fa0
feat(package): remove dayjs and add luxon
manushak Mar 27, 2024
62f8508
test(lib): remove test from watt-time
manushak Mar 27, 2024
6c5152b
feat(lib): change dayjs to luxon, validates credentials
manushak Mar 27, 2024
b53fa47
feat(lib): add description to function
manushak Mar 27, 2024
b119342
feat(lib): improve watt time plugin
manushak Mar 27, 2024
852f194
test(lib): fix global config in watt-time test
manushak Mar 27, 2024
e1f953c
docs(lib): improve readme file of watt-time plugin
manushak Mar 27, 2024
32e708f
feat(lib): add RegionFromLocationResponse type
manushak Apr 2, 2024
15bcde3
feat(lib): depracate v2 version
manushak Apr 2, 2024
fe57332
docs(lib): update the doc, deprecate v2 version
manushak Apr 2, 2024
3473942
test(lib): update and skip the tests
manushak Apr 2, 2024
b832cbb
docs(lib): update geolocation to retrieve CASIO-NORTH region data
manushak Apr 2, 2024
8954529
docs(lib): update yaml in the doc
manushak Apr 3, 2024
11ebf53
docs(lib): update the geolocation
manushak Apr 4, 2024
930ddd9
fix(lib): add check for geolocation/ 'cloud/region-wt-id'
manushak Apr 4, 2024
f75adbc
Merge branch 'main' into watt-time-plugin
manushak Apr 5, 2024
297240b
fix(mocks): removed unused mock data.json file
manushak Apr 5, 2024
c49ace9
fix(mocks): update mock requests data
manushak Apr 5, 2024
fd45c9f
test(lib): update test
manushak Apr 5, 2024
f3da9f6
fix(lib): update error message
manushak Apr 5, 2024
2ebad36
docs(lib): update the doc related to review comments
manushak Apr 5, 2024
91451a8
docs(lib): remove `if-version` from the output
manushak Apr 6, 2024
ca3cc03
feat(mocks): add mock test cases
manushak Apr 6, 2024
bf227d3
fix(lib): fix error message and API data sort
manushak Apr 6, 2024
9dc1f6b
fix(lib): make minor improvments
manushak Apr 6, 2024
1534b7b
test(lib): add full test coverage
manushak Apr 6, 2024
a1afbbb
fix(lib): add `signal_type` to geolocation params
manushak Apr 8, 2024
c2211cb
fix(lib): add console warn if the duration is bellow then 300
manushak Apr 9, 2024
7a0b81e
fix(lib): add catch for authorization error of WattTime API
manushak Apr 9, 2024
682e585
test(lib): add mock data and cover test for axios error
manushak Apr 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 15 additions & 6 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,12 @@
"@azure/identity": "^3.4.1",
"@cloud-carbon-footprint/aws": "^0.15.0",
"@tgwf/co2": "^0.14.1",
"@types/luxon": "^3.4.2",
"@types/tgwf__co2": "^0.0.0",
"axios": "^1.6.0",
"dayjs": "^1.11.10",
"dotenv": "16.3.1",
"js-yaml": "^4.1.0",
"luxon": "^3.4.4",
"typescript": "^5.1.6",
"typescript-cubic-spline": "^1.0.1",
"zod": "^3.22.4"
Expand Down Expand Up @@ -77,4 +78,4 @@
"prepublish": "npm run build",
"test": "jest --verbose"
}
}
}
30 changes: 1 addition & 29 deletions src/__tests__/unit/lib/watt-time/index.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ describe('lib/watt-time: ', () => {

describe('init WattTimeGridEmissions: ', () => {
it('initalizes object with properties.', async () => {
const output = WattTimeGridEmissions({});
const output = WattTimeGridEmissions();

expect(output).toHaveProperty('metadata');
expect(output).toHaveProperty('execute');
Expand Down Expand Up @@ -228,34 +228,6 @@ describe('lib/watt-time: ', () => {
expect(error).toEqual(new APIRequestError(errorMessage));
}
});

it('throws an error when span is more than 32 days.', async () => {
const errorMessage =
'WattTimeGridEmissions: WattTime API supports up to 32 days. Duration of 31537200 seconds is too long.';
process.env.WATT_TIME_USERNAME = 'test1';
process.env.WATT_TIME_PASSWORD = 'test2';

const output = WattTimeGridEmissions();
expect.assertions(2);

try {
await output.execute([
{
geolocation: '37.7749,-122.4194',
timestamp: '2021-01-01T00:00:00Z',
duration: 1200,
},
{
geolocation: '37.7749,-122.4194',
timestamp: '2022-01-01T00:00:00Z',
duration: 1200,
},
]);
} catch (error) {
expect(error).toBeInstanceOf(InputValidationError);
expect(error).toEqual(new InputValidationError(errorMessage));
}
});
});
});
});
160 changes: 100 additions & 60 deletions src/lib/watt-time/README.md
Original file line number Diff line number Diff line change
@@ -1,38 +1,25 @@
# WattTime Grid Emissions plugin
# WattTime Grid Emissions Plugin

> [!NOTE] > `Watt-time` is a community plugin, not part of the IF standard library. This means the IF core team are not closely monitoring these plugins to keep them up to date. You should do your own research before implementing them!
> [!NOTE] >
> `Watt-time` is a community plugin and not a part of the IF standard library. As a result, the IF core team does not closely monitor these plugins for updates. It is recommended to conduct your own research before implementing them.

## Introduction

WattTime technology—based on real-time grid data, cutting-edge algorithms, and machine learning—provides first-of-its-kind insight into your local electricity grid’s marginal emissions rate. [Read More...](https://www.watttime.org/api-documentation/#introduction)

## Scope
## Overview

WattTime plugin provides a way to calculate emissions for a given time in a specific geolocation.

The plugin is based on the WattTime API. The plugin uses the following inputs:

- `timestamp`: Timestamp of the recorded event (2021-01-01T00:00:00Z) RFC3339
- `duration`: Duration of the recorded event in seconds (3600)
- `geolocation`: Location of the software system (latitude in decimal degrees, longitude in decimal degrees). "latitude,longitude"
- `cloud/region-geolocation`: The same as `geolocation`, with calculations performed by the `cloud-metadata` plugin
- `cloud/region-wt-id`: Region abbreviation associated with location (e.g. 'CAISO_NORTH')
- `signal-type`: The signal type of selected region (optional) (e.g 'co2_moer')

Either `geolocation`,`cloud/region-wt-id` or `cloud/region-geolocation` should be provided.

## Implementation

Limitations:

- Set of inputs are to be within 32 days of each other.
- Emissions are aggregated for every 5 minutes regardless of the granularity of the inputs.
The `WattTimeGridEmissions` plugin is designed to compute the average carbon emissions of a power grid over a specified duration. It leverages data from the WattTime API to furnish carbon intensity information for precise locations and timeframes. This plugin proves beneficial for applications requiring carbon footprint monitoring or optimization, such as energy management systems or environmental impact assessments. The plugin supports both v2 and v3 versions of the WattTime API. The API returns data in `lbs/MWh`, which the plugin converts to `Kg/MWh` (g/KWh) by dividing by `0.453592`.

### Authentication

WattTime API requires activation of subscription before usage. Please refer to the [WattTime website](https://watttime.org/docs-dev/data-plans/) for more information.

Create a `.env` file in the IF project root directory. This is where you can store your WattTime authentication details. Your `.env` file should look as follows:
## Prerequisites

Before utilizing this plugin, ensure the following prerequisites are fulfilled:

1. **Environment Variables**: The plugin requires environment variables `WATT_TIME_USERNAME` and `WATT_TIME_PASSWORD` to be set. These credentials are utilized for authentication with the WattTime API.

**Required Parameters:**

Expand All @@ -47,81 +34,134 @@ WATT_TIME_PASSWORD: <your-password>
WATT_TIME_TOKEN: <your-token>
```

### Plugin global config
2. **Dependencies**: Confirm the installation of all required dependencies, including `luxon` and `zod`. These dependencies are imperative for date-time manipulation and input validation, respectively.

- `base-url`: The URL for the WattTime API endpoint.
## Usage

### Inputs
To employ the `WattTimeGridEmissions` plugin, adhere to these steps:

**Required Parameters:**
1. **Initialize Plugin**: Import the `WattTimeGridEmissions` function and initialize it with optional global configuration parameters.

2. **Execute Plugin**: Invoke the `execute` method of the initialized plugin instance with an array of input parameters. Each input parameter should include a `timestamp`, `duration`, and either `geolocation`, `cloud/region-wt-id`, or `cloud/region-geolocation` information.

manushak marked this conversation as resolved.
Show resolved Hide resolved
3. **Result**: The plugin will return an array of plugin parameters enriched with the calculated average carbon intensity (`grid/carbon-intensity`) for each input.

## Input Parameters

The plugin expects the following input parameters:

- `timestamp`: A string representing the start time of the query period.
- `duration`: A number indicating the duration of the query period in seconds.
- `geolocation`: A string representing the latitude and longitude of the location in the format `latitude,longitude`. Alternatively, this information can be provided through `cloud/region-geolocation` or `cloud/region-wt-id` parameters.
- `cloud/region-geolocation`: Similar to `geolocation`, with calculations performed by the `cloud-metadata` plugin.
- `cloud/region-wt-id`: A string representing the region abbreviation associated with the location (e.g., 'CAISO_NORTH').
- `signal-type`: A string representing the signal type of the selected region (optional) (e.g., 'co2_moer').

## Output

The plugin enriches each input parameter with the average carbon intensity (`grid/carbon-intensity`) calculated over the specified duration.

## Error Handling

The plugin conducts input validation using the `zod` library and may throw errors if the provided parameters are invalid or if there are authentication or data retrieval issues with the WattTime API.

#### Algorithm of the Plugin

manushak marked this conversation as resolved.
Show resolved Hide resolved
1. **Initialization**: Authenticate with the WattTime API using the provided credentials. If the `token` is not provided in the environment variables, it uses `username` and `password`, otherwise, it throws an error. To authenticate users, the plugin utilizes the `https://api.watttime.org/login` URL.

2. **Execution**:

- Iterate through each input.

- If `cloud/region-wt-id` is provided, the plugin sets it to `region`, and `signal-type` to `signal_type`, and sends a request to `https://api.watttime.org/v3/forecast/historical` with calculated `start` and `end` times as well. If the `signal_type` is not provided, the plugin requests `https://api.watttime.org/v3/my-access` to obtain access to the account and takes the first signal type.
- If `geolocation` is provided, the plugin parses it to `latitude` and `longitude` and sends a request to `https://api2.watttime.org/v2/data` with calculated `starttime` and `endtime` as well.
manushak marked this conversation as resolved.
Show resolved Hide resolved

- Validate input parameters. If `cloud/region-geolocation` is provided, the `geolocation` is overridden. If `cloud/region-wt-id` is provided, it takes precedence over the `geolocation`.

- Retrieve WattTime data for the specified duration. The WattTime API adds aggregated emissions for every 5 minutes. To address this limitation, the plugin sets the previous emission's value if the specified `duration` is less than 5 minutes.

- `timestamp`: Timestamp of the recorded event (2021-01-01T00:00:00Z) RFC3339
- `duration`: Duration of the recorded event in seconds (3600)
- `geolocation`: Location of the software system (latitude in decimal degrees, longitude in decimal degrees). "latitude,longitude"
- `cloud/region-geolocation`: The same as `geolocation`, with calculations performed by the `cloud-metadata` plugin
- `cloud/region-wt-id`: Region abbreviation associated with location (e.g. 'CAISO_NORTH')
- Calculate average emissions based on retrieved data. The WattTime API returns full data for the entire duration; the plugin checks if the data's period time is within the specified input range and collects data in `kgMWh`.

Either `geolocation`,`cloud/region-wt-id` or `cloud/region-geolocation` should be provided.
3. **Output**: Return results with the average grid emissions for each input.

### Typescript Usage
### TypeScript Usage

```typescript
// environment variable configuration
// export WATT_TIME_USERNAME=test1
// export WATT_TIME_PASSWORD=test2
// use environment variables to configure the plugin
const output = WattTimeGridEmissions();
const result = await output.execute([
```ts
// Initialize the plugin
const plugin = WattTimeGridEmissions();

// Execute the plugin with input parameters
const inputs = [
{
timestamp: '2021-01-01T00:00:00Z',
geolocation: '43.22,-80.22',
duration: 3600,
timestamp: '2024-03-26T12:00:000Z',
duration: 3600, // 1 hour
geolocation: '37.7749,-122.4194', // San Francisco, CA
},
]);
// Add more input parameters as needed
];
const result = await plugin.execute(inputs);

console.log(result);
```

### Manifest Usage

#### Input for manifest
#### Input

```yaml
inputs:
- timestamp: 2021-01-01T00:00:00Z
geolocation: '43.22,-80.22'
duration: 3600
name: watt-time
description: simple demo invoking watt-time
tags:
initialize:
plugins:
watt-time:
method: WattTimeGridEmissions
path: '@grnsft/if-unofficial-plugins'
tree:
manushak marked this conversation as resolved.
Show resolved Hide resolved
children:
child:
pipeline:
- watt-time
inputs:
- timestamp: '2024-03-05T00:00:00.000Z'
duration: 3600
geolocation: 37.7749,-122.4194
```

## Example manifest
#### Output

```yaml
name: watt-time
description: simple demo invoking watt-time
tags:
tags: null
initialize:
plugins:
watt-time:
method: WattTimeGridEmissions
path: '@grnsft/if-unofficial-plugins'
method: WattTimeGridEmissions
outputs:
- yaml
if-version: v0.3.1
tree:
manushak marked this conversation as resolved.
Show resolved Hide resolved
children:
child:
pipeline:
- watt-time
inputs:
- timestamp: 2023-07-06T00:00
- timestamp: '2024-03-05T00:00:00.000Z'
duration: 3600
geolocation: 37.7749,-122.4194
outputs:
- timestamp: '2024-03-05T00:00:00.000Z'
duration: 3600
geolocation: 37.7749,-122.4194
grid/carbon-intensity: 287.7032521512652
```

You can run this by passing it to `ie`. Run impact using the following command run from the project root:
You can execute this by passing it to `ie`. Run the impact using the following command from the project root:

```sh
npm i -g @grnsft/if
npm i -g @grnsft/if-unofficial-plugins
ie --manifest ./examples/manifests/test/watt-time.yml --output ./examples/outputs/watt-time.yml
```

## Position and effects in the manifest:

- Technically, WattTime plugin sets (or overwrites any preconfigured value of) the `grid/carbon-intensity` attribute.
- As such, it should be positioned before the _sci-o_ plugin, if such a plugin is used.
Loading
Loading