-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for AZ-separated quotas #614
Conversation
implementation of AZSeparatedResourceTopology
separated AZ quota will now sync the resource quota and AZ quota to the backend
Add subfunction to sort map keys to keep the order of errors uniform
Remove config option CommitmentIsAZAware with resource toplogy checks. Modify unit tests to include toplogy configs where necessary
5cea98f
to
8f5f123
Compare
rearrange flatresourcetoplogy condition
fix error slice declaration. Co-authored-by: Stefan Majewsky <[email protected]>
return an non fatal error when toplogy is not set. Co-authored-by: Stefan Majewsky <[email protected]>
scrape now makes an explicit choice about how to fill backendQuota. Co-authored-by: Stefan Majewsky <[email protected]>
ACPQ: remove wrongly added allocated value for AZ separated base quota application. Unit test fixes follow. Co-authored-by: Stefan Majewsky <[email protected]>
Because internal algorithms depend on a topology being chosen.
…ead of using service id using the service ID causes an update to all resources in this service disregarding the true topology.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We're getting there. A few more nitpicks, and here's a big thing on top: I don't know how to flag this on a specific line, so I'm going to write it here. If the code works the same as in my head, we will have a neverending series of useless quota syncs on resources with AZSeparatedResourceTopology. As per the spec, these resources should have ResourceUsageReport.Quota == nil
(i.e. no total quota should be reported), which means that project_resources.backend_quota
will be empty. ACPQ currently always fills project_resources.quota
for the resources that it operates on, so on the resource level, we always have quota != backend_quota
and therefore a quota sync is forced during each ACPQ run.
My gut feeling is that ACPQ should just not fill project_resources.quota
for AZSeparateResourceTopology resources. (The absence of the overall quota could then also be a trigger for the UI to hide the region-wide quota/usage bar.) But I have not finished thinking about whether that will induce corner cases elsewhere.
quota sync: avoid overwrite targetQuota map override from other AZs if the map is already initialized. Co-authored-by: Stefan Majewsky <[email protected]>
This would otherwise lead to unecessary quota syncs
I did not know that it is intended for the liquid to return a nil value if it has the AZ separated toplogy. I included a skip for setting the ACPQ quota and provided an explanation for it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have pushed a few nitpick fixes. Also, there is one last bigger thing:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome, thanks!
This implementation handles the introduction of resource topologies. This allows the handling of synchronizing AZ-aware and AZ-separated quotas with the backend.
The plugins will check if a liquid provides a valid toplogy. Invalid AZ's that are detected during scrapes will be rejected. Scraped AZ-separated quota will be included into the
backend_quota
column that resides in theproject_az_resources
table. From limes quota will then be synced with the backend if necessary. Thebackend_quota
values will then be updated to the calculatedquota
values.Things to note:
* The AZ key sorting in the uitlity functions might be able to be improved.*MatchLiquidReportToTopology
might collect the errors for all resources instead of failing at the first mismatch.* SetQuota required a new type to support passing through the AZ-aware qutoas to the backend. We might have to adjust the naming of the structure and its contents.TODOS:
ResourceBehavior.CommitmentIsAZAware