Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flavor based reservation #1

Draft
wants to merge 66 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
a1ddb0c
Fix context unit test
priteau Mar 8, 2022
936afa4
Update .gitreview for stable/yoga
openstackadmin Mar 8, 2022
591b4ac
Update TOX_CONSTRAINTS_FILE for stable/yoga
openstackadmin Mar 8, 2022
e61d4cf
Switch to stestr
Jul 11, 2018
51aed25
Fix references to start and end dates
priteau Mar 10, 2022
8e0e6c3
Synchronise tox.ini with changes from other branches
priteau Aug 25, 2022
590730a
[CI] Move queue setting to project level
priteau May 20, 2022
7f02440
WIP: Record placement resources and traits for hosts
JohnGarbutt Jun 30, 2023
1c5b01b
WIP: Option to use flavor_id to request reservation
JohnGarbutt Jun 29, 2023
1e28e30
HACK: ignore when servers existing on host
JohnGarbutt Jul 6, 2023
d98667f
Hack: remove host with instances
JohnGarbutt Jul 6, 2023
2524dca
Fix up reference to ComputeHostResourceInventory
JohnGarbutt Jul 6, 2023
b20e866
Attempt adding alembic migration
JohnGarbutt Jul 6, 2023
a138dc1
Fill in missing resource details
JohnGarbutt Jul 6, 2023
eb22432
Fetch traits for compute node
JohnGarbutt Jul 6, 2023
8601aec
Add traits into db when adding a host
JohnGarbutt Jul 6, 2023
3e1230a
Add missing tables
JohnGarbutt Jul 6, 2023
cae9021
Remote dead pci_alias code
JohnGarbutt Jul 6, 2023
37febfa
Comment out missing keys for now
JohnGarbutt Jul 6, 2023
57f6955
hack to get flavor_id
JohnGarbutt Jul 6, 2023
1c88acf
Fix up get flavor_id hack
JohnGarbutt Jul 6, 2023
b4b5cb4
Fix client name to flavors
JohnGarbutt Jul 6, 2023
5efbdda
HAck to get extra specs
JohnGarbutt Jul 6, 2023
ba57e8d
Start to use extra specs
JohnGarbutt Jul 6, 2023
bbce89f
Add missing _create_resources param
JohnGarbutt Jul 6, 2023
a250bdd
HACK: stop creating server group!
JohnGarbutt Jul 6, 2023
5fa95e3
Hack up resource_inventory to parse empty string
JohnGarbutt Jul 6, 2023
ddeb67a
Fix server_group workaround
JohnGarbutt Jul 7, 2023
581693b
Hack to let flavor get created
JohnGarbutt Jul 7, 2023
9fa7a15
Make affinity optional
JohnGarbutt Jul 7, 2023
84794d6
Attempt to populate values from the flavor
JohnGarbutt Jul 10, 2023
57143e0
Fix _populate_values_with_flavor_info signature
JohnGarbutt Jul 10, 2023
5fdb916
Try to filter hosts using new resources
JohnGarbutt Jul 10, 2023
0d0d6ea
Attempt in place edit of values
JohnGarbutt Jul 10, 2023
17f656a
Hack in exception to test theory
JohnGarbutt Jul 10, 2023
1a7222e
Try to add flavor_info during allocation_candidates
JohnGarbutt Jul 10, 2023
826534f
Fix typo in _populate_values_with_flavor_info
JohnGarbutt Jul 10, 2023
3f8127a
Remove exception hack
JohnGarbutt Jul 10, 2023
7961d5b
Add extra info into instance reservation db
JohnGarbutt Jul 10, 2023
4efc30a
Copy across extra specs to flavor
JohnGarbutt Jul 10, 2023
12a97e2
Add debug logs for candiate hosts
JohnGarbutt Jul 11, 2023
0b635f1
Hack to skip CPUs for slots
JohnGarbutt Jul 11, 2023
f20dfd9
Debug to check we get resource inventory
JohnGarbutt Jul 11, 2023
404c019
Rewrite get_hosts_list to use resource_inventory
JohnGarbutt Jul 11, 2023
aba720f
Fix up resource_usage_by_event
JohnGarbutt Jul 11, 2023
a9063ff
Improve debug for get_hosts_list
JohnGarbutt Jul 11, 2023
9dd840a
Get more debug from max_usages
JohnGarbutt Jul 11, 2023
7b16e06
Add more debug
JohnGarbutt Jul 11, 2023
687e32e
Don't error if not requested anything
JohnGarbutt Jul 11, 2023
59762b1
Skip zero vpus better
JohnGarbutt Jul 11, 2023
3991f92
Move instance reservation to use placement aggregates
JohnGarbutt Jul 14, 2023
dc5938a
Move create_computehost to use placement aggregates
JohnGarbutt Jul 14, 2023
9d22ba7
Fix getting aggregate uuid
JohnGarbutt Jul 14, 2023
d7d5950
Add missing arg for _put_rp_aggregates
JohnGarbutt Jul 14, 2023
4e8aa5a
Try to fix up remove_host
JohnGarbutt Jul 14, 2023
8be74a7
Add info on host filters
JohnGarbutt Jul 14, 2023
44bdbee
Get more info
JohnGarbutt Jul 14, 2023
3bc0254
Fixup finding extra spec resource requests
JohnGarbutt Jul 14, 2023
9368740
Fix up the value of extra resources
JohnGarbutt Jul 14, 2023
51582a0
Nope, try put custom resources in the inventory
JohnGarbutt Jul 14, 2023
ac29143
Fix up resource class filtering
JohnGarbutt Jul 14, 2023
4411e62
Fix up use of add/remove compute host
JohnGarbutt Jul 14, 2023
553240d
Fix creating lease as non-admin user
JohnGarbutt Jul 14, 2023
ced262b
Fix up nova client admin creds
JohnGarbutt Jul 14, 2023
5a2efa2
Merge remote-tracking branch 'origin/master' into flavor-based-reserv…
JohnGarbutt Apr 29, 2024
ce25e53
Revert local tox.ini changes
JohnGarbutt Apr 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitreview
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@
host=review.opendev.org
port=29418
project=openstack/blazar.git
defaultbranch=stable/yoga
22 changes: 22 additions & 0 deletions blazar/db/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -378,6 +378,28 @@ def host_update(host_id, values):
IMPL.host_update(host_id, values)


# ComputeHostResourceInventory

def host_custom_resource_create(values):
"""Create a Host CustomResource from the values."""
return IMPL.host_custom_resource_create(values)


def host_custom_resource_get_all_per_host(host_id):
"""Return all custom resources belonging to a specific Compute host."""
return IMPL.host_custom_resource_get_all_per_host(host_id)


# ComputeHostTrait

def host_trait_create(values):
return IMPL.host_trait_create(values)


def host_trait_get_all_per_host(host_id):
return IMPL.host_trait_get_all_per_host(host_id)


# ComputeHostExtraCapabilities

def host_extra_capability_create(values):
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Copyright 2023 OpenStack Foundation.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""test

Revision ID: 2ae105568b6d
Revises: e20cbce0504c
Create Date: 2023-07-06 15:46:28.036821

"""

# revision identifiers, used by Alembic.
revision = '2ae105568b6d'
down_revision = 'e20cbce0504c'

from alembic import op
import sqlalchemy as sa


def upgrade():
# ### commands auto generated by Alembic - please adjust! ###
op.create_table('computehost_resource_inventory',
sa.Column('created_at', sa.DateTime(), nullable=True),
sa.Column('updated_at', sa.DateTime(), nullable=True),
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('computehost_id', sa.String(length=36), nullable=True),
sa.Column('resource_class', sa.String(length=255), nullable=False),
sa.Column('allocation_ratio', sa.Float(), nullable=False),
sa.Column('total', sa.Integer(), nullable=False),
sa.Column('reserved', sa.Integer(), nullable=False),
sa.Column('max_unit', sa.Integer(), nullable=False),
sa.Column('min_unit', sa.Integer(), nullable=False),
sa.ForeignKeyConstraint(['computehost_id'], ['computehosts.id'], ),
sa.PrimaryKeyConstraint('id')
)
op.create_table('computehost_trait',
sa.Column('created_at', sa.DateTime(), nullable=True),
sa.Column('updated_at', sa.DateTime(), nullable=True),
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('computehost_id', sa.String(length=36), nullable=True),
sa.Column('trait', sa.String(length=255), nullable=False),
sa.ForeignKeyConstraint(['computehost_id'], ['computehosts.id'], ),
sa.PrimaryKeyConstraint('id')
)
# ### end Alembic commands ###


def downgrade():
# ### commands auto generated by Alembic - please adjust! ###
op.drop_table('computehost_trait')
op.drop_table('computehost_resource_inventory')
# ### end Alembic commands ###
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Copyright 2023 OpenStack Foundation.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""add resource inventory

Revision ID: e20cbce0504c
Revises: 02e2f2186d98
Create Date: 2023-07-06 14:45:39.036229

"""

# revision identifiers, used by Alembic.
revision = 'e20cbce0504c'
down_revision = '02e2f2186d98'

from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects import mysql


def upgrade():
# ### commands auto generated by Alembic - please adjust! ###
op.add_column('instance_reservations', sa.Column('resource_inventory', sa.Text().with_variant(mysql.MEDIUMTEXT(), 'mysql'), nullable=True))
op.add_column('instance_reservations', sa.Column('resource_traits', sa.Text().with_variant(mysql.MEDIUMTEXT(), 'mysql'), nullable=True))
op.add_column('instance_reservations', sa.Column('source_flavor', sa.Text().with_variant(mysql.MEDIUMTEXT(), 'mysql'), nullable=True))
op.alter_column('instance_reservations', 'affinity',
existing_type=mysql.TINYINT(display_width=1),
nullable=True)
# ### end Alembic commands ###


def downgrade():
# ### commands auto generated by Alembic - please adjust! ###
op.alter_column('instance_reservations', 'affinity',
existing_type=mysql.TINYINT(display_width=1),
nullable=False)
op.drop_column('instance_reservations', 'source_flavor_id')
op.drop_column('instance_reservations', 'resource_traits')
op.drop_column('instance_reservations', 'resource_inventory')
# ### end Alembic commands ###
63 changes: 63 additions & 0 deletions blazar/db/sqlalchemy/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -746,6 +746,69 @@ def host_destroy(host_id):
session.delete(host)


# ComputeHostResourceInventory

def host_custom_resource_create(values):
values = values.copy()

custom_resource = models.ComputeHostResourceInventory()
custom_resource.update(values)

session = get_session()
with session.begin():
try:
custom_resource.save(session=session)
except common_db_exc.DBDuplicateEntry as e:
# raise exception about duplicated columns (e.columns)
raise db_exc.BlazarDBDuplicateEntry(
model=custom_resource.__class__.__name__,
columns=e.columns)

return None


def _host_custom_resource_get_all_per_host(session, host_id):
query = model_query(models.ComputeHostResourceInventory, session)
LOG.debug(query)
return query.filter_by(computehost_id=host_id)


def host_custom_resource_get_all_per_host(host_id):
return _host_custom_resource_get_all_per_host(get_session(),
host_id).all()


# ComputeHostTrait

def host_trait_create(values):
values = values.copy()

custom_resource = models.ComputeHostTrait()
custom_resource.update(values)

session = get_session()
with session.begin():
try:
custom_resource.save(session=session)
except common_db_exc.DBDuplicateEntry as e:
# raise exception about duplicated columns (e.columns)
raise db_exc.BlazarDBDuplicateEntry(
model=custom_resource.__class__.__name__,
columns=e.columns)

return None


def _host_trait_get_all_per_host(session, host_id):
query = model_query(models.ComputeHostTrait, session)
LOG.debug(query)
return query.filter_by(computehost_id=host_id)


def host_trait_get_all_per_host(host_id):
return _host_trait_get_all_per_host(get_session(),
host_id).all()

# ComputeHostExtraCapability

def _host_resource_property_query(session):
Expand Down
36 changes: 36 additions & 0 deletions blazar/db/sqlalchemy/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -206,6 +206,9 @@ class InstanceReservations(mb.BlazarBase):
amount = sa.Column(sa.Integer, nullable=False)
affinity = sa.Column(sa.Boolean, nullable=True)
resource_properties = sa.Column(MediumText(), nullable=True)
resource_inventory = sa.Column(MediumText(), nullable=True)
resource_traits = sa.Column(MediumText(), nullable=True)
source_flavor = sa.Column(MediumText(), nullable=True)
flavor_id = sa.Column(sa.String(36), nullable=True)
aggregate_id = sa.Column(sa.Integer, nullable=True)
server_group_id = sa.Column(sa.String(36), nullable=True)
Expand Down Expand Up @@ -253,11 +256,44 @@ class ComputeHost(mb.BlazarBase):
cascade="all,delete",
backref='computehost',
lazy='joined')
computehost_resource_inventory = relationship(
'ComputeHostResourceInventory', cascade="all,delete",
backref='computehost', lazy='joined')
computehost_traits = relationship(
'ComputeHostTrait', cascade="all,delete",
backref='computehost', lazy='joined')

def to_dict(self):
return super(ComputeHost, self).to_dict()


class ComputeHostResourceInventory(mb.BlazarBase):
__tablename__ = 'computehost_resource_inventory'

id = _id_column()
computehost_id = sa.Column(sa.String(36), sa.ForeignKey('computehosts.id'))
resource_class = sa.Column(sa.String(255), nullable=False)
allocation_ratio = sa.Column(sa.Float, nullable=False)
total = sa.Column(sa.Integer, nullable=False)
reserved = sa.Column(sa.Integer, nullable=False)
max_unit = sa.Column(sa.Integer, nullable=False)
min_unit = sa.Column(sa.Integer, nullable=False)

def to_dict(self):
return super(ComputeHostResourceInventory, self).to_dict()


class ComputeHostTrait(mb.BlazarBase):
__tablename__ = 'computehost_trait'

id = _id_column()
computehost_id = sa.Column(sa.String(36), sa.ForeignKey('computehosts.id'))
trait = sa.Column(sa.String(255), nullable=False)

def to_dict(self):
return super(ComputeHostTrait, self).to_dict()


class ComputeHostExtraCapability(mb.BlazarBase):
"""Description

Expand Down
4 changes: 4 additions & 0 deletions blazar/manager/exceptions.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,10 @@ class HostNotFound(exceptions.NotFound):
msg_fmt = _("Host '%(host)s' not found!")


class ResourceProviderNotFound(exceptions.NotFound):
msg_fmt = _("Resource provider for host '%(host)s' not found!")


class InvalidHost(exceptions.NotAuthorized):
msg_fmt = _("Invalid values for host %(host)s")

Expand Down
Loading