Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CSS-4744 - Merge main forward into feature-rebac #984

Merged
merged 51 commits into from
Jul 12, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
543c5bd
Minor fix for the jimmctl model-status command.
alesstimec Nov 11, 2022
263d447
Local dev env (#867)
ale8k Dec 16, 2022
bc3d053
Merge branch 'main' into cmd-model-status-fix-01
alesstimec Jan 16, 2023
0b819ad
Merge pull request #862 from alesstimec/cmd-model-status-fix-01
alesstimec Jan 19, 2023
b545d24
Refactor the add model method.
alesstimec Feb 17, 2023
95a6add
Merge branch 'main' into refactor-jimm-add-model
alesstimec Feb 17, 2023
714ba39
Merge pull request #903 from alesstimec/refactor-jimm-add-model
alesstimec Mar 9, 2023
8888bab
Removed push from ci workflow.
alesstimec Mar 9, 2023
a7e48c6
Merge pull request #908 from alesstimec/workflows-ci-remove-push
alesstimec Mar 9, 2023
df9f601
check and remove jimm snap during upgrades
Mar 15, 2023
32097b1
Merge pull request #910 from tcuthbert/fix_snap_upgrade
alesstimec Mar 15, 2023
b2111b5
Update to Juju 2.9.42 and some compatibility changes.
alesstimec Mar 20, 2023
65473ed
Merge pull request #914 from alesstimec/update-to-juju-2.9.42
alesstimec Mar 21, 2023
bccb20b
Fix add-controller (#923)
ale8k Mar 31, 2023
53fb047
Adds Sphinx documentation according to the Diataxis framework.
alesstimec Apr 13, 2023
0e03b29
Merge branch 'main' into sphinx-documentation
alesstimec Apr 14, 2023
445db94
Merge pull request #927 from alesstimec/sphinx-documentation
alesstimec Apr 14, 2023
735537b
Tutorial on group and access management.
alesstimec Apr 17, 2023
805eee4
Merge pull request #929 from alesstimec/docs-group-management
alesstimec Apr 19, 2023
9efc548
Fix for compose and updated instructions (#943)
kian99 May 17, 2023
7fc363a
Updated to juju 3.1.4
alesstimec Jun 19, 2023
f60f4a5
Merge pull request #959 from alesstimec/update-juju-3.1.4
alesstimec Jun 20, 2023
4c544b6
Switches machine charm to data platform libs.
alesstimec Jun 21, 2023
df43a22
Merge pull request #962 from alesstimec/machine-charm-data-platform-l…
alesstimec Jun 21, 2023
65d37d4
backport ci/cd and supporting files (#963)
kian99 Jun 22, 2023
5a2af0d
Css 4587 backport generic fixes (#965)
kian99 Jun 26, 2023
46c42ea
Css 4116/backport crossmodel queries (#964)
ale8k Jun 26, 2023
3677f62
Fix for vault image (#967)
kian99 Jun 27, 2023
093b9ff
Removed documentation folder (#966)
kian99 Jun 27, 2023
cec35fe
Backport working workflows from feature-rebac (#975)
kian99 Jun 29, 2023
ba68eaa
Backported workflow fixes from feature-rebac (#980)
kian99 Jun 30, 2023
6deb07a
Backports charm changes from feature-rebac into main.
alesstimec Jun 30, 2023
ffb8099
Minor tweaks
kian99 Jul 3, 2023
027864c
Fix for the dashboard relation.
alesstimec Jul 3, 2023
ba3065d
Merge pull request #4 from kian99/ales-backport-2
alesstimec Jul 3, 2023
a34498c
Fix for the dashboard relation.
alesstimec Jul 3, 2023
b8373d8
Merge pull request #982 from alesstimec/backport-charm-changes-from-f…
alesstimec Jul 3, 2023
ea0af4f
Merged main and removed conflicts
kian99 Jul 3, 2023
1700cee
Added build tag to include version info in Docker build (#983)
kian99 Jul 4, 2023
0346934
k8s charm: Fix for the vault relation.
alesstimec Jul 4, 2023
36c3016
Merge pull request #985 from alesstimec/k8s-charm-vault-relation-fix
alesstimec Jul 4, 2023
cbd413a
Merge branch 'main' into merge-main-forward
kian99 Jul 4, 2023
c4791fb
Updated pydantic requirement
kian99 Jul 4, 2023
304ff29
Pin pydantic version (#987)
kian99 Jul 5, 2023
afd1613
General fixes after QA
alesstimec Jul 7, 2023
73c3379
Merge pull request #988 from alesstimec/general-fixes-after-qa-01
alesstimec Jul 7, 2023
87f6d69
JIMM k8s charm fixes (#989)
kian99 Jul 7, 2023
842c07a
Fix for dashboard file (#993)
kian99 Jul 10, 2023
0883ba6
Merged main and fixed conflicts
kian99 Jul 10, 2023
2374232
Merged feature-rebac
kian99 Jul 10, 2023
3395a75
Merge branch 'feature-rebac' into merge-main-forward
kian99 Jul 12, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .air.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ tmp_dir = "tmp"

[log]
time = false
main_only = true

[misc]
clean_on_exit = false
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ COPY . .
RUN echo "${GIT_COMMIT}" | tee ./version/commit.txt
RUN echo "${VERSION}" | tee ./version/version.txt
RUN --mount=type=ssh source /root/.gvm/scripts/gvm && go mod vendor
RUN --mount=type=ssh source /root/.gvm/scripts/gvm && go build -o jimmsrv -race -v -a -mod vendor ./cmd/jimmsrv
RUN --mount=type=ssh source /root/.gvm/scripts/gvm && go build -tags version -o jimmsrv -v -a -mod vendor ./cmd/jimmsrv

# Define a smaller single process image for deployment
FROM ${DOCKER_REGISTRY}ubuntu:20.04 AS deploy-env
Expand Down
35 changes: 18 additions & 17 deletions charms/jimm-k8s/src/charm.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@
"OPENFGA_PORT": "missing openfga relation",
}

JIMM_SERVICE_NAME = "jimm"
DATABASE_NAME = "jimm"
OPENFGA_STORE_NAME = "jimm"
LOG_FILE = "/var/log/jimm"
Expand Down Expand Up @@ -289,7 +290,7 @@ def _update_workload(self, event):
"summary": "jimm layer",
"description": "pebble config layer for jimm",
"services": {
"jimm": {
JIMM_SERVICE_NAME: {
"override": "merge",
"summary": "JAAS Intelligent Model Manager",
"command": "/root/jimmsrv",
Expand All @@ -307,10 +308,10 @@ def _update_workload(self, event):
}
container.add_layer("jimm", pebble_layer, combine=True)
if self._ready():
if container.get_service("jimm").is_running():
if container.get_service(JIMM_SERVICE_NAME).is_running():
container.replan()
else:
container.start("jimm")
container.start(JIMM_SERVICE_NAME)
self.unit.status = ActiveStatus("running")
else:
logger.info("workload container not ready - defering")
Expand All @@ -321,9 +322,9 @@ def _update_workload(self, event):
if dashboard_relation and self.unit.is_leader():
dashboard_relation.data[self.app].update(
{
"controller-url": dns_name,
"identity-provider-url": self.config.get("candid-url"),
"is-juju": str(False),
"controller_url": "wss://{}".format(dns_name),
"identity_provider_url": self.config.get("candid-url"),
"is_juju": str(False),
}
)

Expand All @@ -336,7 +337,7 @@ def _on_stop(self, _):
try:
container = self.unit.get_container(WORKLOAD_CONTAINER)
if container.can_connect():
container.stop("jimm")
container.stop(JIMM_SERVICE_NAME)
except Exception as e:
logger.info("failed to stop the jimm service: {}".format(e))
self._ready()
Expand All @@ -353,9 +354,9 @@ def _on_dashboard_relation_joined(self, event: RelationJoinedEvent):

event.relation.data[self.app].update(
{
"controller-url": dns_name,
"identity-provider-url": self.config["candid-url"],
"is-juju": str(False),
"controller_url": "wss://{}".format(dns_name),
"identity_provider_url": self.config["candid-url"],
"is_juju": str(False),
}
)

Expand Down Expand Up @@ -391,12 +392,12 @@ def _ready(self):

if container.can_connect():
plan = container.get_plan()
if plan.services.get("jimm") is None:
if plan.services.get(JIMM_SERVICE_NAME) is None:
logger.error("waiting for service")
self.unit.status = WaitingStatus("waiting for service")
return False

env_vars = plan.services.get("jimm").environment
env_vars = plan.services.get(JIMM_SERVICE_NAME).environment

for setting, message in REQUIRED_SETTINGS.items():
if not env_vars.get(setting, ""):
Expand All @@ -405,7 +406,7 @@ def _ready(self):
)
return False

if container.get_service("jimm").is_running():
if container.get_service(JIMM_SERVICE_NAME).is_running():
self.unit.status = ActiveStatus("running")
else:
self.unit.status = WaitingStatus("stopped")
Expand Down Expand Up @@ -608,7 +609,7 @@ def _get_dns_name(self, event):
if not self._state.is_ready():
event.defer()
logger.warning("State is not ready")
return
return None

default_dns_name = "{}.{}-endpoints.{}.svc.cluster.local".format(
self.unit.name.replace("/", "-"),
Expand Down Expand Up @@ -654,15 +655,15 @@ def _on_certificate_expiring(self, event: CertificateExpiringEvent) -> None:

new_csr = generate_csr(
private_key=private_key.encode(),
subject=self.dns_name,
subject=dns_name,
)
self.certificates.request_certificate_renewal(
old_certificate_signing_request=old_csr,
new_certificate_signing_request=new_csr,
)
self._state.csr = new_csr.decode()

self._update_workload()
self._update_workload(event)

@requires_state_setter
def _on_certificate_revoked(self, event: CertificateRevokedEvent) -> None:
Expand All @@ -687,7 +688,7 @@ def _on_certificate_revoked(self, event: CertificateRevokedEvent) -> None:
del self._state.chain

self.unit.status = WaitingStatus("Waiting for new certificate")
self._update_workload()
self._update_workload(event)

@requires_state_setter
def _on_ingress_ready(self, event: IngressPerAppReadyEvent):
Expand Down
8 changes: 4 additions & 4 deletions charms/jimm-k8s/tests/unit/test_charm.py
Original file line number Diff line number Diff line change
Expand Up @@ -269,11 +269,11 @@ def test_dashboard_relation_joined(self):

self.assertTrue(data)
self.assertEqual(
data["controller-url"],
"juju-jimm-k8s-0.juju-jimm-k8s-endpoints.None.svc.cluster.local",
data["controller_url"],
"wss://juju-jimm-k8s-0.juju-jimm-k8s-endpoints.None.svc.cluster.local",
)
self.assertEqual(data["identity-provider-url"], "https://candid.example.com")
self.assertEqual(data["is-juju"], "False")
self.assertEqual(data["identity_provider_url"], "https://candid.example.com")
self.assertEqual(data["is_juju"], "False")

@patch("src.charm.JimmOperatorCharm._get_network_address")
@patch("socket.gethostname")
Expand Down
1 change: 1 addition & 0 deletions charms/jimm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ juju deploy openfga
juju add-relation juju-jimm:openfga postgresql:openfga
```


## Developing

Create and activate a virtualenv with the development requirements:
Expand Down
8 changes: 5 additions & 3 deletions charms/jimm/charmcraft.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,12 @@ parts:
- ./templates
- ./files
- README.md
charm-python-packages:
- setuptools
charm-python-packages: [setuptools]
charm-binary-python-packages:
- Jinja2 >= 2.11.3
- markupsafe >= 2.0.1
- pydantic == 1.10.*
- cosl
- pydantic==1.10
bases:
# Ensure run-on is the same or newer than build-on
# since jimm-server is a Go binary using CGO dependencies
Expand Down
2 changes: 1 addition & 1 deletion charms/jimm/lib/charms/grafana_agent/v0/cos_agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,7 @@ class _MetricsEndpointDict(TypedDict):
LIBAPI = 0
LIBPATCH = 3

PYDEPS = ["cosl", "pydantic==1.10"]
PYDEPS = ["cosl", "pydantic == 1.10.*"]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I should be editing this file manually? But we're waiting on this PR to fix it canonical/grafana-agent-k8s-operator#215

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think manual is fine for now, I suspect if you fetch lib again it'll overwrite it


DEFAULT_RELATION_NAME = "cos-agent"
DEFAULT_PEER_RELATION_NAME = "peers"
Expand Down
2 changes: 1 addition & 1 deletion charms/jimm/metadata.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ issues: https://github.com/canonical/jimm/issues

description: |
JIMM is a juju controller, used in conjunction with the JaaS dashboard to provide a seamless way
to manage models, regardless of where their controllers reside or what cloud they may be running on.
to manage models, regardless of where their controllers reside or what cloud they may be running on.

provides:
website:
Expand Down
2 changes: 1 addition & 1 deletion charms/jimm/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,5 @@ Jinja2 >= 2.11.3
ops >= 2.0.0
charmhelpers >= 0.20.22
hvac >= 0.11.0
pydantic == 1.10
pydantic == 1.10.*
cosl
9 changes: 4 additions & 5 deletions charms/jimm/src/charm.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,6 @@ def _on_config_changed(self, _):

with open(self._env_filename(), "wt") as f:
f.write(self._render_template("jimm.env", **args))

if self._ready():
self.restart()
self._on_update_status(None)
Expand All @@ -151,7 +150,7 @@ def _on_config_changed(self, _):
if dashboard_relation:
dashboard_relation.data[self.app].update(
{
"controller-url": self.config["dns-name"],
"controller-url": "wss://{}".format(self.config["dns-name"]),
"identity-provider-url": self.config["candid-url"],
"is-juju": str(False),
}
Expand Down Expand Up @@ -399,9 +398,9 @@ def _snap(self, *args):
def _on_dashboard_relation_joined(self, event):
event.relation.data[self.app].update(
{
"controller-url": self.config["dns-name"],
"identity-provider-url": self.config["candid-url"],
"is-juju": str(False),
"controller_url": "wss://{}".format(self.config["dns-name"]),
"identity_provider_url": self.config["candid-url"],
"is_juju": str(False),
}
)

Expand Down
8 changes: 4 additions & 4 deletions charms/jimm/tests/test_charm.py
Original file line number Diff line number Diff line change
Expand Up @@ -510,7 +510,7 @@ def test_dashboard_relation_joined(self):
harness.charm._agent_filename = tmp.name
harness.update_config(
{
"dns-name": "https://jimm.example.com",
"dns-name": "jimm.example.com",
"candid-agent-username": "username@candid",
"candid-agent-private-key": "agent-private-key",
"candid-agent-public-key": "agent-public-key",
Expand All @@ -526,9 +526,9 @@ def test_dashboard_relation_joined(self):
harness.add_relation_unit(id, "juju-dashboard/0")
data = harness.get_relation_data(id, "juju-jimm")
self.assertTrue(data)
self.assertEqual(data["controller-url"], "https://jimm.example.com")
self.assertEqual(data["identity-provider-url"], "https://candid.example.com")
self.assertEqual(data["is-juju"], "False")
self.assertEqual(data["controller_url"], "wss://jimm.example.com")
self.assertEqual(data["identity_provider_url"], "https://candid.example.com")
self.assertEqual(data["is_juju"], "False")

def test_openfga_relation_changed(self):
id = self.harness.add_relation("openfga", "openfga")
Expand Down
1 change: 0 additions & 1 deletion charms/jimm/tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ envlist = lint, unit
[vars]
src_path = {toxinidir}/src/
tst_path = {toxinidir}/tests/
;lib_path = {toxinidir}/lib/charms/operator_name_with_underscores
all_path = {[vars]src_path} {[vars]tst_path}

[testenv]
Expand Down
2 changes: 1 addition & 1 deletion cmd/jimmctl/cmd/modelstatus.go
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ func (c *modelStatusCommand) SetFlags(f *gnuflag.FlagSet) {

// Init implements the cmd.Command interface.
func (c *modelStatusCommand) Init(args []string) error {
if len(args) < 0 {
if len(args) < 1 {
return errors.E("missing model uuid")
}
c.modelUUID, args = args[0], args[1:]
Expand Down
5 changes: 5 additions & 0 deletions cmd/jimmsrv/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import (
"go.uber.org/zap"

"github.com/CanonicalLtd/jimm"
"github.com/CanonicalLtd/jimm/version"
)

func main() {
Expand All @@ -32,6 +33,10 @@ func main() {

// start initialises the jimmsrv service.
func start(ctx context.Context, s *service.Service) error {
zapctx.Info(ctx, "jimm info",
zap.String("version", version.VersionInfo.Version),
zap.String("commit", version.VersionInfo.GitCommit),
)
if logLevel := os.Getenv("JIMM_LOG_LEVEL"); logLevel != "" {
if err := zapctx.LogLevel.UnmarshalText([]byte(logLevel)); err != nil {
zapctx.Error(ctx, "cannot set log level", zap.Error(err))
Expand Down
9 changes: 4 additions & 5 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ services:
ports:
- 17070:80
environment:
JIMM_LOG_LEVEL: "debug"
JIMM_UUID: "3217dbc9-8ea9-4381-9e97-01eab0b3f6bb"
JIMM_DSN: "postgresql://jimm:jimm@db/jimm"
CANDID_URL: "http://0.0.0.0:8081" # For external client redirects (in the case of compose and running outside)
Expand Down Expand Up @@ -78,9 +79,9 @@ services:
condition: service_healthy
labels:
traefik.enable: true
traefik.http.routers.httpd.rule: Host(`jimm.localhost`)
traefik.http.routers.httpd.entrypoints: websecure
traefik.http.routers.httpd.tls: true
traefik.http.routers.jimm.rule: Host(`jimm.localhost`)
traefik.http.routers.jimm.entrypoints: websecure
traefik.http.routers.jimm.tls: true

db:
image: postgres
Expand Down Expand Up @@ -134,8 +135,6 @@ services:
image: candid:latest
container_name: candid
entrypoint: "/candid.sh"
command: ""
# command: "/etc/candid/config.yaml && echo 'hi' && ls"
expose:
- 8081
ports:
Expand Down
4 changes: 2 additions & 2 deletions internal/dashboard/dashboard.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ const (
// handler will redirect to that URL. If the location is a path on the disk
// then the handler will serve the files from that path. Otherwise a
// NotFoundHandler will be returned.
func Handler(ctx context.Context, loc string) http.Handler {
func Handler(ctx context.Context, loc string, publicDNSname string) http.Handler {
mux := http.NewServeMux()

u, err := url.Parse(loc)
Expand Down Expand Up @@ -85,6 +85,7 @@ func Handler(ctx context.Context, loc string) http.Handler {
continue
}
var w bytes.Buffer
configParams["baseControllerURL"] = publicDNSname
if err := t.Execute(&w, configParams); err != nil {
zapctx.Error(ctx, "error executing config.js.go", zap.Error(err))
continue
Expand Down Expand Up @@ -118,7 +119,6 @@ func Handler(ctx context.Context, loc string) http.Handler {
)
continue
}

type guiArchiveVersion struct {
// Version holds the Juju GUI version number.
Version version.Number `json:"version"`
Expand Down
Loading