Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] rolling-upgrade of a marathon HA cluster with the leading-marathon last #40

Open
lhoss opened this issue Nov 25, 2016 · 0 comments

Comments

@lhoss
Copy link
Contributor

lhoss commented Nov 25, 2016

Motivation:

Do the upgrade with the minimum (1) leader fail-overs (vs worst-case: 'number of HA master/marathon nodes'), as recommended on:
https://mesosphere.github.io/marathon/docs/upgrade/index.html
http://stackoverflow.com/questions/33042343/is-it-possible-to-perform-a-zero-downtime-upgrade-or-reconfiguration-of-mesos

Current State

Currently I've a flexible serial attribute to provide the (optional) rolling upgrade logic, in a typical
playbook:

- hosts: marathon
  become: True
  roles:
    - {role: teralytics.marathon}
  environment: "{{proxy_env}}"
  tags: marathon
  serial: "{{ (rolling_upgrade|default(false)|bool)|ternary(1,0) }}"

Goal

run the upgrade in a specific order, so that the current leader (can be provided for ex. using a consul-DNS or mesos-dns url like 'leader.mesos' ) is deployed last.

I've some ideas, but before doing actual work, wanted to ask the ansible/devOps community for any existing solutions or recommendations.
I see 2 diff approaches:

  • (a) Splitting the playbook into 2 diff runs:
  • 1.run on the list of all selected-nodes, excluding the leader.mesos (could be done with some list mangling, including the resolution of dns entries to IPs to match the leader.mesos with the actual hostname (from the inventory)! )
  • 2.run on the remaining leader.mesos host
  • (b): using a dynamic inventory, resp. extra python logic that returns a single list of hosts in the right order

Ideas!?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant