Rollout mwoffliner 1.14 on farm.openzim.org

[benoit74]

See openzim/zimfarm#988

Below statement assumes there is no requested tasks ongoing, otherwise the table must be updated accordingly.

Check statement:

SELECT id, config::jsonb -> 'flags' ->> 'mwApiPath' as mwApiPath, config::jsonb -> 'flags' ->> 'mwActionApiPath' as mwActionApiPath
FROM schedule
WHERE config::jsonb -> 'image' ->> 'name' = 'ghcr.io/openzim/mwoffliner'
AND config::jsonb -> 'flags' ->> 'mwActionApiPath' is null
AND config::jsonb -> 'flags' ->> 'mwApiPath' is not null;

Update statement:

UPDATE schedule SET config = jsonb_set(config::jsonb, '{flags, mwActionApiPath}', config::jsonb -> 'flags' ->> 'mwApiPath') #- '{flags, mwApiPath}'
WHERE config::jsonb -> 'image' ->> 'name' = 'ghcr.io/openzim/mwoffliner'
AND config::jsonb -> 'flags' ->> 'mwActionApiPath' is null
AND config::jsonb -> 'flags' ->> 'mwApiPath' is not null;

[benoit74]

Enable/Disable mwoffliner currently enabled recipes

Temp table to keep data:

create table _maint_mwoffliner_1_14 (
  id uuid,
  schedulename character varying
);

Populate temp table:

INSERT INTO _maint_mwoffliner_1_14
SELECT id, name
FROM schedule
WHERE enabled = TRUE
AND config::jsonb -> 'image' ->> 'name' = 'ghcr.io/openzim/mwoffliner';

Check

SELECT id, enabled, config::jsonb -> 'image' ->> 'name' AS image_name, config::jsonb -> 'image' ->> 'tag' AS image_tag
FROM schedule
WHERE id in (SELECT id FROM _maint_mwoffliner_1_14)

Disable

UPDATE schedule
SET enabled = FALSE
WHERE id in (SELECT id FROM _maint_mwoffliner_1_14);

Cleanup temp table:

drop table _maint_mwoffliner_1_14;

[benoit74]

@kelson42 @audiodude all mwoffliner recipes in Zimfarm are now disabled.

We still have two mwoffliner tasks running: https://farm.openzim.org/pipeline/b4f58539-454b-4d6d-8392-e1504a7a382b and https://farm.openzim.org/pipeline/94167b91-8b89-41de-a351-6287bab02c49

Once finished, @rgaudin will purge S3 cache for mwoffliner

[rgaudin]

Ping me when I should

[kelson42]

@rgaudin If all mwoffliner recipes are suspended an task running, then pkease purge the S3 cache

[benoit74]

Bucket is now purged. It is possible to delete all files in a bucket at once from the UI by deleting the bucket itself.
What I did:

• rename old bucket
• create new empty bucket with old bucket settings
• delete old bucket (with force delete of files inside)

[benoit74]

DB has been updated to use new flag name.
Only things left are:

• merge Adapt mwoffliner configuration for 1.14 openzim/zimfarm#992
• tests obviously
• release and deploy 1.14

[rgaudin]

Good ; I imagine this feature (bucket removal with files in it) has been added since our last attempt. Very good to know.

[benoit74]

Zimfarm PR openzim/zimfarm#992 has been merged. Work is now in mwoffliner team hands.

@kelson42 @audiodude please inform us when you are ready to configure 1.14 on all recipes + reactivate all recipes.

[kelson42]

@benoit74 Thx, we will request your help to migrate recipes when we will be that far.