incremental backup - update of existing dumps (export/std)

[dassio]

can we do incremental backup? so I can backup daily and have the full history.
I am currently on slack free plan and the 90 days full history has not expired(still 81 days left)
let's say i start from now backup daily, after 81 days, I still have the full history to browser
and after that I will keep backup daily

[dassio]

I saw that we can do delta backup #193 , but is the zip file is merged into one, I don't want to browser each zip file seperately

[rusq]

Hey @dassio, I'll have to think about this. Updating existing dumps is tricky, as the parser for the export files will need to be implemented (currently - slackdump only writes, and doesn't read/understand the export files). Maybe some time in the future, after the initial incremental dump is implemented.

[rusq]

Possibly duplicate of #205

[dassio]

thanks for the reply, will try if I can chip in. been a while since last time touching golang code

[gagomes]

Just adding a vote to raise the importance of this issue. Thanks.

[raine]

My quick and dirty solution to get incremental history for a channel in a free slack:

1. Have a git repo to store history

2. Run slackdump -download -export . -export-type standard "$channel" to get export for the channel

   In this export format, each day gets its own JSON file:

   some_channel
   ├── 2022-08-01.json
   ├── 2022-08-02.json
   ├── 2022-08-03.json
   ├── 2022-08-04.json
   ├── 2022-08-05.json
   ...
   

3. As you run this for example daily, you can keep committing the new changes, but there's a gotcha. The older JSON files available in the current 3 month history window can get some weird changes. For example I've seen messages from the oldest files get removed entirely when running the export. For this reason, I figured it's best to stage and commit changes only for the last 7 days:

   for num in $(seq 0 7); do
       git add "**/$(date -j -v -"$num"d '+%Y-%m-%d'.json)" || true
   done
   git add ./**/attachments
   git add channels.json users.json
   
   # Rest of the changes can be disregarded
   git restore -- **/*.json

4. Now you can visualize the history with slack-export-viewer:

   slack-export-viewer -z .

5. I used launchd to automate daily backup.

[rusq]

@raine that is a very elegant solution, thank you for sharing!