Korg Minilogue XD adaptation - working Beta

[Andy2No]

I've got an adaptation working, reasonably well, for the Korg Minilogue XD (attached, below). I'm not sure what to do to get the patch numbers showing correctly when I import them all, into a blank database, but other than that, it seems to do what you'd expect.

I seemed to need to implement the optional 3rd function, convertToProgramDump(), mentioned in the adaptations guide, to get past the first 100 patches, when reading from the synth. I now get all 500, by claiming there are 4 banks of 128, which is 12 more than there actually are, but that doesn't seem to cause a problem - i just get 13 copies of patch 500.

Currently, it shows the patch numbers as 0-127, 0-127, 0-127, 0-127, instead of 1-500 (or 1-512), to agree with the display on the synth. I have implemented the other optional function, to return a patch number, but that doesn't seem to solve the problem:

def numberFromDump(message):
    progLSB = message[7]
    progMSB = message[8]
    return progLSB + progMSB * numberOfPatchesPerBank()

I probably should have added 1, but either way, that's not the number that's being shown in the boxes for each patch name.

Minilogue_XD.zip

EDIT: See below for newer, dated, versions.

[Andy2No]

Today's version treats the Minilogue XD as having 5 banks of 100 patches, which seems to be what's intended.

The patch dumps don't actually contain a bank number, and the MIDI Implementation document doesn't properly describe bank select... which I haven't yet confirmed works.

Patch numbers are now shown as 0-99, 0-99, ... 0-99, when importing all patches from the synth into a blank database. If I export them all as patch dumps, the sysex files contain the full patch number (1 to 500 as 0-499, as a two byte number with the 7 least significant bits in the first byte and the top two bits in the second), but if I re-import them, into a blank database, all patch numbers are shown as 00.

As far as I can tell, the numberFromDump() result isn't ever used... and maybe the function isn't ever called; I'm not sure.

I assume the patch number behaviour can't be changed, in the current version of KnobKraft Orm, but if it becomes possible in a later version, I'll make whatever changes are needed to the adaptation. For now, this may be as good as I'm going to get it.

Here is the zipped adaptation source. I've given this one a date and time stamp as a version number. I'll leave yesterday's undated one up for now, but this one supersedes it:

Minilogue_XD 2202-01-03_19-20.zip

[christofmuc]

@Andy2No Let me see - the convertToProgramDump() makes sense, because without it you do not have implemented the program dump capability, and the librarian falls back to use only edit buffer dumps.

if (pythonModuleHasFunction(kIsSingleProgramDump)
			&& pythonModuleHasFunction(kCreateProgramDumpRequest)
			&& pythonModuleHasFunction(kConvertToProgramDump)) {
			*outCapability = dynamic_cast<midikraft::ProgramDumpCabability *>(programDumpCapabilityImpl_.get());

So it is surprising but correct in that you need to implement it, and it is not used yet until KK 2.0 is finished containing full bank management (actually working on that now!).

I also understand that the minilogue documentation suggests you have patch numbers 0..499 in the 9 bits offered, but it doesn't work? To test, you can try to implement a single bank with 500 patches, and then splitting the patch number into msb/lsb as you intended.

With the nameFromDump() you might be on to something here... it really seems that there is a bug in the GenericAdaptation class that prevents the nameFromDump() being called (it is called for the native synths like K3, Rev2, OB-6). So the program place is always just counted by the import, which results in the numbers you saw with the 4 banks/5 banks experiment. Good news is that if it is in the patch, we can get the data fixed. I need to look more into that - can you provide me a sysex file with the 500 patches of the minilogue or point me to the factory presets so I can test it with some data?

[Andy2No]

The problems I was having with getting bulk imports from the synth seem to have been caused by doing it with edit buffers, before I added the functions to get patch dumps instead. There are a couple of mistakes in Korg's MIDI Implementation document that made it a bit confusing to sort out.

It turns out that sending Program Change from a keyboard only gives patches 1-100. The MIDI Implementation mentions Bank Select but says the value should be zero - I'm still not sure if that's a typo, or actually the case, and higher numbers can't be selected with Bank and Program Select. I don't currently have anything set up try that, but I'll have a look at it soon.

If I tried to get banks of 128, as edit buffers, it stalled at 100, then started up again at 129. If I tried banks of 100, the second bank was starting with patch 129, instead of patch 101... I might have just got something wrong, at that point, because I was so confused by it.

They document the patch number as 8 bits too, and it clearly needs 9.

Implementing patch dumps meant I could get all of them, but I'd been trying it as banks of 128, assuming the higher two bits were the bank number - which mostly worked out, but wasn't right. I've changed it back to 5 banks of 100, today, and that's now working.

Here's my last full dump of patches - using "Import patches from synth" then "Export into sysex files", using the default, which was patch dumps. I made some small edits to the Adaptation after that, but nothing that affects those. They all work when read back, and clicked on, to send them to the edit buffer, despite being stored in unpacked form, instead of the 8 bit packed form that Korg uses... I'm not entirely clear why that works :)

Minilogue_XD 2202-01-03_19-20.zip

Here's Korg's most recent MIDI Implementation document, version 1.01:

minilogue_xd__MIDIImp.txt

Korg uses their own library format, with their Sound Librarian software. I could attach my last save from that, but I don't know how much use it would be.

Taking sections from that MIDI Implementation document, an edit buffer dump starts like this - note the 0x40 before the packed data:

# (4) CURRENT PROGRAM DATA DUMP                                     R/T
    # +----------------+--------------------------------------------------+
    # |     Byte       |             Description                          |
    # +----------------+--------------------------------------------------+
    # | F0,42,3g,      | EXCLUSIVE HEADER                                 |
    # |    00,01,51    |                                                  |
    # | 0100 0000 (40) | CURRENT PROGRAM DATA DUMP              40H       |
    # | 0ddd dddd (dd) | Data                                             |
    # | 0ddd dddd (dd) |  :         Data Size         Conv. Size          |
    # | 0ddd dddd (dd) |  :      384Bytes (7bit) -> 336Bytes (8bit)       |
    # | 0ddd dddd (dd) |  :                                               |
    # | 1111 0111 (F7) | EOX                        (See NOTE 1, TABLE 2) |
    # +----------------+--------------------------------------------------+
    #
    #  Receive this message & data, save them to Edit Buffer and transmits Func=23 or Func=24 message.
    #  Receive Func=10 message, and transmits this message & data from Edit Buffer.
    #  When "Program Dump" is executed, transmit this message & data from Edit Buffer.

A program/patch dump starts like this - the 0x40 becomes 0x4c, followed by the two byte patch number:

# (5) PROGRAM DATA DUMP (1 PROG)                                    R/T
    # +----------------+--------------------------------------------------+
    # |     Byte       |             Description                          |
    # +----------------+--------------------------------------------------+
    # | F0,42,3g,      | EXCLUSIVE HEADER                                 |
    # |    00,01,51    |                                                  |
    # | 0100 1100 (4C) | PROGRAM DATA DUMP                      4CH       |
    # | 0ppp pppp (pp) | Program No.(LSB bit  6~0)                        |
    # | 0000 000p (PP) | Program No.(MSB bit    7)                        |
    # | 0ddd dddd (dd) | Data                                             |
    # | 0ddd dddd (dd) |  :         Data Size         Conv. Size          |
    # | 0ddd dddd (dd) |  :      384Bytes (7bit) -> 336Bytes (8bit)       |
    # | 0ddd dddd (dd) |  :                                               |
    # | 1111 0111 (F7) | EOX                        (See NOTE 1, TABLE 2) |
    # +----------------+--------------------------------------------------+
    #
    #  Receive this message & data, save them to Internal Memory and transmits Func=23 or Func=24 message.
    #  Receive Func=1C message, and transmits this message & data from Internal Memory.
    #  When "All Dump" is executed, transmit this message & data from Internal Memory.

There are 500 program locations, so these details seem to be wrong - the above only allows for 256, because it shows the MSB as one bit, and the LSB as 7 bits. The MSB actually has 2 bits.

The g in 0x3g, in both cases, is the global MIDI channel.

The patch name is near the beginning of the data block in both types of dunp (starting 4 bytes in, after unpacking to 7 bit sysex), but they use their own encoding, not ASCII. In the patch dumps exported from Orm, it's all unpacked, and the patch names are decoded to ASCII, so they can be seen easily.

I was expecting to see them as the packed sysex format, with Korg's name encoding system, but they work as they are, and it's nice that they're now more readable. I did have to get nameFromDump() right to get the names readable in Orm, though that was early in the process, before implementing the functions to support patch dumps instead of edit dumps... either way, it calls the unescapeSysex() to unpack the dump first, though.

I'm not really clear which functions are being called, and in what circumstances - I just kept hacking away at them until it worked :) I put some print() statements in to try to trace it, but I didn't really see what I expected.

[Andy2No]

I'll try one big bank of 500 next time. I've put it away, for today. I considered doing that, but thinking about how Program Change only works for patches 1-100 made me think they'd intended it to be five banks of 100... even though they don't seem to have mentioned that anywhere.

After importing all the exported patches back into an empty database (or at least, one with no Minilogue XD patches), I'm seeing all the patch numbers as 00, so it seems that even though they're dumped as proper patch dumps, with a patch number, they're being imported back as edit buffer dumps. Presumably, the convertToEditBuffer() funtion is being called, for each one, but I haven't checked, yet.

It would be nice if they could be read back in as patch dumps, preserving the number, but I guess that's a separate issue.

None of this is urgent, of course. I just thought I'd keep you informed with what I've been finding out, while trying to make this adaptation.

[christofmuc]

@Andy2No I tried to look at your patches, but I think you attached the adaptation again and not the patches. Generally, 5 banks of 100 should be what is needed and correct.

Program changes can address at most program number 127, as there are only 7 bits in the program change message (programs 0 to 127 give 128 addressable programs).

Bank select can be used to address the banks, and is the Midi Message Bn 00 bb with n being the MIDI channel and bb the bank (7 bits again).

Your implementation looks good, when you accept that the patch number within the Orm is a single integer running from 0 to 499, and you have to calculate the bank/program number in the Korg from it. It is documented as if it would be an 8 bit number split into two MIDI bytes, but we know 8 bit is not enough, so it could be just an extension to 9. Your code does it when you specify 1 bank of 500. If that doesn't work, maybe Korg has split up in 7 bits of program number but using only 100 values, and the other 2 bits are really the bank number?

If this doesn't work either, we need a different approach - What you can do is to try to implement the Patch Dump Request with a dual message - first send a bank select, then a program dump request within that bank, hoping to be able to address all patches:

def createProgramDumpRequest(channel, patchNo):
    bank = patchNo // numberOfPatchesPerBank()
    programNo = patchNo % numberOfPatchesPerBank()
    return [0xb0 | (channel & 0xf), 0x00, bank] + [0xf0] + [korg_id] + [0x30 + (channel & 0x0f), 0x00, 0x01, 0x51, 0x1c, programNo, 0x00, 0xf7]

[Andy2No]

Sorry. I hadn't noticed I'd attached the wrong file. It should have been this one:

2022-01-03.zip

Those definitely are saved in patch dump format, with a patch number. The most mysterious part is that the name seems to be in the decoded, human readable form too... but with some extra zeros...

Okay, I think I get that part. I blindly implemented translating Korg's definition of the character set, but it's actually just a subset of ASCII, with some characters not defined, so I'll revisit that. Also, the conversion between 7 bit and 8 bit sysex doesn't work the way I thought it did, so runs of characters can remain readable.

Looking at them with patch compare, after re-importing them, they're still in that format, so the patch number is still there - but it just gets shown as 00, in Orm.

I have established that the patch number is a 9 bit number, not split into a program number and a bank number - the first byte can go up to 127. I haven't yet tried the one big bank of 500 patches, but it seems like that should work.

[Andy2No]

I've made the version with one big bank of 500 patches, which works okay.

FWIW, the Minilogue XD appears to completely ignore Bank Select messages, so there doesn't seem to be any way to select more than the first 100 patches, from a MIDI controller. [EDIT: use bank select LSB, not MSB - accepts values 0-4]

I found I could get Orm to show them as 1-500 instead of 0-499 by implementing :

def friendlyProgramName(program):
    return str(program+1).zfill(3)      # By default, import from synth gives 0-499.
        # Add one and pad with leading zeros to make it match the display

If I export them all as sysex dumps, then reimport them, they're all numbered as 001, but I don't think that can be helped, in this version of Orm.

I've made a basic fingerprint comparison function, which just calculates md5 for the body of the patch, ignoring the header - but allowing for the different length headers of edit buffer dumps and program dumps. That's enough to stop it importing dozens/hundreds of "Init Program" patches, or duplicates of patches which are identical including the name, and it doesn't need to translate the data back to 8 bit values to do that, which looks like it's probably time consuming.

I'll add the rename function at some point, but for now, here's today's version:

2202-01-10_09-17 1 bank of 500, with fingerprinting.zip

[markusschloesser]

FWIW, the Minilogue XD appears to completely ignore Bank Select messages, so there doesn't seem to be any way to select more than the first 100 patches, from a MIDI controller

According to https://midi.user.camp/d/korg/minilogue-xd/ it should work, or have you tried this already? (identical to what's in the manual)

[Andy2No]

According to https://midi.user.camp/d/korg/minilogue-xd/ it should work, or have you tried this already? (identical to what's in the manual)

I tried sending b0 0n 00 c0 pp from Bome's SendSX, with various values. It responded to the program change (c0 pp) provided it was different to the last one, and in the range 0-99, by showing the patch name and switching to it. Sending different bank select values made no difference - it only ever selected a patch from the first 100...

Ah! It seems we're supposed to use bank select LSB, not the usual MSB. I missed that. I'll try it soon.

I looked at this bit, and this is what I've been trying, to see if it was another typo (it says the value must be 0):

|   Bn   | 00 (00) | vv    (vv)  | Bank Select (MSB) vv=0                   *5-1    |

I missed seeing this bit:

|   Bn   | 20 (32) | vv    (vv)  | Bank Select (LSB) vv=0~4                 *5-1    |

 *5-1 : This message is recognized when the "MIDI Rx Prog Chg" is set to "On".

[markusschloesser]

Bank select nearly always is very strange (fucked up), and there's way too many "standards". For example my k5000s needs 100 and 103 (iirc) as bank select for bank a and the other, being bank d, now who the fuck would come up with that?? 🤨 😜 😂

[Andy2No]

The Roland ROMplers are really odd for bank select too. This is for the little JV-1010:

Patch		Patch	Bank Select	Program
Group		Number	MSB	LSB	Number
USER		1–128	80	0	1–128
PRESET-A	1–128	81	0	1–128
PRESET-B	1–128	81	1	1–128
PRESET-C	1–128	81	2	1–128
PRESET-D	1–128	81	3	1–128
PRESET-E	1–128	81	4	1–128
Session		1–128	84	0	1–128
Session		129–255	84	1	1–127
WAVE-EXP	1–128	84	2	1–128
WAVE-EXP	129–256	84	3	1–128

Remembering all that is definitely more suited to software than a human. You have to wonder, why they felt the need to start at 80, and why they needed to use 14 bit numbers, to represent so few banks, even allowing for some grand scheme of future expansion, on the same theme.

[Andy2No]

I see there's a Minilogue XD adaptation now included in V1.17.0, but it's not the most recent version I made, which was "2202-01-10_09-17 1 bank of 500, with fingerprinting.zip", a couple of posts above this one:
https://github.com/christofmuc/KnobKraft-orm/files/7837792/2202-01-10_09-17.1.bank.of.500.with.fingerprinting.zip

I preferred treating it as one big bank of 500 patches, so it kept the patch numbers as they appear on the Minilogue XD, but I could see the argument for making it five banks of 100 to speed up transferring a range of patches over DIN MIDI instead of USB... I could make two different versions, but maybe that's going too far.

I probably ought to revisit it and add patch renaming too.

[christofmuc]

@Andy2No Oh, I must have missed that. Will update with the next release. Do we already have Minilogue test data in the test data adaptation subdirectory? I have a new test suite now that runs the adaptations during the build step when they provide test syx files.

[Andy2No]

@christofmuc

I don't see any in the test adaptation directory. I attached a full dump, from the Orm, in a post above. Will that do? It's "2022-01-03.zip":

https://github.com/christofmuc/KnobKraft-orm/files/7835472/2022-01-03.zip

Those are saved in patch dump format, with a patch number starting at byte 7, immediately after the 0x4c, which identifies it as a PROGRAM DATA DUMP.

The first couple of hundred (by patch number) are factory patches. The higher numbered ones are ones I made, a lot of which need custom oscillators loaded, to work as intended. The Minilogue XD doesn't store the custom oscillators in the patch, it only refers to them using a slot number. I don't suppose that matters much, other than that they won't play the same way, without the right custom oscillators loaded into the right slots.

[vincenttessier]

Hi everyone,
I'm trying to wrap my head around the Korg Minilogue XD program change, couldn't find much online except this thread!
Only problem is i'm not coding or anything, i'm just trying to send program change from the Roland MC 707 to the XD, and the MC 707 requires MSB & LSB.
Can anyone explain to me the mapping table from MSB + LSB + prgm change number to the XD 500 presets?

[Andy2No]

@vincenttessier it should be Bank Select MSB 0, LSB 0-4 then Program Change 0-99. Some software uses numbers starting from one, for Program Change (so 1-100, in this case) but actually still sends 0-99.

The Bank Select MSB doesn't actually need to be sent, because the XD just ignores it, AFAIK, but if you have to send one, use 0.

LSB 0 is for patches 1-100, LSB 1 is for 101-200, and so on. The only remaining confusion is whether you need to refer to patch 1 as 0, for example, which should be easy enough to work out.

It's normal for nothing to immediately happen when you send Bank Select (e.g. just LSB, or both). The change happens when you send Program Change. The MIDI standard (for MIDI in general, not this particular synth) suggests that Bank Select may get forgotten if there isn't a Program Change almost immediately after it, but I don't think that applies to the XD... or to a lot of other synths.

It's always okay to just send a Program Change, once you've successfully changed to a program in the bank you want. To change to a one in a different bank, the standard says you need to send all three pieces - MSB, LSB, then Program Change with preferably nothing else in between. In practice, you can mostly send them whenever you want, and in any order, but the change to a different patch still only happens after a Program Change.