NEORV32 on a Cologne Chip GateMate FPGA

[stnolting]

I have been wanting to experiment with the GateMate FPGAs from @colognechip for some time now. Fortunately, @OLIMEX offers a low-cost and open source dev board (GateMateA1-EVB) that I had to get right away! It costs only 50€ (plus taxes and shipping), offers some nice on-board peripherals and has many external interfaces.

Downloading and "installing" the GateMate toolchain is very easy. I am using the Linux version for synthesis (on Ubuntu-based WSL) and the Windows version for programming (on Win10). The provided examples worked out of the box! 🚀 However, it would be nice if the toolchain would be available also on GitHub.

The Setup

So I created a simple NEORV32 setup featuring a standard RISC-V CPU configuration, on-board memories and some peripherals like a UART, a timer and some basic IO. The synthesis flow uses the GHDL plugin for Yosys for processing the VHDL sources. You can find all the sources including a simple Makefile and some brief documentation in https://github.com/stnolting/neorv32-setups/tree/main/cologne_chip/GateMateA1-EVB.

This what the results looks like on my desk: the setup is operational and the bootloader is waiting to be fed with an executable.

(GIF made with imgflip.com)

Findings

Finally, here are some insights, pitfalls and open questions I discovered during my very first (and very botched) experiments. As always, comments and feedback are very welcome!

---

The toolchain seems to be compiled with some hardcoded relative paths. I had to copy the toolchain's bin folder right into the root directory of neorv32-setups. Otherwise, (Yosys+)GHDL cannot find the built-in IEEE libraries:

warning: ieee library directory '../../bin/yosys/share/ghdl/ieee/v93/' not found
error: cannot find "std" library
ERROR: vhdl import failed.

Or maybe this is just a bug in my Makefile?!

---

What about those "CC_MX8" multiplexer primitives? Why can't they be used (-nomx8 synthesis flag)?

---

If the toolchain is not provided with a pin mapping file (*.ccf), the implementation still runs smoothly without issuing an error. Had to learn this the hard way.

---

The toolchain generates a *.bit file. However, the actual bitstream is the *.cfg file.

---

For some processor configurations (nothing too specific) P&R terminated with an error like this:

FATAL ERROR: CPE_OUTMUX congestion at Component 2785 with 2780

Other configurations went through implementation without problems, but caused run-time errors like broken UART output or did not even run at all (for example, when decreasing the UART0 FIFO sizes from 64 to 16 entries). I am still trying to figure which specific configurations/combos are causing this issue (and why).

---

I had to enable the synthesis retiming option (--retime). Without this flag, Yosys keeps generating an invalid netlist that still contains a conditional assign like this:

assign _1563_[2] = \neorv32_top_inst.core_complex.neorv32_cpu_inst.neorv32_cpu_control_inst.4477.B [110] ? \neorv32_top_inst.core_complex.neorv32_cpu_inst.alu_cmp [1] : \neorv32_top_inst.core_complex.neorv32_cpu_inst.alu_cmp [0];

I seems like it always fails to map a certain ALU signal. Need to investigate further.

---

I copied the PLL instance from the provided blink example to get a higher clock speed for the processors. Unfortunately, the PLL does not seem to lock. I read somewhere that some Olimex boards suffer from an improper PLL power supply (too much ripple?).

---

According to the board's schematic the on-board RP2040 provides a "debug UART". I can successfully connect to that virtual COM port using my terminal program. It also seems to send data quite nicely (processor -> host computer). However, when sending large data (like executables) from the computer to the board most received bytes are broken. That's why I am using an external USB-UART bridge connected to the board's PMOD header.

---

Sometimes the FPGA does not "boot" after bitstream upload (although it actually should according to the documentation) or it behaves in a strange way (weird UART output, LED flashing at some crazy frequency). I need to press the "FPGA_RST1" button and upload the bitstream again. Maybe this is a reset issue of the processor or maybe I just misunderstood the documentation.

---

To be continued...

[tmeissner]

Nice. I've also done some experiments with the Gatemate FPGA architecture last year. I've got some designs running correctly, but never the neorv32 design I tried. Yosys always generated a mysterious $mux_1 cell so the gatemate p&r tool rejected the synthesized netlist. I never found the reason. Maybe Yosys can synthesize the current neorv32 codebase without this error, I have to try again soon.

[pu-cc]

We have already investigated this ourselves and suggest the following fix, which is not yet upstream in yosys.
pu-cc/yosys@fa0c5c1

This also solves the above-mentioned issue with invalid assign statement.

[stnolting]

Thanks for the hint!

Btw, is there any timetable for the next release of the Cologne Chip toolchain?

I have seen that you can easily download the toolchain from the CC homepage (no registration required) - and I really like that! 👍
I wonder if you could also put a release on GitHub (e.g. for easy CI-integration)? 🤔

[pu-cc]

What about those "CC_MX8" multiplexer primitives? Why can't they be used (-nomx8 synthesis flag)?

A CPE could be configured as MUX8 using it's 8 data inputs and CLK, SR and EN for select. We found out that CC_MX8 has had a rather negative impact on the implementation result so far.

Very brief background information: The MUX8 configuration bypasses the LUTs in the LUT-tree. This means that it is not possible to invert incoming signals at the CPE. In order to save resources in silicon for routing, signals may also arrive inverted at the CPE; this has so far been fixed "for free" using the LUTs. We would therefore have to plan additional resources for the inversion, which usually has a negative effect on the result. That's why we limit P&R to only map MUX2 and MUX4 for now. However, if there is a need for it, we can surely find and work on a solution.

[stnolting]

Thank you for the detailed explanation!

We found out that CC_MX8 has had a rather negative impact on the implementation result so far.

Will this primitive then be removed from future FPGAs? Or will the toolchain be better adapted?

[pu-cc]

If the toolchain is not provided with a pin mapping file (*.ccf), the implementation still runs smoothly without issuing an error. Had to learn this the hard way.

That's a good tip, thank you! The next P&R version will throw an error if the file is missing.

[pu-cc]

The toolchain generates a *.bit file. However, the actual bitstream is the *.cfg file.

Both files *.cfg.bit and *.cfg contain the same bitstream. The *.cfg variant is just easier to read. :)

[stnolting]

As far as I can see, openFPGAloader can only consume the *.cfg file. If that is correct, what is the actual purpose of the *.bit file?

[pu-cc]

FATAL ERROR: CPE_OUTMUX congestion at Component 2785 with 2780

This issue "actually" only occurs when block RAM collides with the connections to the IOs at the bottom or top edge of the chip. There are CPEs whose outputs can be routed either to the block RAM or the IOs. We can avoid it by repositioning the block RAMs or IOs. Do you have an example netlist with CCF for me so that i could take a closer look at it?

[stnolting]

Do you have an example netlist with CCF for me so that i could take a closer look at it?

Thanks for the offer! 👍

I will try to reconstruct the situation. Should I post the netlists here or should I send them to you directly by e-mail?

I have also seen that some (rare) configurations cannot be routed (aborting with an error). Would you also be interested in such scenarios?

[Mitilesh139]

Hi,

I am facing the issue - "CPE_OUTMUX" error.
Can anyone please suggest how to reposition block RAM or IOs?

Thanks in Advance,
Mitilesh

[pu-cc]

I copied the PLL instance from the provided blink example to get a higher clock speed for the processors. Unfortunately, the PLL does not seem to lock. I read somewhere that some Olimex boards suffer from an improper PLL power supply (too much ripple?).

Boards delivered after Feb 29th have an improved filter and should no longer suffer from an improper power supply:
https://github.com/OLIMEX/GateMateA1-EVB/blob/main/HARDWARE/GateMateA1-EVB-Rev.A/IMPORTANT-NOTE.txt

[stnolting]

Ah, there it is! 👍

Luckily, my board is not affected by this. I tried the PLL again and it works 🎉 - at least at 20 MHz. Higher frequencies don't work yet... I'll have to check the timing reports (which I haven't found yet). 😅

[chili-chips-ba]

@stnolting any performance and area metrics you can share on the NEORV32?!

The following two in particular:

• CPI
• LUT

We'd like to contrast NEORV32 to our eduBOS5.

BTW, GateMate issues uncovered by our openCologne project may also be of interest, such as this one...

[stnolting]

We have some (outdated) exemplary results for some specific processor configurations:

• https://stnolting.github.io/neorv32/#_fpga_implementation_results
• https://stnolting.github.io/neorv32/#_cpu_performance

However, I can try to generate hardware utilization results for a specific configuration and FPGA if you like.

Regarding CPI: what do you use as reference algorithm?

We'd like to contrast NEORV32 to our eduBOS5.

Very nice architecture (somehow similar to NEORV32 😅)! Is this an open-source project (yet)?

BTW, GateMate issues uncovered by our openCologne project may also be of interest, such as chili-chips-ba/openCologne#25...

Thanks for the link - I'll have a look!

[chili-chips-ba]

For apples-to-apples comparison, we'd like to get your utilization numbers for:

• Gowin TangNano20K i 9K
• ULX3S-12
• Basys3
• GateMate Olimex

@tarik-ibrahimovic will be glad to provide full detail on our CPI reference algorithm. In a nutshell, all but memory operations (Load and Store) complete in 2 cycles, and memory ops take 3 cycles. Here is more about it.

eduBOS5 is not in the open-source domain.

[tarik-ibrahimovic]

Our reference algorithm for CPI is Dhrystone, we're still to run CoreMark. Our GateMate porting of the eduBOS5 hasn't been a smooth sail either, and faced similar issues.

[TarikHamedovic]

The toolchain seems to be compiled with some hardcoded relative paths. I had to copy the toolchain's bin folder right into the root directory of neorv32-setups. Otherwise, (Yosys+)GHDL cannot find the built-in IEEE libraries:

warning: ieee library directory '../../bin/yosys/share/ghdl/ieee/v93/' not found
error: cannot find "std" library
ERROR: vhdl import failed.

Or maybe this is just a bug in my Makefile?!

Your Makefile isn't incorrect; the relative path you used for the GHDL plugin is one way to resolve the issue. However, there's an alternative approach that might simplify your setup, as detailed here. This method involves some additional steps when installing the GHDL-Yosys plugin, which can help avoid path-related problems.

Additionally, it might be helpful to review the config.mk file from the same repository. It contains some commands and configurations that could be of help. Also, @pu-cc it would be maybe a good idea to update the Toolchain Install with these added instructions regarding GHDL?