Tuesday, October 31, 2017

RISC-V RTEMS port is Upstream


Having the RISC-V's GCC and Binutils upstream, as well as the increasing popularity and support RISC-V is getting pushed us to upstream RTEMS/RISC-V port, and it is gonna be part of the upcoming major RTEMS release.

RTEMS (The Real-Time Executive for Multiprocessor Systems) is a contemporary Real-time embedded OS, started as a project back in 1988. Since then, it has been used in space (e.g. by NASA), military, robotics and many other embedded applications.

This blog post gives a brief status of the porting efforts (that's upstream), as well as a quick how to build/run the port on Spike.


[RTEMS Kernel/OS]

  • Compatible with RISC-V priv-1.10.
  • Has interrupt support.
  • Only tested on Spike.
  • Only timer ISR is handled.
  • Uses HTIF for console IO, and power off.
  • Works entirely in Machine mode, standalone (doesn't rely on bbl).
  • Tier-2 RTEMS Architecture (For info about different RTEMS Tiers see this).
  • BSD-2 License.
  • Some of the code is copied from riscv-pk. 
The port is divided into 2 main parts:
  • CPU Port (riscv32, and riscv64)
    • General CPU port that can be used by different RISC-V-based boards. 
    • Fairly complete (but not optimized) for single core systems.
  • BSP port (Board Support Package).
    • generic_riscv that runs on Spike.
      • Only timer and console drivers are implemented.
    • generic64_riscv 64-bit of generic_riscv.
  • No cache or MMU management.
  • No FPU support.

 [RTEMS Source Builder] Toolchain

RTEMS has, maintains and tests its own toolchain for each CPU architecture (target), which relies on source code upstream from GNU (and others). For example, by using RSB to build RTEMS/RISC-V toolchain, RSB will fetch GCC and Binutils from GNU repos/servers, extract, build and install them for RTEMS. This is an example output after installing the RISC-V/RTEMS toolchain:

Installed riscv*-rtems toolchain (RSB)

We currently use the following revisions/releases for the RISC-V/RTEMS toolchain:

Tools built by RSB for riscv32

Furthermore, Spike/fesvr can be built using RSB (fetched from GitHub).

 [RTEMS Tester]

Scripts have been added to RTEMS Tester in order to be able to run the > 500 RTEMS tests on Spike. Currently, most of the tests pass on Spike.

Final output after running RTEMS Tester on riscv32 port


Binary sizes

With -Os flag:

minimum.exe (simplest RTEMS app that does nothing)

➜  build riscv32-rtems4.12-size riscv32-rtems4.12/c/riscv_generic/testsuites/samples/minimum/minimum.exe
   text    data     bss     dec     hex filename
  33944    8433 268393028       268435405       fffffcd riscv32-rtems4.12/c/riscv_generic/testsuites/samples/minimum/minimum.exe

hello.exe (Hello World App - Uses console driver, printf)
➜  build riscv32-rtems4.12-size riscv32-rtems4.12/c/riscv_generic/testsuites/samples/hello/hello.exe   
   text    data     bss     dec     hex filename
  75612    7768 268352004       268435384       fffffb8 riscv32-rtems4.12/c/riscv_generic/testsuites/samples/hello/hello.exe

ticker.exe (Uses clock driver and console driver, printf)
➜  build riscv32-rtems4.12-size riscv32-rtems4.12/c/riscv_generic/testsuites/samples/ticker/ticker.exe 
   text    data     bss     dec     hex filename
  66480    8744 268360196       268435420       fffffdc riscv32-rtems4.12/c/riscv_generic/testsuites/samples/ticker/ticker.exe



Final output after running RTEMS Tester on riscv32 port


Final output after running RTEMS Tester on riscv64 port


Hardware Platforms: We need to test on actual hardware in order to level up RISC-V/RTEMS port to a Tier-1 architecture (the highest, on par with ARM and x86). Issues with current RISC-V HW platforms (that I know):
  • HiFive1: We had a student working on HiFive1 port this summer part of Google Summer of Code. The main challenge was the memory size limitation.
  • FPGAs: Currently, I'm aware of the Rocket Chip as an FPGA/HW target. It relies on bbl as a bootloader. Furthermore, it changes to S-Mode before jumping to the payload entry. That's not the case for RTEMS as it works in M-Mode.
  • Do you have a RISC-V HW implementation: Please get it touch, we need more RISC-V/HW BSPs (with reasonably big enough memory for RTEMS e.g. > 128KiB). Only M-Mode is needed (for CPU) and console/timer (for BSPs). It would be also great if it can be remotely powered off/reset (for RTEMS Tester).

SMP support: Adding SMP support is easy for OSes working in S-Mode (just calling sbi_xxx functions). However, I've not investigated yet how feasible it is to implement SMP support for M-Mode-based implementations.

QEMU: Hasn't been updated for a while (and not upstream), and the latest privileged mode is 1.9. Tried to run the port on it though, but didn't work.

GDB: Would be great if it's gonna be upstream soon (with target sim?). RSB tries to build GDB for all targets/architectures, which is used part of RTEMS Tester as well (and to run basic sample apps).

Bugs: Will try to go over failed tests from RTEMS Tester and try to fix the bugs, unless someone beats me to it.

Quick HowTo

 To build the tools from source, RTEMS Source Builder is needed.

$ git clone git://git.rtems.org/rtems-source-builder.git
$ cd 

Setup development directories and export PATH:

$ mkdir -p ~/development/rtems
$ export PATH=$HOME/development/rtems/4.12/bin:$PATH
$ export RTEMS_DEV=$HOME/development/

[Build Toolchain]


rtems-source-builder git:(master) ✗ cd rtems/
$ ../source-builder/sb-set-builder --log=l-riscv32.txt --prefix=$RTEMS_DEV/rtems/4.12 4.12/rtems-riscv32


$ ../source-builder/sb-set-builder --log=l-riscv64.txt --prefix=$RTEMS_DEV/rtems/4.12 4.12/rtems-riscv64

[Build Spike]

rtems-source-builder git:(master) ✗ cd  bare
$ ../source-builder/sb-set-builder --log=spike --prefix=$RTEMS_DEV/rtems/4.12 devel/spike

[Build RTEMS]

Clone RTEMS:

 $ git clone git://git.rtems.org/rtems.git
 $ cd rtems && ./bootstrap -p && ./bootstrap 

To build generic_riscv BSP (32-bit)

$ cd .. && mkdir build && cd build
$ ../rtems/configure --target=riscv32-rtems4.12  --enable-rtemsbsp=riscv_generic --enable-tests=samples

To build generic64_riscv BSP (64-bit)

$ cd .. && mkdir build64 && cd build64
$ ../rtems/configure --target=riscv64-rtems4.12  --enable-rtemsbsp=riscv64_generic --enable-tests=samples



➜  build spike --isa=RV32IMAFDC -m0x10000000:0x10000000 riscv32-rtems4.12/c/riscv_generic/testsuites/samples/hello/hello.exe


hello.exe (Hello World App - Uses console driver, printf)

➜ build64 spike -m0x10000000:0x10000000 riscv64-rtems4.12/c/riscv64_generic/testsuites/samples/hello/hello.exe
Hello World
LD ***


(Uses clock driver and console driver in a multithreading test)

➜ build64 spike -m0x10000000:0x10000000 riscv64 rtems4.12/c/riscv64_generic/testsuites/samples/ticker/ticker.exe
TA1 - rtems_clock_get_tod - 09:00:00 12/31/1988
TA2 - rtems_clock_get_tod - 09:00:00 12/31/1988
TA3 - rtems_clock_get_tod - 09:00:00 12/31/1988
TA1 - rtems_clock_get_tod - 09:00:05 12/31/1988
TA1 - rtems_clock_get_tod - 09:00:10 12/31/1988
TA2 - rtems_clock_get_tod - 09:00:10 12/31/1988
TA1 - rtems_clock_get_tod - 09:00:15 12/31/1988
TA3 - rtems_clock_get_tod - 09:00:15 12/31/1988
TA1 - rtems_clock_get_tod - 09:00:20 12/31/1988
TA2 - rtems_clock_get_tod - 09:00:20 12/31/1988
TA1 - rtems_clock_get_tod - 09:00:25 12/31/1988
TA3 - rtems_clock_get_tod - 09:00:30 12/31/1988
TA1 - rtems_clock_get_tod - 09:00:30 12/31/1988
TA2 - rtems_clock_get_tod - 09:00:30 12/31/1988

[Run RTEMS Tester]



➜  build $RTEMS_DEV/rtems/rtems-tools/tester/rtems-test --log=riscv_generic.txt --rtems-bsp=riscv_generic --rtems-tools=$RTEMS_DEV/rtems/4.12 --timeout=40 riscv32-rtems4.12/c/riscv_generic/testsuites

➜  build $RTEMS_DEV/rtems/rtems-tools/tester/rtems-test --log=riscv64_generic.txt --rtems-bsp=riscv64_generic --rtems-tools=$RTEMS_DEV/rtems/4.12 --timeout=40 riscv64-rtems4.12/c/riscv64_generic/testsuites