Inside low-level details of RTEMS/MMU project

Objectives :

This post aims to reveal low-level details of libmm for ARM BSPs. The main objectives are to know how low-level layer works and how to port libmm for a new (ARM) BSP.

ARM/LIBMM interaction :

Since libmm is about managing memory protection, Caches and MMU, libmm relies on ARM Co-processor 15: the primary co-processor which manages memory at the HW level for all modern ARM cores. CP-15 contains some control registers that help us to manage memory both at startup and dynamically.

The main CP-15 control register (C1) contains flags that we set/unset during RTEM initialization and application run-time. Figure-1 shows control register format on ARMv6. Please note that some of CP-15 C1 register flags may exist for a specific ARM architecture and absent for another one, also some flags maybe deprecated in modern architectures but still there not to hurt compatability. You can find each flag use and its meaning at the reference manual for ARM architectures [1] and CPU you develop in.

Figure-1 ARM CP-15 Control Register Format

The most important flags we use are M and C flags. M flag is used to enable/disable MMU, while C flag is used to enable/disable Cache unit.
CP-15 also contains other useful registers like :

• Translation Table Base Register : Points to the First level of page-tables (Sections or Super Sections)
• Domain Access Control : The purpose of the Domain Access Control Register is to hold the access permissions for a maximum of 16 domains.
• Cache Operations .
• TLB Operations.
• Interrupt Status Register.

 ARM CP-15 API in RTEMS

RTEMS has a header file (arm-cp15.h) the contains code needed to do any operation on CP-15 and its registers. This should be the backend API for ARM BSPs that implements low-level libmm API. Another new header file called arm-cp15-start.h is created to help initializing CP-15 at BSP startup. The next section illustrates how ARM CP-15 is initialized.

arm-cp15-start.h initialization 

The main components and areas that involve libmm/MMU initialization are

•  Linker defines : These defines at linker-symbols.h export macros and variables that point to registers like translation table base (page-table directory), object addresses for some sections and other useful linker variables. Figure-2 shows some linker variables quoted from ARM linker-symbols.h

• 

  Figure 2: Example linker symbols from ARM linker-symbols.h

•  mm-config-table is an initialization table which contains entries each of which describes a section of the program object file by its starting address, end address and CPU memory attributes it "should" have. Every BSP should define its own init table which is passed to mminit function. Figure-3 shows the format of such a table.

Figure-3 mm_config_table format

For CPU Memory Attributes field, it simply represents a page-table entry for the CPU libmm is ported for. Figure-4 shows an example of RaspberryPi/ARM1176JZF-S CPU first-level page table entries formats.

Figure-4 Backwards-compatible first-level descriptor format.

An example of mm_config_table is the table for RaspberryPi BSP. It uses some linker defines to set starting addresses and sizes of sections. For the CPU Memory Attributes field, arm-cp15.h macros are used. Figure-5 is a real code for mm_config_table for RaspberryPi BSP.

Figure-5 raspberrypi mm_config_table example.

•   linkcmds : This is the linker script passed to the linker during compilation/linking process. It manages how sections are organized in the final executable file and reserve memory areas for run-time demands. What concerns us about the linker file is to tell the linker where should be the global page-table and its size. Also this file should export the translation table base to other rtems low-level layers (such as libmm). For RaspberryPi BSP on RTEMs, the 128 MB memory is mapped as shown in figure-5.

Figure-6 RTEMS/RaspberryPi BSP memory map

•  mminit.c : This file contains an implementation for bsp_memory_management_initialize which is a part of libmm API. Every BSP that supports libmm should implement this function. mminit.c should contain all variables, macros, routines that initialize memory management HW. For RaspberryPi, the implementation is direct wrapper to arm_cp15_start_setup_translation_table_and_enable_mmu_and_cache included from arm-cp15-start.h. mm_config_table and its size are passed to this function which iterates over the table entries, and set HW page table entries according to mm_config_table entries.

Setting page table entries dynamically 

Some applications may need to set memory attributes for a specific area of memory dynamically (within application phase). Examples of these applications are dynamic linker/loader and POSIX mman.h API. libmm supports dynamic setting of memory attributes through a call to bsp_memory_management_set_attributes routine which is a part of libmm low-level API. Every BSP that supports libmm should implement this function. In RTEMS/ARM, we share mm.c file to support this feature. mm.c should translate high-level HW-independent libmm attributes (at score/mm.h file) to CPU specific attributes, thus, we can share high-level APIs between different BSPs and architectures. Figure-7 is a shared translation macro for different ARM BSPs (quoted from mm.c).

Figure-7 Translating high-level LIBMM attributes to CPU attributes.

At the run-time, the process of dynamically setting page-table attributes involves:

1. Disable interrupts.
2. Disable MMU and Cache unit.
3. Get page table indices for the region to apply attributes on.
4. Set page-table attributes.
5. Enable MMU and Cache unit.
6. Enable interrupts.

How to port libmm for a new BSP ? porting !

To port libmm for a new BSP you should simply follow these steps :

For static initialization part :

• Create mm_config_table according to the memory map of the new BSP. For ARM BSPs that share mminit.c and mm.c this table name should be the same (currently it's called bsp_mm_config_table).
• Define the size of the previous table with a shared name (bsp_mm_config_table_size).
• implement bsp_memory_management_initialize for the new BSP.

For dynamic part : 

• Create a translation attributes that convert high-level HW independent attributes (at score/mm.h) to the CPU of your BSP.
• implement bsp_memory_management_set_attributes

File that contains these implementation should include <libbsp/mm.h> which has prototypes for libmm low-level API.

How to make sure that the new port works ? porting !

To make sure your new port works fine you have to :

• Run sample tests (hello.exe, ticker.exe). If these tests runs as expected that means libmm initialization is done successfully, Otherwise the program should hang and give no output.
• Run mmtest2.exe (at libtests/mmtest2) that tests dynamic part of libmm and triggers some exceptions. Figure-8 is an example of mmtest2 output that runs successfully.

Figure-8 Output of mmtest2.exe

Acronyms : 

CP15    Co-Processor 15
libmm    memory management library
BSP      Board Support Package
MMU    Memory Management Unit
API       Application Programming Interface
TLB      Translation Lookaside Buffer
mminit    memory management initialize
init         initialization

References :

[1] ARM Architecture Reference Manual.
[2] github repo for the project.