libmm design on RTEMS for ARM architecture

First, if you do not know what libmm is about you should review that introduction post.
In this topic, I will give a brief overall picture of how libmm is designed and some implementation specific details for ARM.

Changes : 

Some changes were made to the API mentioned in the previous post. The major change is that both _Memory_management_Install_entry and _Memory_management_Uninstall_entry interfaces were merged and unified into a new interface called _Memory_management_Set_attributes. The new function takes three arguments :
base : Which is the starting address of the memory region to apply new attributes on.
size : The size of the memory address in bytes.
attributes : Abstract target-independent high-level attributes to be applied for a memory region. This argument describes common memory attributes for memory protection and caches.

The reason of the this re-factoring is that we got rid of control structs at high-level and low-level layers, So there is no "entries" to be installed or uninstalled anymore. This interface is similar to POSIX memory protection interface: mmprotect.

Design :

There are three major layers in RTEMS for libmm: score layer, cpu layer, low-level (implementation) layer.

score layer is the front-end API which the developer should deal with. The functions of the score layer were described in the introduction post and changes are mentioned here too. It also acquires locks to manage libmm implemented for SMP environment (like realview). cpu layer is a shared header file that is considered a link or a bridge between high-level score layer and implementation. All BSPs and targets that implement libmm should include this file. implementation layer is the target-dependent code that interfaces directly with hardware memory management system for a specific target. Figure1 shows the three layers and relations between each of them. Arrows indicate that one layer (on the "from" side of the arrow) depends another layer (on the "to" side of it).

Figre1: Relations between libmm layers.

ARM Implementation :

Shareability

Thanks to the good ARM architecture design and backward compatibility, RTEMS can share code for different architectures that deal with memory management part. This way, we can avoid low-level/libmm code duplication for different ARM BSP on RTEMS. libmm depends mostly at coprocessor15 for ARM targets that have MMU to configure memory management systems.

As an example of code shareability, is that RTEMS can use the same peace of code to initialize MMU and Caches at BSP startup as well as dynamically changing memory attributes at page tables. At the time of writing this post, RaspberryPi RTEMS BSP which has ARM1176JZF-S (ARMv6) processor share the same code with RealView BSP which has a Cortex™-A9 (ARMv7) processor. 

Performance and page sizes

For RTEMS, as being RTOS, the most important aspect to consider when making an implementation oriented decision is the performance and the speed. ARM MMU has two levels of page tables, each of which can describe memory attributes for different region sizes. The first level page-table can describe 16 MB super-sections or 1 MB sections. The second level page-table can describe 64 KB large pages or 4 KB pages. To keep TLB misses as few as possible beside avoiding the overhead of memory references and page table walks, we chose 1 MB sections as a basic unit for memory regions. libmm aligns up any region to 1 MB boundary.

Supported attributes

 Currently the supported attributes at high-level libmm layer are read-only, write-enable and no-access. These attributes can be easily extended to support Cache attributes too.

[GSoC] libmm project updates

* Setup RTEM environment for libmm. This action includes :

• libmm score API.
• libcpu/mm.h Header file to serve as an interface for low-level libmm API.
• Stubs for BSPs that do not implement libmm.
• mmtest1 test case.
•  Changes to all BSPs Makefiles to accommodate former files.

* libmm MPU implementation (not tested) : A simple implementation  that wraps ARMv7 MPU API.

* Change API prototypes and delete MME struct. Now low-level API takes base, size, attributes as args instead of MME struct pointer. 

* Adjust the current ARM MMU code and integrate it into Raspberry PI BSP (built successfully). The code is to be tested/debugged once I get the USB/TTL cable. 

[GSoC 2013] LIBMM for RTEMS

Introduction : 

libmm is memory management library for RTEMS. libmm aims is to centralize different memory management techniques supported by HW targets including MMU, MPU, Caches. Last year I worked on designing and implementing libmm and ported it for PSIM, gumstix BSPs [1] during GSoC 2012 program. This year I am working on enhancing libmm and port it for other targets and BSPs [2].

Till now, libmm has three major functions : 

• _Memory_management_Initialize : used to initialize memory resources (MMU, MPU and/or Caches) and any other required data structures. Also this function creates a static mapping for memory sections like .text, .data, IO pages, and others with suitable memory attributes.  It's invoked at BSP startup.
• _Memory_management_Install_entry : used to install entry into the target HW with supplied memory attributes flags representing cache policies, memory protection flags. It takes the start address for the area of memory to be installed, size, and flags. It can be used to dynamically install entries during application phase as well.
• _Memory_management_Uninstall_entry : When a memory management entry is no longer needed, this function should be invoked to clean up HW/SW resources and revert back to the state before calling  _Memory_management_Install_entry.

 BSPs support for libmm:

Currently libmm is ported for :

• PSIM : PowerPC/mpc6xx BSP (603e core).
• Gumstix : ARM based board (arm920t core).
• Realview_pbx_a9 : running on QEMU (Cortex-a9) new!
• Xilinx_zynq : running on QEMU (Cortex-a9) new!

Raspberry Pi is to be supported soon on real hardware.

Added features for libmm :

To use libmm within SMP environment, smplock calls are added to _Memory_management_Install_entry, _Memory_management_Uninstall_entry. 

For ARM/Realview :

-new extension function is implemented to view more info about address caused data abort exception and which error caused the exception.

- As page tables are shared resource between cores and threads, It's mapped as "SHARED", thus, let the HW ensure caches/memory coherency for that area of memory.

Tests cases :

mmtest1: Simple tests that tries to install memory management entries.

mmtest2: 

•  Install entries with specific memory attributes (e.g read only region) :
•  Check for memory protection violations (writing to read only blocks)
•  Reading from read only blocks.
•  Write/Read to/from unmapped region (error!).
•  Write to a valid entry that was installed and then uninstalled (error!).

mmtest3 (on-going): new!

Tests for libmm behavior on SMP environments.

• Create tasks for each core and start it.
• Check for memory consistency and page tables and memory attributes validity.

Snapshot for mmtest2 running on Realview/QEMU:

References :

[1] https://github.com/heshamelmatary/rtems-gsoc2012
[2] https://github.com/heshamelmatary/rtems-gsoc2013

[RTEMS] High-level Design for Arena Manager

Introduction to Arena :

The main problem we want to solve is how to make it easier for users to apply memory protection attributes in a flexible way. Developer who want to control this area of memory and apply/modify or even delete memory protection attributes should make use of libmmu [1] (the result of previous GSoC2009/2011 projects that provide memory protection through mmu). But developer has to know how to work with this API and some low-level knowledge about how the core implements memory protection. To satisfy the need for usable memory protection, I am introducing arenas, which control memory protection attributes of an area of memory. Arena only has the start address and size of an area of memory and needed memory protection attributes to be applied on this area. No libmmu or low-level knowledge is required by a developer who uses Arena to apply memory protection, only the memory area description and memory protection attributes.

Arena could be created by a developer by passing the chunk of memory that needs to have memory protection attributes. Arena can support sharing if there is an area of memory that a developer wants to share between many tasks with the same access attributes (i.e code segment ). Arena could be defined as a construct which specifies memory attributes that could be shared among tasks.

Arenas and tasks have a many-to-many symmetric relationship. All tasks having access to an Arena are "attached" to that Arena. Also a set of Arenas that a specific task has access to are said to be its "Attached Arenas". Context switch code deactivates all Arenas the current task is attached to, and activates the next task’s attached Arenas therefore the CS code needs a way to access all of the arenas attached to a given task. Representing attached Arenas for a task in a linked list/array to keep track of relationship between a task and its attached Arenas is useful so that a context switch code can know which Arenas to deactivate by just have a pointer to current task’s attached Arenas list in its TCB and walk through it to deactivate all the list. When Arena is deleted, all attached tasks TCBs must be updated by deleting this Arena from their lists. The process of updating TCBs could be done by representing attached tasks in a linked list of pointers to TCBs in the ACB, So when an Arena is deleted it traverses the list and updates each attached task’s TCB.

Further (optional) goal is to embed an allocator to an Arena which could be. Then an Arena could be used to allocate memory from its pool, providing tasks a way to share and divide protected memory regions efficiently. Another further goal is to provide Posix mmap support and implementing it using arenas. Also i consider adding naming capability to an Arena.

[1] : http://code.google.com/p/gsoc2011-rtems-mmu-support-project/

Proposed Interfaces :

struct Arena_Control { Objects_Control Object; /* starting address of the arena’s memory pool */ void *start_address ; /* size of the arena’s memory pool */ size_t size ; /* memory attributes like w/r/x, cacheable */ uint32_t attributes; /* flag to indicate whether arena is active or not */ bool is_active; /* linked list of tasks that are attached to this arena. Points to * an Arena_Per_task structure embedded in a Thread_Control. */ Chain_Control attached_tasks;}





// The TCB would have a pointer to an array of the following struct.struct Arena_Per_task { /* node of the task in attached_arena->attached_tasks list */ Chain_Node node; /* pointer to a specific Arena which this task is attached to. */ Arena_Control *attached_arena;}





struct Thread_Control { … /* array of Arena_Per_task of fixed maximum size. NULL if arenas are not used. */ Arena_Per_task *attached_tasks; …}

Arena_Status _Arena_Initialize( ) ;
/* This function will apply setup fields in Arena_Control and apply memory protectionattributes using libmmu API. Also initializing an empty list of attached_tasks. */Arena_Control* _Arena_Create(void* start_address , size_t size , uint32 attributes ) ;
/* This function returns attributes of a given Arena */ uint32_t _Arena_Get_attributes (Arena_Control*);
/* This function returns the starting address of that Arena in memory void* _Arena_Get_start_address (Arena_Control* Arena) ;
/* This function returns the size of the Arena size_t _Arena_Get_size (Arena_Control* Arena);
/* This function update the current attributes for an Arena with new_attributes , could be used for deleting protection */ Arena_Status _Arena_Update_attributes(Arena_Control* , uint32_t new_attributes) ;
/* This function create a link between an Arena and a task by adding a pointer to ACB in TCB’s attached arenas list and By adding a pointer to TCB in ACB by adding a pointer to attached task list */ Arena_Status _Arena_Attach(Arena_Control* , Thread_Control* task);
/* This function cut the link between a given Arena one of its attached tasks */ Arena_Status _Arena_Detach(Arena_Control* , Thread_Control* task);
/* This function mark an Arena as Deactivated when the current task use it and a context switch happens, so that the next task can not access this deactivated Arena. */ Arena_Status _Arena_Deactivate(Arena_Control*) ;
/* This function mark an Arena as Activated when a context switch happens, and the next task is attached to that Arena */ Arena_Status _Arena_Activate(Arena_Control*);
/* This function delete the Arena and update the Arena list in every attached task TCB by deleting the ACB entry of the deleted Arena */Arena_Status _Arena_Delete(Arena_Control* );

Optional interfaces :

Arena_Control* _Arena_Create_with_allocation(void* xxx_control , uint32 attrib ) ;
-xxx_control : is the control struct like heap_control
attrib may consist of the following
- attributes : that indicates rwx of that Arena and other attributes
- used allocator : that indicates the allocator used by this arena ( if it relies on an allocator )
- The other bits are to be filled (optional)

Arena_Status arena_allocate ( Arena_Control* , uint32 size ) ;


Arena_Status arena_free ( Arena_Control* , void* adress);


// Would we define ARENA_MAX_NUMBER and it initialization declare Arena_Control structs ?