Skip to main content

Linux Kernel Board Bringup Series - SMP Developement

Plat SMP Developement in ARM Board
Reference: ZED Board (Zynq ZC702, Dual ARM Cortex A9)


The initialization of an SMP operating system typically runs on only one core - the primary core. When the system is fully booted, the remaining cores are brought online and this distinction between the primary core and the secondary cores is lost.

2. If all of the cores come out of reset at the same time, they normally all start executing from the same reset vector. The boot code then reads the cluster ID to determine which core is the primary.
 The primary core performs the initialization and then signals to the secondary ones that everything is ready.
An alternative method is to hold the secondary cores in reset while the primary core does the initialization. This method requires hardware support to coordinate the reset.

3.Processor ID
Booting provides a simple example of a situation where particular operations must be performed only on a specific core. Other operations perform different actions, depending on the core on which they are executing
The CP15:MPIDR Multiprocessor Affinity Register provides an identification mechanism in a multi-core system.
MPIDR reg. provides up to three levels of affinity identification, with eight bits identifying individual blocks at each level, that is, Affinity Level 0, 1, and 2.

4. This information is useful to an operating system scheduler, as it indicates the magnitude order of the cost of migrating a process to a different core, processor, or cluster.

5. SMP boot in Linux
The boot process comprises the following main stages:

    1. The kernel decompresses itself.
    2. The processor-dependent kernel code is executed to   initialize the CPU and memory.
    3. The processor-independent kernel code is executed to start the ARM Linux SMP kernel by       booting up all cores and initialize all the kernel components and data structures.

6.To boot the primary core, the kernel allocates a 4KB page as the vector page. It maps this using devicemaps_init() in the file arch/arm/mm/mmu.c to the location of the exception vectors, virtual address 0xFFFF0000 or 0x00000000. This step is invoked very early in the ARM system boot.

7. trap_init in arch/arm/kernel/traps.c copies the exception vector table, exception stubs, and kuser helpers into the vector page.
copy the exception vector table to the start of the vector page. Use a series of memcpy() operations to copy the exception stubs to address 0x200, and copy kuser helpers to the top of the page, at 0x1000 - kuser_sz.

8.  The primary core invokes the  boot_secondary() method to get a secondary core booted into the operating system.
 Implement this method for each mach type that supports SMP. Most of the other SMP boot functionality is extracted out into generic functions in linux/arch/arm/kernel.

9.While the primary core is booting, the secondary cores are held in a standby state, using the WFI instruction.
The primary core provides a startup address for the secondary cores and wakes them using an Inter-Processor Interrupt (IPI), an SGI signaled through the GIC.
Booting of the secondary cores is serialized, using the global variable pen_release.

10.The secondary cores being in a holding pen and being released one at a time, under the control of the primary core.
The variable pen_release is set by the kernel code to the ID value of the processor to boot and then reset by that core when it has booted
When an inter-processor interrupt (IPI) occurs, the secondary core checks the value of pen_release against their own ID value by using the MPIDR register.

11. Booting of the secondary core proceeds in a way similar to the primary core.
The secondary core enables the MMU. It enables the interrupt controller interface to itself and calibrates the local timers.
It sets a bit in cpu_online_map and calls cpu_idle(). The primary processor can detect the setting of the appropriate bit in cpu_online_map and set pen_release to the next secondary core.

12. SMP opeartions

struct smp_operations {
void (*smp_init_cpus)(void);
void (*smp_prepare_cpus)(unsigned int max_cpus);
void (*smp_secondary_init)(unsigned int cpu);
int (*smp_boot_secondary)(unsigned int cpu,
struct task_struct *idle);


-Setup the set of possible CPUs (via cpu_possible()).
-Can be removed if the CPU topology is up to date in the device tree.
-Called very early during the boot process (from


 * Initialise the CPU possible map early - this describes the CPUs
 * which may be present or become present in the system.
static void __init zynq_smp_init_cpus(void)
        int i;

        ncores = scu_get_core_count(zynq_scu_base);

        for (i = 0; i < ncores && i < CONFIG_NR_CPUS; i++)
                set_cpu_possible(i, true);


 -Enables coherency.
 -Initializes cpu_possible map.
  -Prepares the resources (power, ram,    clock...).

 -Called early during the boot process (before the initcalls but after setup_arch()).

static void __init zynq_smp_prepare_cpus(unsigned int max_cpus)


-Supposed to “perform platform specific initialization of the specified CPU”.
- using pen_release function or         secondary_start_kernel() function

 - Called from secondary_start_kernel() on the     CPU.

-Actually boots a secondary CPU identified by   the CPU number given in parameter.

-Called from cpu_up() on the booting CPU.


Popular posts from this blog

Linux Kernel Boot Process in RISC V - SiFive Boards -Introduction

Linux Boot Process in RISC V SiFive boards using BBL  1. RISC V Boards are having 4 (Four) Modes.

     User mode -- User space programs runs
     Supervisor mode -- Kernel runs here
     Hypervisor mode -- Unspecified
     Machine mode --  Machine instructions
SBI (Supervisory Binary Interface) is interaction between Machine Mode & Supervisor Mode
it provide set of api for Supervisor to interact with Machine mode , like set_timer, system_shutdown, system_reset etc. 
2. In ARM CPU No.s & same is represented in RISC V as Hart Ids.
Boot CPU, or Hartid 0 is given to BBL for Initial boot process,
other HartIds are keep spining still HartId reaches to kernel init process.
3.The mhartidCSR is read,  so Linux can be passed a unique per-hart identifier.4. A PMP is set up to allow supervisor mode to access all of memory.5. Machine mode trap handlers, including a machine mode stack, is set up. bbl's machine mode code needs to handle both unimplemented instructions and machine-mode interrupts…

DMA Debug - Zynq ZC702 - XMD Debugger

DMA Debug in Baremetal using XMD

Writing Startup Code for ARM Cortex A9 - Board bringup series - ZED Board

ZED Board uses Dual ARM Cortex A9
ZED board internally presents Debugger called XMD (Xilinx Microprocessor Debugger)
Xilinx provides Xilinx - SDK which supports Bare metal driver debug using XMD (XMD is having Node locked License)

This boot process is equal to debug using Hardware debugger like Lauterbach /ARM DS-5 /Segger Debugger & the Same is not possible with KGDB.
KGDB is supported grom GNU tools which can enter the Code in RAM (Maxium 4 Stack frames), Debugging code present outside the RAM is possible through Physical debuggers only.

Cortex A9 Processor boot method in Zynq SoC:

   1. The boot.S file contains a minimal set of code for transferring control from the processor's reset      2. Location to the start of the application. It performs the following tasks.
       Invalidate L1 caches, TLBs, Branch Predictor Array, etc.
       Invalidate L2 caches and initialize L2 Cache Controller.
      Enable caches and MMU
     Load MMU translation table base address into TLB  …