This article is still work in progress...

Pietrzak Roman
Kemu Studio - yosh.ke.mu
First version: 2017
Last changes: 2017-08-07
All rights reserved

Intro

This is short explanation on how to "take over" one (or more) CPU cores for user-space processing on Raspberry Pi with Linux.
Of course the general process is similar on any other linux platform.

Reasons

I was working on a project, where Raspberry Pi was used to generate specific waves in >Mhz bandwidth. The important part of the task, was to send the waveform to the GPIO outputs with high-accuracy timing requirements.
The Linux Kernel rescheduled my process from time to time - simply "stealing" my user-process time.

Linux is not real-time operating system. User-space threads/processes are rescheduled from time to time (Preemption) - this is how it works.
You can switch to RTOS to have better timing accuracy. However, Linux environment is rich in tools that may be hard to let go.

For 1-core CPU there is not much choice, Linux may be hard/impossible to tune to "almost realtime" performance.
However, for multi-core CPU, the idea of allocating one CPU-core to time-constrained task, while isolating this core from Linux scheduler is a potential solution.

Isolating the core

The official way of isolating the CPU-core from Linux scheduler is to use kernel param isolcpus.

The file for passing arguments to the Linux kernel can be found in:

/boot/cmdline.txt

The full kernel params list for Raspberry Pi:
https://github.com/raspberrypi/linux/blob/rpi-3.2.27/Documentation/kernel-parameters.txt

isolcpus param

The full explanation of isolcpus parameter:

isolcpus= [KNL,SMP] Isolate CPUs from the general scheduler.
Format:
    <cpu number>,...,<cpu number>
    or
    <cpu number>-<cpu number>
    (must be a positive range in ascending order)
    or a mixture
    <cpu number>,...,<cpu number>-<cpu number>

    This option can be used to specify one or more CPUs
    to isolate from the general SMP balancing and scheduling
    algorithms. You can move a process onto or off an
    "isolated" CPU via the CPU affinity syscalls or cpuset.
    <cpu number> begins at 0 and the maximum value is
    "number of CPUs in system - 1".

    This option is the preferred way to isolate CPUs. The
    alternative -- manually setting the CPU mask of all
    tasks in the system -- can cause problems and
    suboptimal load balancer performance.

Checking current status

To check what cores are currently isolated:

/sys/devices/system/cpu/isolated

Default is:

$ cat /sys/devices/system/cpu/isolated

empty line means no CPU-core isolated.

After setting isolcpus:

isolcpus=3

the system should report:

$ cat /sys/devices/system/cpu/kernel_max
3

Your process

To see the assignment of core to process really works, start:

top

and press "f", then enable:

P       = Last Used Cpu (SMP)

and back in process list you should find a new column showing the "last used core".

If what you're running is a non-main thread, then start top with -H:

top -H

to see child threads on the list.