futex2 at Linux Plumbers Conference 2022

In-person conferences are finally back! After two years of remote conferences, the kernel development community got together in Dublin, Ireland, to discuss current problems that need collaboration to be solved.

As in the past editions, I took the opportunity to discuss about futex2, a project I’m deeply involved in. futex2 is a project to solve issues found in the current futex interface. This years’ session was about NUMA awaress of futex. Currently, futex allocates a single hash table in the kernel to store all data. If this is done in a NUMA machine, the table is allocated in a single node. This increases the cost of futex operations when done from any other node, given that will need to access a memory outside of it’s node.

To solve this, futex2 will allocate a table per NUMA node and have a NUMA-aware interface to tell the kernel which node to work on, using this struct:

struct futexX_numa {
   __uX value;
   __sX hint;
};

Where X can be a range of values [8, 16, 32, 64], for each available futex size in futex2. value is the futex value and hint is the node where you want to operate on. It can be -1 for the current node or [0, MAX_NUMNODES) to specify the node. So, for example:

struct futex32_numa f = {.value = 0, .hint = -1};

futex_wait(&f, 0, FUTEX_NUMA | FUTEX_32, NULL);

The session was short, but very productive, with Peter Zijlstra and Thomas Gleixner giving most feedback about the proposal. Here’s a list of ideas that were covered in the session and are to be consider when implementing this interface:

• Userspace always initialize .hint = -1. The kernel then overwrites -1 with the current node being used by the first waiter. In that way, every following futex operation (wait or wake) will be guaranteed to be on the same node.
• After the last waiter is waked, .hint can be set as -1 again.
• To avoid racing, any change to the struct futex_numa needs to be done atomically for the whole structure.
• The struct needs to have explicitly alignment and to be __packed.
• Benchmark data will for sure attracts reviewers and users. perf bench is the right place to add that.
• The local hash table doesn’t need to be as big as is the global futex table, probably num_possible_cpus() will be enough.
• Support for a struct futex64_numa might be tricky, so we might just restrict for smaller sizes. hint may be always s16 giving that MAX_NUMNODES isn’t that huge.

Here are my slides from this year. If you need any help with futex, you can get in touch with me andrealmeid@igalia.com.

Apart from futex, there were other talks to highlight. As a trend in recent Plumber editions, Rust was once again a hot topic in the conference. From a single session at LPC 2020, this year we had Rust in the main track, in the Maintainers Summit and its own Microconference. The initial mistrust of adding a second language to the kernel is long gone, and now the conversations are around the final bits on how to get it merged (Will the patches go though the maintainers tree, or directly to Torvalds? Which Rust version to use? Stable or unstable?). This whole effort is a good showcase of the conference ability to join efforts from different companies, communities and projects along the stack (GCC, clang) to solve complex problems together. Oh, and if you are wondering, Rust will likely be available in Linux 6.1 or 6.2.

From the GPU world we had a session to discuss how to deal with accelerator drivers in the kernel, as there’s some disagreement on how to accept (and where to accept) these new drivers. Along with that, a BoF to talk about getting a userspace console to be used in case the graphics stack crashes and how to get an implementation of cgroups for GPUs.

The Kernel Testing & Dependability Microconference was a stage to get news from testing tools, like syzbot, kselftests, KernelCI and KUnit. There was some discussion on how to integrate all frameworks as well, given that there’s a lot of testing nowadays, but not much communication and deploying of all this. Last but not least, Tales Aparecida (my GSoC mentee) presented the challenges of using KUnit to test device drivers.

That’s it for 2022 edition. Thanks Igalia for sponsoring my travel to LPC and see ya next year!