Opt-in busywait mode for futexes

[kateinoigakukun]

We are heavily using wasi-libc + wasip1-threads on browsers. Since wasi-libc uses memory.atomic.wait32 to implement futex, and the usage of memory.atomic.wait32 on the main thread is prohibited on browsers, we currently need to write code carefully not to reach the instruction.

Emscripten addresses this limitation by employing a busy-wait on the main thread as a workaround. Rust has always employs busywait regardless of the current thread is main. This approach, while effective for browsers, introduces unnecessary overhead in environments where memory.atomic.wait32 is permitted.

This change provides a similar solution by introducing an opt-in busy-wait mode for futexes. By making this an optional feature, we avoid imposing the overhead of busy-waiting in non-browser environments while ensuring compatibility with browser-based restrictions.

#include <wasi/libc-busywait.h>

/// Enable busywait in futex on current thread.
void __wasilibc_enable_futex_busywait_on_current_thread(void);

This change slightly adds some runtime overheads in futex to check if we should use busywait, but it can be optimized away as long as __wasilibc_enable_futex_busywait_on_current_thread is not used by user program and LTO is enabled.

[yamt]

We are heavily using wasi-libc + wasip1-threads on browsers. Since wasi-libc uses memory.atomic.wait32 to implement futex, and the usage of memory.atomic.wait32 on the main thread is prohibited on browsers, we currently need to write code carefully not to reach the instruction.

what happens when the instruction is executed on the main thread? does it trap?

[yamt]

while currently futex might be only the user of the blocking instruction, i guess it's better to name the api to make it clear it's about any blocking instructions, not necessarily for futex.
what do you think?

[kateinoigakukun]

Yeah, I'm also not a fan of the current name. Do you have any better idea?

[kateinoigakukun]

what happens when the instruction is executed on the main thread? does it trap?

Yes, it traps. From the JS embedder's view, it throws a JS exception.

[yamt]

after skimming the relevant discussions, i have a question.
how is the new api supposed to be used?
in the test code in this PR, it's hardcoded in main().
however, in general, a thread, even if it's C main(), can't know if it's on the web "main" thread or not, right?
if it's fine to hardcode the knowledge in the user program, the program can just override __wasilibc_futex_wait or whatever, can't it? i guess we can even provide a small library to override relevant libc logic.

[kateinoigakukun]

how is the new api supposed to be used?
in the test code in this PR, it's hardcoded in main().
however, in general, a thread, even if it's C main(), can't know if it's on the web "main" thread or not, right?

Yes, that's right. A user program needs to have a main entry point for each platform (web or not) and it's fine (at least for us) to hardcode the platform knowledge in the entry point of main and threads.

If we really don't want to hardcode the platform knowledge into the user program, it might be an option to export the enable function and ask embedders to call it.

if it's fine to hardcode the knowledge in the user program, the program can just override __wasilibc_futex_wait or whatever, can't it? i guess we can even provide a small library to override relevant libc logic.

Actually we are using the approach now, overriding the futex functions in the user program but we consider it as a workaround. We thought we just need to hook __wasilibc_futex_wait but we realized we also need to hook other functions calling __wasilibc_futex_wait in the same compile-unit (for now it's only __wait though).

So to make it overriding the default mode reliably, we have some options:

1. Add a builtin opt-in mode for busywait like this PR
2. Split out __wasilibc_futex_wait into a dedicated file to allow applications reliably overriding it
3. On the top-of 2, we can provide a supplemental library that contains __wasilibc_busywait.c added in this PR.

1 or 3 are best for us and other on-browser p1-threads users but 2 is better than nothing for us.