Let 32bit machines with limited memory size to use ubsan friendly

[ericchancf]

This patch series let 32 bit machines with limited memory size can use ubsan firendly.
Provide some changes as below:

1. Fix ssize_t compilation error with 32 bit machine
2. Fix compilation error which include arm cmsis with C++
3. Add some ubsan handlers which didn't exist yet.
4. Add test_ubsan for testing.
   becasue ubsan may cause out of memory limiation in compilcation time.
   So add a module only for testing ubsan.

[travisg]

Unfortunately this isn't really always true, it varies by architecture. For LK I've bee forcing size_t to be a long always, even if it's against the default the compiler sets it to, and it works reasonably well as long as the code is mostly using stuff built in tree.

Is there a strong need for this? If so we may be able to set it for particular architectures that its an issue with (such as linux-x86, where i know size_t is unsigned int).

[travisg]

Can you move the ubsan test module under /lib/ubsan/test instead of /lib/ubsan_test?

Otherwise looks pretty good except for the question about ssize_t.

[travisg]

ping.

[travisg]

Thanks for this, and apologies for the long delay. I've landed it as four commits (8c61f61..f8dd002) with your authorship preserved.

The core of it — making ssize_t the real signed counterpart of size_t instead of a hardcoded long — was the right call, and it removes the %zd warning class at the root. Changes from the PR as posted:

• typedef __PTRDIFF_TYPE__ ssize_t; rather than #define unsigned signed around __SIZE_TYPE__. It's the same type on every target LK builds for — I checked against the arm, aarch64, riscv32, riscv64, i386, x86_64, m68k, microblaze and or1k gcc toolchains and against clang — and it avoids the token-rewriting trick. (The __LP64__ switch in the first revision of the PR would still have been wrong on i386 and m68k, where __SIZE_TYPE__ is long unsigned int.)
• Alignment handler: the reported alignment was 1 << ffs(real_ptr), which is twice the real value, since lk's ffs() is the 1-based __builtin_ffs. Now real_ptr & -real_ptr; a pointer of 0x3 correctly reports 1-byte alignment.
• __ubsan_handle_float_cast_overflow was printing the source type name under a "value %s" label and the opaque value handle as an address. The handle's representation depends on the source type and val::to_string() only decodes integers, so it now reports the conversion that overflowed instead.
• test_ubsan moved to lib/ubsan/test as discussed in review, wired up via the normal MODULE_OPTIONS := test mechanism.
• The %zd sweep was extended to code added since you posted: app/cdcserialtest, app/irc, app/moot, app/ndebugtest, dev/block/ahci, dev/virtio/block, lib/elf, lib/klog, lib/ptable, lib/sysparam, platform/stm32f7xx, platform/zynq. lib/uefi/blockio_protocols.cpp was printing a size_t with %ld, which was wrong beforehand too.
• Removed the stale note saying __ubsan_handle_function_type_mismatch had been dropped from both compilers — clang 18 still emits a call to it from a plain -fsanitize=undefined build of C, so your handler is doing real work there.

Verified with scripts/buildall -q -e across all 58 projects (the 32-bit targets are the ones that matter for this), the clang and UBSAN buildall passes, and run-qemu-boot-tests.py on six architectures with and without --ubsan. Each commit builds standalone — worth noting the ssize_t change on its own broke UBSAN=1 builds until the three %ld sites in ubsan.cpp moved into that same commit.

One thing worth recording for anyone reading later: gcc doesn't emit checks for function_type_mismatch, alignment_assumption or float_cast_overflow, so the three handlers you added only fire under clang. UBSAN=1 TOOLCHAIN=clang LD=ld.lld scripts/do-qemuarm -6 -A 'lk.autorun=test_ubsan all' exercises them.

Closing in favour of the merged commits.