Commit Graph

159 Commits

Author SHA1 Message Date
Dmitry Muhomor
03883eb2ce mte: rename arm_mte_store_tags_and_clear() to arm_mte_tag_and_clear_mem() 2023-10-30 14:20:53 -04:00
Dmitry Muhomor
7a6dbd8152 mte: add comment about the reserved slab canary value 2023-10-30 14:20:53 -04:00
Dmitry Muhomor
93aa9eefe4 mte: make h_malloc_disable_memory_tagging() thread-safe 2023-10-30 14:20:53 -04:00
Dmitry Muhomor
01a199e19e mte: move is_memtag_enabled to read-only allocator data 2023-10-30 14:20:53 -04:00
Dmitry Muhomor
f042a6b9b0 android: add function for disabling MTE at runtime
On Android, MTE is always enabled in Zygote, and is disabled after fork for apps that didn't opt-in
to MTE.

Depends on the slab canary adjustments in the previous commit.
2023-10-30 14:20:53 -04:00
Dmitry Muhomor
001fc86585 mte: disable slab canaries when MTE is on
Canary with the "0" value is now reserved to support re-enabling slab canaries if MTE is turned off
at runtime.
2023-10-30 14:20:53 -04:00
Dmitry Muhomor
70c91f4c3e mte: disable write-after-free check for slab allocations when MTE is on
Freed slab memory is tagged with a reserved tag value that is never used for live allocations.
2023-10-30 14:20:53 -04:00
Dmitry Muhomor
e3686ae457 add support for Arm MTE memory tagging
- tag slab allocations with [1..14] tags
- tag freed slab allocations with the "15" tag value to detect accesses to freed slab memory
- when generating tag value for a slab slot, always exclude most recent tag value for that slot
(to make use-after-free detection more reliable) and most recent tag values of its immediate
neighbors (to detect linear overflows and underflows)
2023-10-30 14:20:53 -04:00
Christian Göttsche
9cb4e6daf6 do not skip init() in realloc()
If N_ARENA is greater than 1 `thread_arena` is initially to N_ARENA,
which is an invalid index into `ro.size_class_metadata[]`.

The actual used arena is computed in init().

Ensure init() is called if a new thread is only using realloc() to avoid
UB, e.g. pthread_mutex_lock() might crash due the memory not holding an
initialized mutex.

Affects mesa 23.2.0~rc4.

Example back trace using glmark2 (note `arena=4` with the default
N_ARENA being 4):

    Program terminated with signal SIGSEGV, Segmentation fault.
    #0  ___pthread_mutex_lock (mutex=0x7edff8d3f200) at ./nptl/pthread_mutex_lock.c:80
            type = <optimized out>
            __PRETTY_FUNCTION__ = "___pthread_mutex_lock"
            id = <optimized out>
    #1  0x00007f0ab62091a6 in mutex_lock (m=0x7edff8d3f200) at ./mutex.h:21
    No locals.
    #2  0x00007f0ab620c9b5 in allocate_small (arena=4, requested_size=24) at h_malloc.c:517
            info = {size = 32, class = 2}
            size = 32
            c = 0x7edff8d3f200
            slots = 128
            slab_size = 4096
            metadata = 0x0
            slot = 0
            slab = 0x0
            p = 0x0
    #3  0x00007f0ab6209809 in allocate (arena=4, size=24) at h_malloc.c:1252
    No locals.
    #4  0x00007f0ab6208e26 in realloc (old=0x72b138199120, size=24) at h_malloc.c:1499
            vma_merging_reliable = false
            old_size = 16
            new = 0x0
            copy_size = 139683981990973
    #5  0x00007299f919e556 in attach_shader (ctx=0x7299e9ef9000, shProg=0x7370c9277d30, sh=0x7370c9278230) at ../src/mesa/main/shaderapi.c:336
            n = 1
    #6  0x00007299f904223e in _mesa_unmarshal_AttachShader (ctx=<optimized out>, cmd=<optimized out>) at src/mapi/glapi/gen/marshal_generated2.c:1539
            program = <optimized out>
            shader = <optimized out>
            cmd_size = 2
    #7  0x00007299f8f2e3b2 in glthread_unmarshal_batch (job=job@entry=0x7299e9ef9168, gdata=gdata@entry=0x0, thread_index=thread_index@entry=0) at ../src/mesa/main/glthread.c:139
            cmd = 0x7299e9ef9180
            batch = 0x7299e9ef9168
            ctx = 0x7299e9ef9000
            pos = 0
            used = 3
            buffer = 0x7299e9ef9180
            shared = <optimized out>
            lock_mutexes = <optimized out>
            batch_index = <optimized out>
    #8  0x00007299f8ecc2d9 in util_queue_thread_func (input=input@entry=0x72c1160e5580) at ../src/util/u_queue.c:309
            job = {job = 0x7299e9ef9168, global_data = 0x0, job_size = 0, fence = 0x7299e9ef9168, execute = <optimized out>, cleanup = <optimized out>}
            queue = 0x7299e9ef9058
            thread_index = 0
    #9  0x00007299f8f1bcbb in impl_thrd_routine (p=<optimized out>) at ../src/c11/impl/threads_posix.c:67
            pack = {func = 0x7299f8ecc190 <util_queue_thread_func>, arg = 0x72c1160e5580}
    #10 0x00007f0ab5aa63ec in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
            ret = <optimized out>
            pd = <optimized out>
            out = <optimized out>
            unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139683974242608, 2767510063778797177, -168, 11, 140727286820160, 126005371879424, -4369625917767903623, -2847048016936659335}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0,
              0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
            not_first_call = <optimized out>
    #11 0x00007f0ab5b26a2c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
2023-09-26 20:03:02 -04:00
Christian Göttsche
7d75acc62a use strict prototype
h_malloc.c:83:21: error: function declaration isn’t a prototype [-Werror=strict-prototypes]
       83 | static inline void *get_slab_region_end() {
          |                     ^~~~~~~~~~~~~~~~~~~
2023-06-10 14:18:27 -04:00
Daniel Micay
64dad0a69f drop legacy glibc support 2023-06-10 14:04:46 -04:00
Daniel Micay
d3152b8e8f preserve errno for free calls
This is a future POSIX requirement recently implemented by musl and
glibc.
2023-02-17 13:07:26 -05:00
Daniel Micay
2e9daf3122 merge fprintf/fputs calls in malloc_info 2023-02-17 13:07:26 -05:00
Daniel Micay
2250130c53 remove unnecessary UNUSED marker 2022-09-16 01:03:47 -04:00
Daniel Micay
8f38bbdee6 add configuration for self-init
This needs to be disabled for compatibility with the exploit protection
compatibility mode on GrapheneOS. hardened_malloc shouldn't be trying to
initialize itself when exploit protection compatibility mode is enabled.
This has to be handled in our Bionic integration instead.
2022-09-14 03:41:31 -04:00
jvoisin
b511696c55 clean up includes and remove non-portable includes
This marginally increases the portability of hardened_malloc,
eg. on OSX.
2022-02-07 07:14:51 -05:00
Daniel Micay
448170a412 fix case for non-macro constant 2022-01-21 23:59:37 -05:00
Daniel Micay
995ce07d45 add is_init likely/unlikely markers 2022-01-21 19:46:49 -05:00
Daniel Micay
c9d1abcd7e explicitly mark fatal error conditions unlikely 2022-01-21 19:45:05 -05:00
Daniel Micay
8f0b252c33 mark more out-of-memory conditions as unlikely 2022-01-21 19:03:02 -05:00
Daniel Micay
3cffc1e1af treat zero size malloc as unlikely
Calls to malloc with a zero size are extremely rare relative to normal
usage of the API. It's generally only done by inefficient C code with
open coded dynamic array implementations where they aren't handling zero
size as a special case for their usage of malloc/realloc. Efficient code
wouldn't be making these allocations. It doesn't make sense to optimize
for the performance of rare edge cases caused by inefficient code.
2022-01-21 18:27:04 -05:00
Daniel Micay
b3d78bd5f6 use static const for local constants 2022-01-16 21:02:17 -05:00
Daniel Micay
8d61e63274 add comment about special small size classes 2022-01-16 20:50:49 -05:00
Daniel Micay
81cf2f27a0 calculate slab size class instead of array loop 2022-01-16 16:18:14 -05:00
Daniel Micay
d8cb2d9f7a use consistent wrappers around clz/ffs 2022-01-16 15:39:59 -05:00
Daniel Micay
86f9c739ee define constant for u64 bit width 2022-01-16 15:06:36 -05:00
Daniel Micay
536f852538 reuse a single size alignment implementation 2022-01-16 14:44:28 -05:00
Daniel Micay
2a5662948e rename bitmap manipulation functions 2022-01-04 12:14:55 -05:00
Daniel Micay
d1c39edc9b use const for malloc_object_size API 2022-01-04 10:14:41 -05:00
Daniel Micay
5f32942263 get rid of canary_value when canaries are disabled 2022-01-03 20:39:30 -05:00
Daniel Micay
3696f071a4 use SLAB_CANARY for conditional checks 2022-01-03 02:17:04 -05:00
Daniel Micay
8ae78237ae avoid unnecessarily mixing 32-bit and 64-bit ints
It's ever so slightly faster to stick to stick to 64-bit arithmetic and
it avoids clang tidy being unhappy about the implicit widening.
2022-01-03 00:54:43 -05:00
Daniel Micay
3f8e9d3184 make MREMAP_MOVE_THRESHOLD into size_t constant
This avoids a clang-tidy warning and is a bit cleaner.
2022-01-03 00:32:06 -05:00
jvoisin
9142a9376b Add a bunch of const qualifiers 2021-12-30 21:25:16 -05:00
jvoisin
0655c1d024 Add a missing const 2021-12-26 18:19:59 -05:00
Daniel Micay
e41d37c3de remove unnecessary else 2021-09-30 10:57:05 -04:00
Daniel Micay
be6dde66f9 fix missing include for Intel MPK support 2021-05-21 09:07:28 -04:00
Daniel Micay
27fcfccb67 make __GLIBC_PREREQ check for mallinfo2 portable 2021-05-12 22:53:20 -04:00
Daniel Micay
da190f1469 mark pvalloc error path as unlikely 2021-05-12 21:01:13 -04:00
Daniel Micay
b0f81365a8 reuse code for aligned allocation API entry points 2021-05-12 20:59:04 -04:00
Daniel Micay
c9820b6e37 mark alloc_aligned_simple error path unlikely 2021-05-12 20:41:46 -04:00
Daniel Micay
f1cdc1e484 remove disconcerting newline 2021-05-12 20:34:18 -04:00
Daniel Micay
26b74b87bf improve code reuse for malloc API entry points 2021-05-12 20:28:50 -04:00
Daniel Micay
89faba4232 set errno in malloc_get_state to match glibc 2021-05-12 20:19:12 -04:00
Daniel Micay
a45dacc57b add support for glibc mallinfo2 2021-05-12 20:07:15 -04:00
Daniel Micay
f9a8e7216b purge slab memory even if using MAP_FIXED fails 2021-05-12 00:45:19 -04:00
Daniel Micay
5c974bdf82 use region quarantine even if MAP_FIXED call fails
This is a more sensible way of handling an out-of-memory failure in this
edge case. It doesn't matter much in practice.
2021-05-12 00:20:03 -04:00
Daniel Micay
2335f56713 add wrapper function for getting slot count 2021-05-10 07:04:50 -04:00
Daniel Micay
13a3aa16d0 improve naming of adjust_size_for_canaries 2021-05-07 04:23:49 -04:00
Daniel Micay
8bfa1a7dd5 use 1 slot for all extended size classes
This reduces memory usage and improves security in combination with the
guard slab feature.
2021-05-01 22:10:20 -04:00