Threadpool ndk#15

Closed

max-krasnyansky wants to merge 48 commits intomasterfrom

Commits on Aug 27, 2024

Introduce ggml_compute_threadpool

fmz
authored and
fmz
committed
Minor fixes

fmz
authored and
fmz
committed
fixed use after release bug

fmz
authored and
fmz
committed
fixed a harmless race condition

fmz
authored and
fmz
committed
Fix Android bulid issue

fmz
authored and
fmz
committed
fix more race conditions

fmz
authored and
fmz
committed
fix deadlock for cases where cgraph.n_nodes == 1

fmz
authored and
fmz
committed
threadpool: use cpu_get_num_math to set the default number of threadpool threads

max-krasnyansky
authored and
fmz
committed
bench: create fresh threadpool for each test

max-krasnyansky
authored and
fmz
committed
atomics: always use stdatomics with clang and use relaxed memory order when polling in ggml_barrier

max-krasnyansky
authored and
fmz
committed
threadpool: make polling the default to match openmp behavior

max-krasnyansky
authored and
fmz
committed
threadpool: do not wakeup threads in already paused threadpool

max-krasnyansky
authored and
fmz
committed
fix potential race condition in check_for_work
fmz
committed
threadpool: do not create two threadpools if their params are identical

max-krasnyansky
authored and
fmz
committed
threadpool: reduce pause/resume/wakeup overhead in common cases

max-krasnyansky
authored and
fmz
committed
threadpool: add support for hybrid polling

max-krasnyansky
authored and
fmz
committed
threadpool: reduce the number of barrier required

max-krasnyansky
authored and
fmz
committed
threadpool: remove special-casing for disposable threadpools

max-krasnyansky
authored and
fmz
committed
threadpool: do not clear barrier counters between graphs computes (fixes race with small graphs)

max-krasnyansky
authored and
fmz
committed
threadpool: use relaxed order for chunk sync

max-krasnyansky
authored and
fmz
committed
threadpool: remove abort_callback from threadpool state

max-krasnyansky
authored and
fmz
committed
threadpool: better naming for thread/cpumask releated functions

max-krasnyansky
authored and
fmz
committed
threadpool: consistent use of int type for n_threads params

max-krasnyansky
authored and
fmz
committed
threadpool: add support for ggml_threadpool_params_default/init

max-krasnyansky
authored and
fmz
committed
threadpool: move typedef into ggml.h

max-krasnyansky
authored and
fmz
committed
threadpool: fix apply_priority() function name

max-krasnyansky
authored and
fmz
committed
threadpool: fix swift wrapper errors due to n_threads int type cleanup

max-krasnyansky
authored and
fmz
committed
threadpool: enable --cpu-mask and other threadpool related options only if threadpool is enabled

max-krasnyansky
authored and
fmz
committed
threadpool: replace checks for compute_thread ret code with proper status check

max-krasnyansky
authored and
fmz
committed
threadpool: simplify threadpool init logic and fix main thread affinity application

max-krasnyansky
authored and
fmz
committed
threadpool: update threadpool resume/pause function names

max-krasnyansky
authored and
fmz
committed
threadpool: enable openmp by default for now

max-krasnyansky
authored and
fmz
committed
threadpool: don't forget to free workers state when omp is enabled

max-krasnyansky
authored and
fmz
committed
threadpool: avoid updating process priority on the platforms that do not require it

max-krasnyansky
authored and
fmz
committed
threadpool: update calling thread prio and affinity only at start/resume

max-krasnyansky
authored and
fmz
committed
llama-bench: turn threadpool params into vectors, add output headers, etc

max-krasnyansky
authored and
fmz
committed
llama-bench: add support for cool off between tests --delay
max-krasnyansky
committed