Skip to content

Releases

b12255

b12255 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Nov 19:11
940a30e

b12254

b12254 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Nov 18:54
45c8ea8

b12253

b12253 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Nov 18:36
c90aa99

common : escape control characters in tool argument strings (#31937)

normalize_quotes_to_json copies raw control characters verbatim while
converting pythonic tool arguments to JSON, so calls like
python(code='ab') produce invalid JSON in the arguments returned to
clients; replaying that history then fails hard. Escape C0 control
characters inside quoted strings via escape_json_string_inner.

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b12252

b12252 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Nov 18:14
a2c942d

b12250

b12250 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Nov 17:13
ae4893a

vulkan: add tiled-transpose concat shader for Qwen models (#30591)

  • vulkan: tiled transpose for dim-0 concat with a transposed source

The delta-net conv-state path does ggml_transpose() straight into a dim-0
ggml_concat(), so the generic concat kernel reads src1 with the transposed
stride, one element per thread, and neighbouring lanes land on different cache
lines. Stage a 32x32 tile through shared memory instead, which keeps both the
load and the store coalesced. 33 columns pads away the bank conflicts.

Gated narrowly: dim 0, 2D, unquantized 4-byte elements, contiguous src0/dst
rows, and src1 actually transposed. Everything else falls through to the
existing kernel.

Qwen3.6-35B-A3B on gfx1151/RADV, 30 gated-delta-net layers: the CONCAT op goes
11877 us -> 957 us, and pp2048 at ub2048 goes 1224 -> 1628 t/s. Generated text
is unchanged.

The approach follows the tiled transposed concat Gaetan Puleo wrote for the HIP
backend in his Strix Halo fork.

Co-Authored-By: Claude Opus 5 noreply@anthropic.com

  • enable by default, add smaller shader sizes

  • clean up comments

  • improve shader

  • Add non-contiguous concat tests

Co-authored-by: ravel7524 58877666+ravel7524@users.noreply.github.com

  • handle x or y axis overflow

Co-authored-by: Nathan Wilson nath.flagman151@passmail.com
Co-authored-by: Claude Opus 5 noreply@anthropic.com
Co-authored-by: ravel7524 58877666+ravel7524@users.noreply.github.com

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b12249

b12249 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Nov 16:57
6122b56

ggml-rpc : fail gracefully on unreachable endpoints and failed handshakes (#30415)

  • ggml-rpc: fail gracefully on unreachable endpoints and failed handshakes

rpc_dispatcher::start() aborted the whole process (GGML_ABORT -> SIGABRT) when
the endpoint could not be parsed, the transport failed to initialise, the socket
could not connect, or the HELLO handshake failed. A typo in --rpc or a
temporarily down node killed the host application, and the reason went to a log
stream the host may never see.

Make the connect-time failure reportable instead of fatal: start() returns bool
(LOG_ERROR + false; the dispatcher thread only starts after a successful
handshake), negotiate_hello() returns false on a half-open socket,
get_dispatcher() returns nullptr and does not cache failed attempts, and every
consumer treats nullptr as "endpoint did not answer" with the conservative
behaviour of its domain (allocation -> nullptr, alloc-size -> lower bound,
device memory -> 0/0, events -> no-op, device count -> 0, and add_server()
already maps 0 devices to nullptr).

Mid-stream failures keep the existing RPC_STATUS_ASSERT by design (fail fast
against silent corruption) and are intentionally out of scope.

Field motivation: on Android (llama.rn + phone RPC cluster) an unreachable
worker crashed the app; the crash buffer showed
abort <- ggml_abort <- rpc_dispatcher::start <- ggml_backend_rpc_add_server.

  • tests : add a regression test for unreachable rpc endpoints

Calls ggml_backend_rpc_add_server with a socket that nothing listens on and with a
malformed endpoint, and expects nullptr from both. Without the fix in this branch
the first call aborts the process, so the test cannot even fail cleanly.

  • ggml-rpc remove low-value comments

  • tests : drop the internal device-count check after the rpc registry refactor


Co-authored-by: figkcoci-debug figkcoci-debug@users.noreply.github.com

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b12248

b12248 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Nov 16:08
3837bf0

b12247

b12247 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Nov 15:01
c1a4feb

b12246

b12246 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Nov 13:09
78ffbaa

vulkan: cap scalar flash attention at 8 rows for large heads on Qualcomm (#30829)

With head sizes above 128 and KV >= 1024, scalar flash attention uses
16 rows per workgroup. On the Qualcomm Adreno driver this variant
spills registers to scratch memory (264 bytes per invocation on the
X1-85, none at 8 rows) and runs about 40x slower. The largest
test-backend-ops case then becomes a single dispatch of several
seconds, past the Windows GPU timeout, and the test never completes on
the X1 Elite.

Use at most 8 rows per workgroup on Qualcomm when hsk or hsv > 128.
Smaller heads keep 16 rows, which is faster for them on the X2 Elite.

Adreno X1-85, FLASH_ATTN_EXT hsk=256 kv=4096: nb=64 711 ms -> 17 ms,
nb=512 no result -> 125 ms; test-backend-ops completes. Adreno X2-90:
Gemma 4 E4B pp512 at depth 4096 with flash attention 81-90 -> 117 t/s.

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b12244

b12244 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Nov 07:25
d5bbcd1