Skip to content

bug fixes for running on metal - #142

Open
yogthos wants to merge 1 commit into
Anbeeld:mainfrom
yogthos:metal-bugfixes
Open

bug fixes for running on metal#142
yogthos wants to merge 1 commit into
Anbeeld:mainfrom
yogthos:metal-bugfixes

Conversation

@yogthos

@yogthos yogthos commented Aug 30, 2026

Copy link
Copy Markdown

Overview

  • kernels/quantize.h was missing quantize_q2_0s and quantize_q2_1. The kernel instantiations referencing them were added, but the functions themselves were never written, so the quantize Metal library failed to compile at runtime. Ported the implementations from ggml-quants.c (quantize_row_q2_0s_ref / quantize_row_q2_1_ref).
  • dequantize_iq4_nl assigned floats directly into a bfloat matrix element, which the Xcode 26.1 Metal compiler rejects. Now computes into a float4x4 temp and casts once at the end.
  • kernel_get_rows_f copied between mismatched scalar types (floatbfloathalf) without a cast. Added (T) on the assignment.

Requirements

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant