You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Added vulkan support for GGML_OP_GATED_LINEAR_ATTN
This backend op used to not be supported on vulkan and fell back to cpu. Now the kernel follows the existing wkv6.comp pattern w/ a GLA-specific update-before-read ordering and an output "scale" push constant.
supports_op is limited to F32 and head_size == 64 (shader hardcodes BLOCK_SIZE 64, same as WKV6).
Modeled on the vulkan WKV6 path. Checked against the CPU reference in ggml_compute_forward_gla_f32
Test results (AMD Radeon 780M, Windows)
test-backend-ops.exe test -b Vulkan0 -o GATED_LINEAR_ATTN 4/4 tests passed
adjacent recurrence ops after rebase:
RWKV_WKV6: 4/4
RWKV_WKV7: 4/4
GATED_DELTA_NET: 36/36
Full Vulkan0 suite: 14400/14443. Remaining failures are preexisting DIV(type=f16, ...)`NMSE misses on this device/driver. reproduced on clean master with this change stashed and they remain.
AI usage disclosure:
YES: AI was used in the beginning to understand the codebase and for easier search and navigation to different similar points. Code was all handwritten and then AI was used to review correctness and any gaps/oversights.
Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:
PR Template not respected: Please respect the template when creating a new pull request. Make sure to fill out all required sections.
Large PR: Large changes require prior discussion (e.g. an issue or RFC) and maintainers may not be able to review this PR as-is. Consider splitting it into smaller, focused PRs.
Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.
Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:
PR Template not respected: Please respect the template when creating a new pull request. Make sure to fill out all required sections.
Large PR: Large changes require prior discussion (e.g. an issue or RFC) and maintainers may not be able to review this PR as-is. Consider splitting it into smaller, focused PRs.
Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.
Not a Large PR. maybe the bot thinks so due to the vulkan.csv file updated with this:
test-backend-ops support --output csv -b Vulkan0 > docs/ops/Vulkan.csv
PR follows the template
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
documentationImprovements or additions to documentationggmlchanges relating to the ggml tensor library for machine learningVulkanIssues specific to the Vulkan backend
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Added vulkan support for GGML_OP_GATED_LINEAR_ATTN
This backend op used to not be supported on vulkan and fell back to cpu. Now the kernel follows the existing wkv6.comp pattern w/ a GLA-specific update-before-read ordering and an output "scale" push constant.
supports_op is limited to F32 and head_size == 64 (shader hardcodes BLOCK_SIZE 64, same as WKV6).
Part of #14909
Additional information
Modeled on the vulkan WKV6 path. Checked against the CPU reference in ggml_compute_forward_gla_f32
Test results (AMD Radeon 780M, Windows)
test-backend-ops.exe test -b Vulkan0 -o GATED_LINEAR_ATTN 4/4 tests passed
adjacent recurrence ops after rebase:
Full Vulkan0 suite: 14400/14443. Remaining failures are preexisting DIV(type=f16, ...)`NMSE misses on this device/driver. reproduced on clean master with this change stashed and they remain.
Requirements
YES: AI was used in the beginning to understand the codebase and for easier search and navigation to different similar points. Code was all handwritten and then AI was used to review correctness and any gaps/oversights.