Skip to content

z-image support npu#12979

Open
luren55 wants to merge 27 commits intohuggingface:mainfrom
luren55:main
Open

z-image support npu#12979
luren55 wants to merge 27 commits intohuggingface:mainfrom
luren55:main

Conversation

@luren55
Copy link
Copy Markdown

@luren55 luren55 commented Jan 15, 2026

What does this PR do?

Issue: The Diffusers Z-Image pipeline fails on Ascend NPU because aclnnIndex lacks support for complex64. When freqs_cis is stored as a complex tensor on the NPU and then indexed, the call crashes.

Fix: Rework the RoPE frequency handling so that the NPU never needs to index complex64 tensors.

Running Environment:
I run this model base on cache-dit(https://github.com/vipshop/cache-dit/) which depends on diffusers to inference.

Command:
python3 generate.py zimage --model-path /home/weights/Z-Image-Turbo --attn _native_npu

Error before fixed:
image

Result after fixed:
image

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@luren55
Copy link
Copy Markdown
Author

luren55 commented Jan 16, 2026

@yiyuxuxu Hello, could you please review this pr, thank you.

@luren55
Copy link
Copy Markdown
Author

luren55 commented Jan 17, 2026

@yiyixuxu @sayakpaul Hello, could you please review this pr, thank you.

@sayakpaul sayakpaul requested a review from yiyixuxu January 17, 2026 05:14
return x


class RopeEmbedderNPU:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it not possible to modify the existing RopeEmbedder class to account for the NPU hardware?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we could modify the current RopeEmbedder class same as RopeEmbedderNPU?It could work for both device

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No I mean we keep the class name as RopeEmbedder and modify the definition such that it also works on NPU alongside current devices.

@luren55
Copy link
Copy Markdown
Author

luren55 commented Jan 26, 2026

@sayakpaul I merged RopeEmbedderNPU into RopeEmbedder class, please review the modification

@luren55 luren55 requested a review from sayakpaul January 27, 2026 23:56
Copy link
Copy Markdown
Member

@sayakpaul sayakpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@sayakpaul
Copy link
Copy Markdown
Member

@bot /style

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jan 28, 2026

Style bot fixed some files and pushed the changes.

@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@luren55
Copy link
Copy Markdown
Author

luren55 commented Feb 5, 2026

@sayakpaul can you merge this pr? I have verified both gpu and npu case, they can both work. thanks

@github-actions github-actions Bot added models size/M PR with diff < 200 LOC labels Apr 29, 2026
@github-actions github-actions Bot added the CI label Apr 29, 2026
@github-actions github-actions Bot added size/L PR with diff > 200 LOC and removed size/M PR with diff < 200 LOC labels Apr 29, 2026
@github-actions github-actions Bot added size/L PR with diff > 200 LOC and removed size/L PR with diff > 200 LOC labels May 6, 2026
@github-actions github-actions Bot added size/L PR with diff > 200 LOC and removed size/L PR with diff > 200 LOC labels May 6, 2026
@github-actions github-actions Bot added size/L PR with diff > 200 LOC and removed size/L PR with diff > 200 LOC labels May 6, 2026
@github-actions github-actions Bot added size/L PR with diff > 200 LOC and removed size/L PR with diff > 200 LOC labels May 6, 2026
@github-actions github-actions Bot added size/L PR with diff > 200 LOC and removed size/L PR with diff > 200 LOC labels May 6, 2026
@github-actions github-actions Bot added size/L PR with diff > 200 LOC and removed size/L PR with diff > 200 LOC labels May 6, 2026
@github-actions github-actions Bot added size/L PR with diff > 200 LOC and removed size/L PR with diff > 200 LOC labels May 6, 2026
@yiyixuxu
Copy link
Copy Markdown
Collaborator

yiyixuxu commented May 6, 2026

@luren55
sorry I did not merge this earlier
I'm ok with changes in the transformer file, but what's going on with other changes? can you revert them?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI models size/L PR with diff > 200 LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants