remove hardcoded device arguments for device agnostic support

[chinyixiang]

Related Issues

• N/A (small portability fix)

Proposed Changes

Several cookbook notebooks hardcode the device when creating embedders and transcribers. I noticed Haystack's _get_default_device() already resolves the device automatically (cuda > xpu > mps > cpu). Hardcoding device is redundant and will cause issues when running on non-cuda devices:

device=ComponentDevice.from_str("cuda:0")

On any machine where PyTorch is not built with CUDA, these examples crash at warm_up() before producing a result:

• SentenceTransformers embedders → AssertionError: Torch not compiled with CUDA enabled
• LocalWhisperTranscriber → RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False

This PR removes the hardcoded device=... argument so the components fall back to automatic device resolution. On a GPU machine the GPU is still selected; on CPU-only / non-CUDA machines the examples now run instead of crashing. No other behavior changes.

Notebook	Occurrences removed
multilingual_rag_podcast.ipynb	3
llama3_rag.ipynb	2
zephyr-7b-beta-for-rag.ipynb	2
improve-retrieval-by-embedding-metadata.ipynb	2
using_speaker_diarization_with_assemblyai.ipynb	2
prometheus2_evaluation.ipynb	1

6 files changed, 12 hardcoded device arguments removed.

Tested On

Test environment:

• GPU: Intel Arc Pro B60 (XPU), torch.cuda.is_available() == False

• PyTorch 2.12.1+xpu, Python 3.12

• Haystack core 2.31.0rc0

• Before (original code): reproduced AssertionError: Torch not compiled with CUDA enabled.

• After (device argument removed): each affected component was constructed with the exact model each notebook uses and confirmed to load + run on the auto-resolved device (xpu:0 on this box):

  Component	Models tested
  SentenceTransformersDocumentEmbedder / SentenceTransformersTextEmbedder	BAAI/bge-small-en-v1.5, sentence-transformers/all-MiniLM-L6-v2, Snowflake/snowflake-arctic-embed-l, thenlper/gte-large, intfloat/multilingual-e5-large
  LocalWhisperTranscriber	whisper-small

  All resolved to the available device and produced correct output. Notebooks re-validated as valid JSON / nbformat; edited code cells parse cleanly.

Notes for the reviewer

• Pure portability fix — no new dependencies, no API changes, GPU behavior unchanged when a GPU is present.
• The same device="cuda:0" pattern appears in haystack-tutorials (33_Hybrid_Retrieval.ipynb); happy to open a matching PR there if useful.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[Copilot]

Pull request overview

• Improves notebook portability by removing hardcoded device=ComponentDevice.from_str("cuda:0") so Haystack components fall back to automatic device resolution across CUDA/XPU/MPS/CPU.

Changes:

• Removed hardcoded device=... arguments from SentenceTransformers embedders and a Transformers generator across affected notebooks.
• Simplified a commented LocalWhisperTranscriber example to avoid forcing CUDA in the sample snippet.
• Minor notebook code-cell edits to keep cells syntactically valid after argument removal.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
notebooks/multilingual_rag_podcast.ipynb	Removes hardcoded device usage in embedder cells and updates commented Whisper snippet.
notebooks/llama3_rag.ipynb	Removes hardcoded device usage for document/text embedders.
notebooks/zephyr-7b-beta-for-rag.ipynb	Removes hardcoded device usage for document/text embedders.
notebooks/improve-retrieval-by-embedding-metadata.ipynb	Removes hardcoded device usage in embedder setup for indexing/retrieval.
notebooks/using_speaker_diarization_with_assemblyai.ipynb	Removes hardcoded device usage for speaker/text embedders.
notebooks/prometheus2_evaluation.ipynb	Removes hardcoded device usage for TransformersChatGenerator.

Suppressed comments (5)

notebooks/zephyr-7b-beta-for-rag.ipynb:1038

• After removing the device argument, this call now has a dangling trailing comma + an empty continuation line, which reads like an incomplete argument list. Consider collapsing it to a single clean line.

        "rag.add_component(\"text_embedder\", SentenceTransformersTextEmbedder(model=\"thenlper/gte-large\", \n",
        "                                                                    ))\n",

notebooks/zephyr-7b-beta-for-rag.ipynb:189

• ComponentDevice is now imported but no longer used anywhere in this notebook after removing the hardcoded device arguments. Consider removing the import to avoid confusion.

        "indexing.add_component(\"doc_embedder\", SentenceTransformersDocumentEmbedder(model=\"thenlper/gte-large\",\n",
        "                                                                            meta_fields_to_embed=[\"title\"]))\n",

notebooks/improve-retrieval-by-embedding-metadata.ipynb:171

• After removing the device argument, this call now has a dangling trailing comma + an empty continuation line, which looks like an accidental formatting artifact. Collapse it to a single line (or remove the trailing comma).

        "  retrieval.add_component(\"text_embedder\", SentenceTransformersTextEmbedder(model=\"thenlper/gte-large\",\n",
        "                                                                            ))\n",

notebooks/using_speaker_diarization_with_assemblyai.ipynb:612

• ComponentDevice is now imported but no longer used in this cell after removing the hardcoded device argument. Consider dropping the unused import.

        "Answer:\n",
        "\"\"\"\n",
        "\n",
        "retriever = InMemoryEmbeddingRetriever(speaker_document_store)\n",
        "text_embedder = SentenceTransformersTextEmbedder()\n",

notebooks/multilingual_rag_podcast.ipynb:218

• ComponentDevice is now imported but no longer used anywhere in this notebook after removing the hardcoded device arguments. Consider removing the unused import.

        "    \"embedder\",\n",
        "    SentenceTransformersDocumentEmbedder(\n",
        "        model=\"intfloat/multilingual-e5-large\",  # good multilingual model: https://huggingface.co/intfloat/multilingual-e5-large\n",
        "        prefix=\"passage:\",  # as explained in the model card (https://huggingface.co/intfloat/multilingual-e5-large#faq), documents should be prefixed with \"passage:\"\n",
        "    ))\n",

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[kacperlukawski]

Thank you, @chinyixiang Copilot has pointed out some issues remaining after making these changes. Would you be able to have another look?

[chinyixiang]

Attended to all comments pointed out by copilot. Removed redundant imports and edited explanation to give example instead referring to the code.

[chinyixiang]

Hi
I noticed that the repository does not appear to contain a LICENSE file.

I understand that contributors are required to sign the project's CLA. Could you please clarify:

What license currently applies (or is intended to apply) to this repository?
Under what license are accepted contributions distributed?
If a license has not yet been formally added to the repository, could you provide a written statement regarding the intended licensing model?

Thank you for your assistance and clarification.