feat: add AI generation language preferences#1839
Conversation
superagent-security
Bot
added
contributor:verified
![tembo[bot]](https://avatars.githubusercontent.com/in/1223475?s=60&v=4){kind=small}
There was a problem hiding this comment.
Since transcription language can now be forced, it might help to include the selected language in the Deepgram error message (makes unsupported/invalid language codes easier to debug).
| throw new Error( | |
| `Deepgram transcription failed (language=${language}): ${error.message}`, | |
| ); |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| export function getDeepgramTranscriptionOptions( | ||
| language: AiGenerationLanguage, | ||
| ) { | ||
| const baseOptions = { | ||
| model: "nova-3", | ||
| smart_format: true, | ||
| utterances: true, | ||
| mime_type: "audio/mpeg", | ||
| } as const; | ||
|
|
||
| if (language === AI_GENERATION_LANGUAGE_AUTO) { | ||
| return { | ||
| ...baseOptions, | ||
| detect_language: true, | ||
| }; | ||
| } | ||
|
|
||
| return { | ||
| ...baseOptions, | ||
| language, | ||
| }; | ||
| } |
There was a problem hiding this comment.
SUPPORTED_LANGUAGES was originally built for a translation API and includes codes that Deepgram Nova-3 does not accept — among them bn, gu, pa, mr, te, fa, he, ur, and possibly sk, ar, zh (Deepgram expects zh-CN, not bare zh). When a Pro user selects one of these, transcribeWithDeepgram passes the bare code to Deepgram, which returns an error. That error is rethrown as an exception inside a "use step" — but by then validateVideo has already set transcriptionStatus = "PROCESSING" with no recovery path, so every subsequent video for that org will be stuck in PROCESSING indefinitely.
The two fix options are: (a) define a separate DEEPGRAM_SUPPORTED_LANGUAGES allowlist that only contains codes Deepgram Nova-3 accepts, and use that for the language picker; or (b) add a mapping layer that converts zh → zh-CN and rejects codes with no Deepgram equivalent before calling the API.
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/web/workflows/transcribe.ts
Line: 288-309
Comment:
**Unsupported language codes will break all transcriptions for an org**
`SUPPORTED_LANGUAGES` was originally built for a translation API and includes codes that Deepgram Nova-3 does not accept — among them `bn`, `gu`, `pa`, `mr`, `te`, `fa`, `he`, `ur`, and possibly `sk`, `ar`, `zh` (Deepgram expects `zh-CN`, not bare `zh`). When a Pro user selects one of these, `transcribeWithDeepgram` passes the bare code to Deepgram, which returns an error. That error is rethrown as an exception inside a `"use step"` — but by then `validateVideo` has already set `transcriptionStatus = "PROCESSING"` with no recovery path, so every subsequent video for that org will be stuck in PROCESSING indefinitely.
The two fix options are: (a) define a separate `DEEPGRAM_SUPPORTED_LANGUAGES` allowlist that only contains codes Deepgram Nova-3 accepts, and use that for the language picker; or (b) add a mapping layer that converts `zh → zh-CN` and rejects codes with no Deepgram equivalent before calling the API.
How can I resolve this? If you propose a fix, please make it concise.| export function getDeepgramTranscriptionOptions( | ||
| language: AiGenerationLanguage, | ||
| ) { | ||
| const baseOptions = { | ||
| model: "nova-3", | ||
| smart_format: true, | ||
| utterances: true, | ||
| mime_type: "audio/mpeg", | ||
| } as const; | ||
|
|
||
| if (language === AI_GENERATION_LANGUAGE_AUTO) { | ||
| return { | ||
| ...baseOptions, | ||
| detect_language: true, | ||
| }; | ||
| } | ||
|
|
||
| return { | ||
| ...baseOptions, | ||
| language, | ||
| }; | ||
| } |
There was a problem hiding this comment.
nova-3 + detect_language combination
When language === "auto", the code sends detect_language: true to Deepgram Nova-3. Deepgram's own documentation notes that language detection queries against "35 supported languages", but the Nova-3 model page lists a much smaller subset. If detection returns a language code that Nova-3 can't decode (which can happen with less common audio), the result may silently produce a poor-quality transcript. Consider adding detect_language=en&detect_language=es&... with only the languages Nova-3 supports to constrain the detection pool, instead of relying on unconstrained auto-detect.
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/web/workflows/transcribe.ts
Line: 288-309
Comment:
**`nova-3` + `detect_language` combination**
When `language === "auto"`, the code sends `detect_language: true` to Deepgram Nova-3. Deepgram's own documentation notes that language detection queries against "35 supported languages", but the Nova-3 model page lists a much smaller subset. If detection returns a language code that Nova-3 can't decode (which can happen with less common audio), the result may silently produce a poor-quality transcript. Consider adding `detect_language=en&detect_language=es&...` with only the languages Nova-3 supports to constrain the detection pool, instead of relying on unconstrained auto-detect.
How can I resolve this? If you propose a fix, please make it concise.
richiemcilroy
force-pushed
the
ai-generation-language-preferences
branch
from
7d65afa to
7fc140a
Compare
|
please re-review the pr @greptileai |
|
|
||
| return { | ||
| ...baseOptions, | ||
| language, |
There was a problem hiding this comment.
getDeepgramTranscriptionOptions passes any non-"auto" org setting straight through as language. Since AiGenerationLanguage currently includes codes outside DEEPGRAM_DETECTABLE_LANGUAGES (and some APIs expect mapped codes like zh-CN), this can turn into a hard failure for every transcription in that org.
If DEEPGRAM_DETECTABLE_LANGUAGES is effectively the allowlist for nova-3, consider guarding here (or constraining the picker) so unsupported codes fall back to auto-detect instead of erroring:
| language, | |
| if ( | |
| language === AI_GENERATION_LANGUAGE_AUTO || | |
| !DEEPGRAM_DETECTABLE_LANGUAGES.includes(language) | |
| ) { | |
| return { | |
| ...baseOptions, | |
| detect_language: [...DEEPGRAM_DETECTABLE_LANGUAGES], | |
| }; | |
| } | |
| return { | |
| ...baseOptions, | |
| language, | |
| }; |
1 participant
Summary
Validation
pnpm exec biome check --write packages/web-domain/src/Language.ts packages/web-domain/src/index.ts packages/database/schema.ts apps/web/actions/videos/translation-languages.ts apps/web/actions/organization/settings.ts apps/web/app/(org)/dashboard/settings/organization/components/CapSettingsCard.tsx apps/web/app/s/[videoId]/Share.tsx apps/web/workflows/transcribe.ts apps/web/workflows/generate-ai.ts apps/web/__tests__/unit/generate-ai-title.test.tspnpm --dir apps/web exec next typegen && pnpm exec tsc -b packages/web-domain packages/database apps/webpnpm --dir apps/web test __tests__/unit/generate-ai-title.test.ts __tests__/integration/transcribe.test.tsGreptile Summary
This PR adds a per-organization AI generation language preference that flows through Deepgram transcription and all Groq AI prompt paths (titles, summaries, chapters, key points). It also addresses two prior review concerns: a
markErrorrecovery path prevents videos from being stuck inPROCESSINGwhen transcription fails, anddetect_languageis now constrained to a curated list of Nova-3-supported codes rather than the unboundeddetect_language: true.packages/web-domain/src/Language.ts— new module centralisesSUPPORTED_LANGUAGES, defines the narrowerAI_GENERATION_LANGUAGE_CODESset (Punjabi excluded), and exports type guards/parse helpers used across the stack.apps/web/workflows/transcribe.ts— wraps the transcription+save steps in a try/catch that callsmarkErroron failure;getDeepgramTranscriptionOptionsis extracted as a pure, testable function and selects eitherlanguageor a constraineddetect_languagearray.apps/web/workflows/generate-ai.ts— joinsorganizationsto readaiGenerationLanguage, derives alanguageInstructionstring injected into every AI prompt, and adds a "Keep JSON property names exactly as shown" guard to prevent key translation.Confidence Score: 5/5
Safe to merge — the new error-recovery path, constrained detect_language list, and type-narrowed viewer settings are all well-implemented with no new correctness issues introduced.
The changes are incremental and well-scoped: language constants are centralised, the AI workflows gain a pure options-builder that is unit-tested, and the transcription workflow now correctly marks videos as ERROR instead of leaving them in PROCESSING on failure. The two inline suggestions are minor edge-case robustness improvements that do not affect the happy path.
apps/web/workflows/transcribe.ts — the placement of markNoAudio inside the try block and the unguarded cleanupTempAudio call in the catch are worth a quick look.
Important Files Changed
Prompt To Fix All With AI
Reviews (2): Last reviewed commit: "feat: add AI generation language prefere..." | Re-trigger Greptile