Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified public/authors/julian-m-kleber.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added public/posts/13-02-24-semester-end/banner.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added public/posts/13-02-24-voicebox/banner.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added public/posts/13-02-24-voicebox/ogImage.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
30 changes: 30 additions & 0 deletions src/pages/blog/13-02-2024-Voicebox.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
---
layout: ../../layouts/BlogLayout.astro
slug: 13-02-24-voicebox
title: Voicebox - Text-Guided Multilingual Universal Speech Generation at Scale
description:
Voicebox - Text-Guided Multilingual Universal Speech Generation at Scale
authors:
- Julian M Kleber
format: paper
tags:
- transcription
- audio
- reading group
- music
- sheet music
ogImage:
src: /posts/13-02-24-voicebox/ogImage.png
alt: Paper Image
banner:
src: /posts/13-02-24-voicebox/banner.png
alt: Paper Title
published: 02/13/2024
---

Large-scale generative models such as GPT and DALL-E have revolutionized the research community. These models not only generate high fidelity outputs, but are also generalists which can solve tasks not explicitly taught. In contrast, speech generative models are still primitive in terms of scale and task generalization. In this paper, we present Voicebox, the most versatile text-guided generative model for speech at scale. Voicebox is a non-autoregressive flow-matching model trained to infill speech, given audio context and text, trained on over 50K hours of speech that are not filtered or enhanced. Similar to GPT, Voicebox can perform many different tasks through in-context learning, but is more flexible as it can also condition on future context. Voicebox can be used for mono or cross-lingual zero-shot text-to-speech synthesis, noise removal, content editing, style conversion, and diverse sample generation. In particular, Voicebox outperforms the state-of-the-art zero-shot TTS model VALL-E on both intelligibility (5.9% vs 1.9% word error rates) and audio similarity (0.580 vs 0.681) while being up to 20 times faster.

Link to paper: [https://arxiv.org/abs/2306.15687](https://arxiv.org/abs/2306.15687

[Banner](https://arxiv.org/abs/2111.03017)
[Picture](https://vectorportal.com/de/vector/jukebox-machine.ai/15161)
28 changes: 28 additions & 0 deletions src/pages/blog/13-02-2024-semester-end.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
layout: ../../layouts/BlogLayout.astro
slug: 13-02-24-semester-end
title: Community Event - BLISS in SS24 - feedback phase and plans for the next semester
description:
Community Event - BLISS in SS24 - feedback phase and plans for the next semester
authors:
- Julian M Kleber
format: event
tags:
- BLISS
- organization
- Community Event
ogImage:
src: /posts/13-02-24-semester-end/banner.jpg
alt: Paper Image
banner:
src: /posts/13-02-24-semester-end/banner.jpg
alt: Paper Title
published: 02/13/2024
---

During the event we will do a brainstorming session about possible future directions of bliss, and planning of the lecture free period.

Link to paper: [https://arxiv.org/abs/2306.15687](https://arxiv.org/abs/2306.15687

[Banner](https://commons.wikimedia.org/wiki/File:Das_letzte_Abendmahl,_mit_Liniendivergenz_der_Perspektive_und_Versatz_in_der_R%C3%BCckwand.jpg)
[Picture](https://commons.wikimedia.org/wiki/File:Das_letzte_Abendmahl,_mit_Liniendivergenz_der_Perspektive_und_Versatz_in_der_R%C3%BCckwand.jpg)