This library is built for FlutterFlow users who want to add natural, speech-to-speech AI conversations to their app without exposing secret keys in the client. It connects your FlutterFlow project to ElevenLabs Conversational AI v2 using the official ElevenLabs Flutter SDK under the hood, while keeping the setup focused on what you can actually configure inside the FlutterFlow interface.
This README is written for a FlutterFlow builder who is:
- installing the library from the FlutterFlow Marketplace
- configuring values in App State, API Calls, page actions, and widgets
- avoiding direct code edits unless FlutterFlow explicitly asks for platform configuration
If you are using this library in FlutterFlow, the supported install path is the Marketplace import flow.
This library is designed so your ElevenLabs API key never lives in FlutterFlow App State, page state, custom actions, or client-side code.
- You create an agent in ElevenLabs
- You store your ElevenLabs API key in a secure backend service such as BuildShip
- Your FlutterFlow app calls that backend to request a temporary conversation token
- The library uses that token to start a secure real-time conversation
- Audio streams through WebRTC without exposing your secret key in the app
- Your API key is not shipped in the app
- Your FlutterFlow project remains safe to share with collaborators
- You can rotate backend secrets without changing app screens
- Tokens are short-lived and safer than using a raw API key on-device
- Real-time speech-to-speech AI conversations
- Low-latency streaming audio powered by WebRTC
- Interruptions and turn-taking for more natural conversations
- Transcripts and conversation state exposed to FlutterFlow
- Automatic reconnect behavior for unstable connections
- Playback-aware recording behavior to reduce audio feedback
- Custom widget for recording interaction
- Custom actions to initialize and stop the session
- App State integration so your UI can react visually
- Theme-aware behavior for FlutterFlow UI styling
- Permission handling support for microphone and audio routing
- Open the library in the FlutterFlow Marketplace: ElevenLabs Conversational AI v2
- Click Add to Project
- Complete the FlutterFlow import flow
- Wait for FlutterFlow to finish syncing dependencies
That is the installation path FlutterFlow users should follow.
Before configuring the library in FlutterFlow, make sure you have:
- an ElevenLabs account
- an ElevenLabs Agent ID
- a secure backend endpoint that returns a conversation token
- a FlutterFlow project with the marketplace library added
The easiest low-code pattern is:
- FlutterFlow → sends request to your backend
- BuildShip → uses your ElevenLabs API key securely
- BuildShip → returns a temporary token response (
token, optionally withsignedUrl) - FlutterFlow → passes your agent ID and backend endpoint into the library
- The library → fetches the token and starts the conversation internally
In ElevenLabs:
- Open Conversational AI
- Create or select your agent
- Copy the Agent ID
You will store this value in FlutterFlow App State later.
Use a backend platform such as BuildShip to create an endpoint that:
- accepts your
agentId - uses your secret ElevenLabs API key on the server
- returns a temporary conversation token payload the library can read
The current library implementation accepts either of these backend response shapes:
{
"token": "your-temporary-token"
}or:
{
"signedUrl": "wss://api.elevenlabs.io/...",
"token": "your-temporary-token"
}token is the field the current implementation uses for the ElevenLabs SDK session.
signedUrl may also be present for compatibility/debugging, but it is not required when token is returned.
Do not put your ElevenLabs API key into:
- FlutterFlow App State
- page parameters
- custom action inputs
- API call headers inside FlutterFlow
- local storage on the device
Only the backend should know the API key.
In FlutterFlow, go to App State and add these variables.
| Variable | Type | Default | Purpose |
|---|---|---|---|
wsConnectionState |
String | disconnected |
Tracks connection status |
elevenLabsAgentId |
String | empty | Stores your ElevenLabs Agent ID |
elevenLabsConversationTokenEndpoint |
String | empty | Stores your backend token endpoint URL |
isRecording |
bool | false |
Tracks whether the user is currently recording |
conversationMessages |
List < JSON > or List < dynamic > | empty list | Stores transcript/message history |
isAgentSpeaking |
bool | false |
Tracks whether the AI is currently speaking |
isInitializing |
bool | false |
Tracks initialization state |
- Set
elevenLabsAgentIdto your real ElevenLabs Agent ID - Set
elevenLabsConversationTokenEndpointto your deployed backend URL
For the current library implementation, you do not need to create a separate FlutterFlow API Call or manually map a token response into the initialize action.
Instead:
- store your backend URL in
FFAppState().elevenLabsConversationTokenEndpoint - store your agent ID in
FFAppState().elevenLabsAgentId - let
initializeConversationServicecall the backend for you
The initialize flow makes a POST request to your configured endpoint with this JSON body:
{
"agentId": "[your agent id from App State]"
}Return one of the supported payloads shown above, including a token field.
If you want to test your backend manually in FlutterFlow or Postman, use the same request body and confirm the response includes token.
This library needs microphone access and may also rely on Bluetooth/audio routing behavior depending on the device.
Make sure your generated Android app includes:
RECORD_AUDIOINTERNETMODIFY_AUDIO_SETTINGSACCESS_NETWORK_STATE- Bluetooth permissions if your audio path depends on them
- foreground audio service support if your app uses ongoing conversation flows
Make sure your generated iOS app includes:
- microphone usage description
- Bluetooth usage description if applicable
- audio background modes if your use case requires it
If FlutterFlow exposes these through project settings, configure them there. If your project requires platform-specific config outside the UI, use the generated platform settings carefully after export.
On the page where the conversation should start, configure your action flow in FlutterFlow.
- Populate App State with your
elevenLabsAgentIdandelevenLabsConversationTokenEndpoint - Request microphone permission (recommended for clearer UX)
- Run the custom action to initialize the conversation
- If initialization fails, surface the returned error string in the UI
- Action 1: Ensure App State contains
elevenLabsAgentIdandelevenLabsConversationTokenEndpoint - Action 2: Request microphone permission
- Action 3: Run
initializeConversationService - Action 4: If initialization fails, show an error message or snackbar
The initialize action should receive:
- current page
context - your
agentId - your backend
endpoint
The library will fetch the token internally.
Place the included recording widget on your page.
- gives the user a clear talk button
- reacts to conversation state
- helps prevent recording while the agent is already speaking
- supports a cleaner low-code setup than manually wiring recording state yourself
You can place it inside:
- a floating action area
- a chat composer area
- a bottom sheet
- a persistent footer on a conversation screen
Use a ListView or repeating UI pattern in FlutterFlow and bind it to:
FFAppState().conversationMessages
Recommended display ideas:
- user messages on one side
- agent messages on the other side
- loading indicator when
isInitializingis true - speaking indicator when
isAgentSpeakingis true - connection status badge using
wsConnectionState
On Page Dispose or when the user exits the conversation screen:
- run
stopConversationService
This helps avoid leaving audio sessions active after navigation.
If you are testing iOS voice input, use a physical iPhone.
The underlying WebRTC stack used by the ElevenLabs SDK does not reliably capture microphone input in iOS Simulator.
That means:
- your UI may load correctly
- your connection may still initialize
- but voice capture and transcript behavior may fail or show incomplete results
For final validation, always test on a real device.
- keep your API key on the backend only
- store only the Agent ID and token endpoint URL in App State
- use FlutterFlow API Calls for the token request
- show UI feedback for connecting, speaking, recording, and errors
- test microphone permissions on both Android and iOS
- do not paste your ElevenLabs API key into FlutterFlow
- do not hardcode secrets into custom actions
- do not rely on simulator testing for final audio validation
- do not skip the backend token step
FlutterFlow projects include internal package constraints that are not always compatible with the default dependency chain used by the public ElevenLabs Flutter SDK.
This library includes the compatibility work needed to make the integration function cleanly in a FlutterFlow environment.
As a FlutterFlow user, you do not need to manually manage those package-level fixes through the FlutterFlow interface. The important part for you is:
- install from the Marketplace
- configure App State
- set up your token API call
- wire the provided actions and widget into your page
| Problem | What to check in FlutterFlow |
|---|---|
| Conversation does not start | Confirm your backend endpoint returns a valid token and that elevenLabsConversationTokenEndpoint points to the correct URL |
| Permission error | Confirm microphone permissions are enabled in project settings and granted on device |
| No transcript on iPhone simulator | Test on a physical device |
| Agent does not respond | Confirm elevenLabsAgentId is correct |
| UI never leaves loading state | Check whether isInitializing is being reset and whether the token call succeeded |
| Connection drops | Check network conditions and show state from wsConnectionState in the UI |
Before you test, confirm all of the following:
- Marketplace library added to your FlutterFlow project
- ElevenLabs Agent created
- backend token endpoint deployed
elevenLabsAgentIdadded to App StateelevenLabsConversationTokenEndpointadded to App State- page action flow wired for App State → permission → initialize
- recording widget added to the page
stopConversationServiceadded on exit/dispose- tested on a real mobile device
Ready to build voice-native experiences in FlutterFlow? Install the Library from Marketplace and get conversing!