feat(sdk): add active-tool audio support#119
Conversation
|
Okay, I think this is an awesome foundation. Few things I'd like to note from a practicality and end-user ux standpoint (ignoring the foreground service stuff, which you folks are already working on lol): This PR would introduce an internal ExoPlayer instance managed entirely by the SDK. Which is great for 99.9% of use cases, but music streaming is a tricky business 😭 The nice thing is that media3 already has a pretty decent streaming pipeline. That's what the TIDAL half of phono uses (since TIDAL serves raw streaming URLs). I can definitely rework the Spotify half towards that. Currently we have a fully custom sink, since Spotify serves encrypted chunks. But it would absolutely be possible to decrypt, cache, etc., then feed directly to media3 as a custom source. The problem at the moment is that media3's proper streaming capabilities live in ExoPlayer.Builder, which is walled off. While it would be possible to just feed through some of these params as Light-denoted APIs, I think that's probably not the best long-term solution since devs may need more over time, and since it increases surface area for breaking changes from exoplayer itself I would ask that the SDK allow handoff of pre-configured ExoPlayer instances, instead of LightOS always making its own. Even if LightOS were to copy the config into a new ExoPlayer build, the cache and priority setup we would be using to properly stream decrypted ogg wouldn't come along. AudioAttributes, AudioSink, focus listening, pause/play/etc., would remain gated, but the tool can configure the input half (source factory, cache, priority, load control). SDK can own and reassert sink, attributes, focus, lifecycle after it adopts the player. I'm happy to prototype the handoff here. The TIDAL backend runs a fully configured media3 player already, so would prob be a low lift to work and post some proto-code here |
|
As an aside, it's really exciting to collaborate on the sdk with you guys. Community has been proposing more backends for phono, which could mean a mostly unified streaming client, which is pretty sick. lmk if you'd like me to make a PR w/ a rough prototype of how this could all work EDIT: realizing that the above two posts sound a little AI-ish. I wrote them myself I promise lol |
|
@jonathancaudill At the highest level, you're saying you could get what you need by allowing devs to inject an exoplayer instance/factory instead of using our default? I'd definitely be open to that as long as it doesn't make things any more complex for the 99.9% of cases. I'm gonna stack a PR on top of this with some kotlin'y stuff in just a bit here but after that, feel free to open your own. (we're not gonna wait to merge this but it should still be safe to work off of after my changes go in) Edit: my changes are in #123 - want to test them in the morning before I merge. |
Adds foreground audio support to the SDK client through a small, opinionated API built on Android's audio libraries:
LightAudiocreates the different audio components.LightAudioPlayerprovides queue-based playback for files, assets, and URLs usingandroidx.media3.LightAudioRecorderrecords microphone input to MPEG-4 files with AAC audio usingandroid.media.MediaRecorderLightAudioCaptureexposes real-time microphone input as aFlowof PCM buffers usingandroid.media.AudioRecordLightAudioVoiceplays short PCM buffers for synthesized sounds and samples usingandroid.media.AudioTrackandroid.media.AudioManager.android.permission.RECORD_AUDIOto tool permission allowlist.Persistent audio (that continues playing after the tool is left) support is not included in this first public surface. Since it requires a bit of additional work on LightOS, we'll first focus on getting the active-tool API stable.
The





audio-demoexample showcases the usage of the library:Other minor changes:
orientationfield to TOML config to define tool screen orientation