Heads up! I've decided to self-host my blog. I'm leaving this post live as to not "break the internet" but feel free check out the post on my new blog here!
This is part 11 in a series about building a browser-based audio live looper
- Part 12: v1.0 release, and project retro
- Part 11: Exporting stems and changing inputs
- Part 10: Keyboard bindings
- Part 9: Visualizing the waveform
- Part 8: Building and hosting
- Part 7: Latency and adding Track functionality
- Part 6: Workers and AudioWorklets
- Part 5: Record and loop a track
- Part 4: Adding a Scene
- Part 3: Metronome click
- Part 2: Adding a Metronome
- Part 1: New project: building a web-based audio looper!
- Add the ability to export stems
- Add the ability to change inputs per-track
Lots has happened since the last update! I found myself going down rabbit hole after rabbit hole. As I got the two main "goals" finished, I realized that the most important aspect of the app - recording audio - was still working very poorly. There was a bizarre attenuation effect on the audio recorded from my audio interface that made the app effectively unusable. Trying to fix this led me down several refactors that were ultimately unnecessary. Bizarrely, the fix was extremely simple and unrelated to any code I actually wrote. Rather, it was an implementation detail of the Web Audio API.
Another bug I noticed was that, even with the timing correction I made in #23, loops were experiencing noticeable drift over the course of several loops. I decided that the most pragmatic solution was to stop using the
loop parameter on the
AudioBufferSourceNode and simply restart the buffer on each loop start event. This seems to introduce a bit more clipping on loop start/end, but for the moment it's a worthwhile tradeoff for accurate timing.
In addition to fixing those two major bugs, I did add the ability to export stems, and also change inputs on a track. Both of these were important. Changing inputs is important because the "default device" is not always what a user might expect in a browser. And exporting audio is important because it's fun to have artifacts of what you record! I used another
Worker for the exporter, and copied some code from a Google project to write the WAV file blob, so it was pretty easy to implement. The hardest part was learning that an
AudioBuffer couldn't be serialized directly in a
Worker message, so I had to extract the
Float32Arrays from the channel data directly to export it. I'm not convinced this was worth it, because writing the WAV file is pretty fast and could probably happen on the UI thread, but it doesn't hurt anything and now it's done 🤷🏻. Adding the option to change inputs was also pretty straight forward. The Web Audio API provides an option to enumerate devices, so it was just a matter of setting the
MediaStream when the device changed.
There were lots of minor bug fixes and improvements, but those are the big updates to cover! I tried to keep notes in the PRs for each specific fix that I made.
- Audio constraints are important! As noted above, I was getting a really horrible attenuation modulation effect when recording tracks. I thought it was due to some weird React architecture but it turns out that specifying constraints on the user media request totally fixed it! Very strange, but I'm glad it was an easy solution
Workermessages cannot handle all data types.
AudioBufferscannot be passed in a
postMessagefunction, which surprised me. I imagine that most types can be serialized just fine, but it's good to know that certain native classes cannot be. Luckily, it's fairly easy to extract the channel data as raw
Float32Arrays, which work just fine as a
- Keyboard bindings deserve to be defined at the component level, rather than in a global context. Defining them globally introduces all sorts of weird edge cases around duplicate key handlers and weird behavior when mounting/unmounting components rapidly. Using a hook that can be called at the component level made this design much more straightforward.
- Be careful of extra renders! One of the rabbit holes I went down when trying to debug the recording quality issue was fixing extra renders in the
Trackcomponents. Originally, I nested the
Metronomecomponent. This is a fine choice in itself, but unfortunately, I was attaching the
currentTickproperty to an object that was passed to every child. This meant that every child was re-rendered on every beat. This is no good! Lot's of extra work being done. I realized there was only 1 piece of UI that actually needed to update on each beat: the
BeatCountercomponent. I moved the relevant event listener and state into that component, and it fixed the issue. I also wanted to de-couple the components of my app a bit more, so I placed
Sceneas a peer component of
Metronomeand just initialized the
Clockworker as a parent. This was a nice simplification and I think made the data flow a lot easier to follow.
- Looping in time is hard! I have no doubt that using scheduled events on the
AudioContextclock would fix this issue, but I still think that would make my life very challenging to deal with changes to the metronome settings. For my purposes, it seems that starting a new
AudioBufferSourceNodeon each loop start is the best way to achieve consistent timing with the metronome.
State of the app
- Merged PRs
- Most major features are working!
- Exporting stems
- Changing input per track
- Keyboard bindings
- Looping playback with reasonable accuracy
- Potentially improve garbage collection by disconnecting unused components
- A few minor UI improvements
- See if there is an easy way to record a performance