Controlling Web Audio With React and Redux Middleware


Let’s Build a TouchTone Keypad!

If you’ve built React/Redux applications before, you know there is a standard pattern of uni-directional data flow. The UI dispatches an action. A reducer handles the action, returning a new application state. The UI reorganizes itself accordingly.

But what if you need a Redux action to trigger interaction with a complex system? Say, a collection of Web Audio components used to create or analyze sound. Those are not serializable objects. They shouldn’t be managed by a reducer. Nor should a UI component manage them because it could be subject to unmounting at runtime, causing a loss of the audio system.

Instead, we use middleware, and that is the focus of this article.

Middleware is long-lived, and can therefore be expected to maintain whatever audio system we construct for the life of the application. Furthermore, it has access to the store, so it can inspect the application state, dispatch actions in response to events from its charge, e.g., a Web Audio system, and can respond to actions that direct it to interact with such a system.

In this demo, we will keep it simple and just trigger some sounds in response to an action. The goal is simply to demonstrate how to use middleware to adapt the Web Audio API system to a Redux application.

To celebrate the fact that I just this month ditched my landline after 12 years, (porting the number to the awesome openphone.co online service), we will simulate a TouchTone telephone keypad.

Classy pic of nutty old phone by Joe Haupt on Flickr

Researching the Problem Domain

DTMF – The Magic Frequencies

TouchTone telephones use a system called DTMF (Dual-tone multi-frequency) signaling, which triggers two separate frequencies when a key is pressed.

The linked Wikipedia article contains lots more trivia, like the fact that the ‘#’ symbol was called an “octothorpe” by the original engineers. But, for the most part, all we need to know is contained in the one table you see here.

In our application, two oscillators in a small Web Audio API system can generate those frequencies, and we’ll use a Redux middleware function to act as the go-between.

Building the App

First, we just need to encode this table of magic frequencies in such a way that we can easily create a keypad that uses that information.

The brute force method would be to declare each button separately and hard-code the appropriate frequencies in each button’s click handler. The optimal way, however, would be to arrange the data so that we can actually generate the keypad from it. Needless to say, the latter is the choice we’ll use here.

Representing the Domain

We define the row frequencies and column frequencies first, then create an array for each key with the frequency constants for its row and column position. Finally, an array of arrays representing keypad rows is built, with each key being represented by an array containing a label and the array of tones for that key.

dtmf.js

dtmf.js

UI to Middleware Messaging

Before we get into either the UI for representing the keypad or the middleware for playing the DTMF tones, let’s have a quick look at the message that will be sent between the two.

The action creator playDTMFPair will accept a pair of tones as defined in the KEY_ constants above, and return an action of type PLAY_DTMF_PAIR via which the tones can be dispatched from the UI to the middleware each time a key is pressed.

actions.js

actions.js

Creating the UI

Beep boop boop — beep beep boop beep

The demo is a standard React/Redux setup.

Additionally, it uses React Bootstrap and Styled Components to achieve the look and feel of a typical TouchTone keypad with big, square, shaded buttons arranged in tight rows.

You can review the project code for the styling aspects, but below are the two main React components used to render the keypad.

The App component is, as usual, the main container. In its render method, it creates a StyledKeypad which is basically a columnar flexbox with content centering and some upper margin. Inside that, it renders a StyledKeypadRow container for each row of the keypad (you guessed it, a row-oriented flexbox). Finally, inside each of those, it renders a KeypadKey component for every key in the row, passing a label, the tones that the key needs to trigger, and a dispatcher for the action.

App.js

App.js

The KeypadKey component is a simple functional component which accepts the label, tones, and handleClick function we pass as props. It returns a StyledKeypadButton, which is just a big square Bootstrap button with no outline, a readable font size, and an onClick handler that calls the handleClick function, passing the tones array.

KeypadKey.js

KeypadKey.js

The Web Audio API

Complete polyphonic synthesizers have been built using the Web Audio API, such is the awesome breadth of its implementation. And choosing React/Redux for your overall application framework is a great way to start such a project. While our demo will be piddling in comparison, the architecture could easily be adapted to such a grand purpose.

If you’re new to it, a fantastic introduction to the Web Audio API is available at css-tricks. In fact, a section of it forms the basis for our TouchTone class. The main differences are that we get the audio context in the constructor rather than accepting it as an argument, we create two oscillators instead of just one, we start the sound immediately rather than accepting a time, and we turn the sound off immediately after half a second rather than ramping off exponentially.

Since our focus is on adapting an audio system to Redux, I’ll let the css-tricks article describe the particulars of the Web Audio API touched on here.

TouchTone.js

TouchTone.js

The Redux Middleware

When you first read the official introduction to Redux middleware it can easily twist your melon. This is because they go through a whole bunch of “wrong ways” to do things before arriving at their solution. That’s why I thought it would be nice to have a dead simple example to get you started.

It’s really not that complicated at all. A simple function accepts the Redux store and returns a function that accepts the next piece of middleware in the chain’s callback. That callback accepts an action. Inside that function we can handle or ignore any action passing through, but when we’re done, we need to call the next piece of middleware’s callback, passing it the action.

The key things to remember are that it’s going to be around for the life of the application, and it will be given a chance to handle any dispatched action. This makes it a perfect mediator for interaction with non-serializable parts of our applications like sockets or audio components. It will have access to the store, so you can refer to the state and you can dispatch actions from it if you need to. In our case, we’re only going to respond to a single action dispatched by the keypad.

middleware.js

middleware.js

Conclusion

Obviously, with a different interface and audio system, this application could implement a musical device, triggering much more pleasing tones than DTMF. The main focus was on how to adapt such a system.

Using middleware to control the Web Audio API is pretty easy and architecturally the right way to go in a React/Redux application. So don’t let the convoluted documentation on the Redux site put you off if you’ve never built a piece of middleware.

More ambitious things that can happen in the middleware are that it could respond to events from an AudioListener or ScriptProcessorNode and dispatch actions by calling store.dispatch(). This would allow the app to perform audio spatialization or visualization.

You can download the project from GitHub: https://github.com/cliffhall/react-dtmf-dialer

3 thoughts on “Controlling Web Audio With React and Redux Middleware”

  1. I originally saw this on some kind of dev tutorial web site, left a comment there, then had to doublecheck something when I built something myself with this architecture, so I found this (I think original) article. Not sure if you’ll see the other so I just want to say again how much I appreciate this! It’s very useful to me, I think I’ll probably use this approach quite a bit (in personal projects). And it’s illuminating on the workings of middleware, too. Thanks again!
    One question if you see this and have a minute. I think I’m understanding/seeing that the touchTone is initialized just once? Which I think means that audioMiddleWare is a function that constructs the middleware, which is why it also runs only on intitialization? best wishes.

    1. Yes, the audioMiddleWare function returns a function that takes a store and returns its own function which returns a function. That function is long lived and used as needed be by Redux.

      Saying all this now, trying to explain it, I am reminded of what an overly clever mechanism it is. It serves, but when you get into functions that return functions that return functions ad nauseum, it makes it really hard to reason about. But it’s just the standard pattern for doing middleware with Redux. While you’re knee-deep in its implementation, you can get your head around it, or at least get the pattern-recognition of the idiom working in your favor, but then walk away from it for 6 months doing no other middleware work and when you come back, it pokes you in the eye.

Leave a Reply to David Sabine Cancel reply

Your email address will not be published. Required fields are marked *