My stepson, Ben, has recently begun streaming on Twitch.¹ He was wondering what kind of customization he could do on his streams, and I pointed out that I happen to be a programming expert² who could help him out.

We ended up building commands that viewers could enter in the Twitch chat to play specific sounds on the stream. In this post, I’ll show you how we built that using tmi.js, Glitch, and OBS Studio.

A whole web browser at our fingertips

I already knew from my own streaming experience³ that OBS, the popular streaming software, can capture a web page and combine it with the rest of your stream. OBS works by defining “scenes” that consist of a number of “sources”. Typical sources include display capture and webcams, but there’s also the browser source.

Browser source uses the Chromium Embedded Framework (CEF) to render a URL. Custom CSS sets the page background to transparent, and the whole thing is composited with the rest of your OBS scene. This makes it easy to draw custom overlays, play animations, etc.

We decided to use this to play our sound effects.

Glitch: beginner-friendly hosting

I had never tried Glitch before, but it had a few features I wanted that would make our development process easier:

1. It has a good in-browser editor, so we didn’t need to install anything locally.
2. The editor supports realtime collaboration.
3. It has a live preview so we could see changes as we made them.

I was a little surprised by how Glitch handles binary assets. When you drag a JavaScript file into Glitch’s explorer, it adds the file to your project. But when you drag in something like an MP3, it goes into the project’s “assets” and gets a CDN URL.

Ben quickly figured out how to get the URL, and once that was sorted out, it was easy to add sounds and access them from our page.

tmi.js: browser-based Twitch chat integration

Our ultimate goal was to play sound effects in response to things said in Twitch chat, so it was convenient to be able to connect to chat directly from inside the browser.

tmi.js connects to Twitch’s WebSocket IRC proxy. Twitch allows anonymous lurkers in chat, so we did not need to set up a bot user and get an OAuth token.

Putting it all together

For each sound effect, we uploaded the asset and added its CDN URL to an audio tag on the page⁴:

<audio id="creeper" src="https://cdn.glitch.com/....mp3" />
<audio id="ghast" src="https://cdn.glitch.com/....mp3" />

We also added <script> tags for tmi.js and our own JavaScript:

<script src="tmi.min.js"></script>
<script src="/script.js" defer></script>

Finally, our JavaScript connects to the right Twitch channel and looks for specific messages to be sent. When the right message is sent, the appropriate audio tag is played. Most of this code is directly from the snippets on the tmi.js homepage:

const channel_name = 'sharkiller8';

const client = new tmi.Client({
	connection: {
		secure: true,
		reconnect: true
	},
	channels: [ channel_name ]
});

client.connect();

client.on('message', (channel, tags, message, self) => {
  handleMessage(message);
});

function handleMessage(msg) {
  if (msg == "Creeper!") {
    document.getElementById('creeper').play();
  }
  if (msg == "Ghast!") {
    document.getElementById('ghast').play();
  }  
}

Bonus: testing

We actually tested this by just using some other streamer’s chat.⁵ But once we were confident of the chat handling, we added a textbox to our web page to make it easier to test things without using Twitch. Here’s the HTML:

<input type="text" id="typey">

and the JavaScript, which just invokes the same handleMessage() function:

document.getElementById('typey').onchange = function () {
  handleMessage(this.value);
  this.value = "";
}

Endless possibilities

A web page composited on top of a Twitch stream is pretty useful. You can use this to play sounds, as we did, but you can also overlay all sorts of visual information, to show notifications when someone follows or subscribes, etc.

I appreciate being able to use standard HTML, CSS, and JavaScript to enhance live video.