Musical Collaboration Online

If only playing in realtime online were as simple as firing up a Zoom session, but it's not.  Round-trip latency, or the amount of time it takes a sound to travel out of your instrument or microphone, into your computer, over the internet to your collaborators, and back, is the enemy.  It is the dreaded echo - you play your note locally, then hear it a moment later through your headphones with an annoying delay. 
 

The discussion for the next few sections applies to all collaboration software.  Optimizing your hardware and software, and understanding the limitations of sound will be more important than the software you choose.

​

Before going further you should be familiar with the three tools we use both in the home studio for recording, as well as for collaborating online:  the recording interface, the computer, and the Digital Audio Workstation (DAW) software.  If you need to learn more, please read through the first three sections of the home studio recording gear page on this website.

​

Our brains can cope with around 10-25 milliseconds (thousandths of a second) of "one-way" latency without a problem. A 25 millisecond delay is what you hear live when you are approximately 25 feet from the monitor wedge.  Above 40 becomes unworkable.  But that's just half the picture - if you are collaborating with another musician online the total round-trip latency you experience between any two collaborators is the sum of the latency that each of the pairs of musicians hears.

Latency comes from several sources.
 

• A Suboptimal Audio Buffer Size:  Buffer size is the number of samples that your computer bites off at a time to process.  You adjust this either in the software that drives your recording interface, or if your online collaboration software communicates with the interface, you can change it from there.  This table to the left is an example of how buffer size and sample rate affects latency for a standard USB recording interface.  Your mileage may vary.  Thunderbolt interfaces are much faster than USB interfaces because the signal moves more quickly between your interface and your computer) and latency also varies by manufacturer.  Your interface software will tell you the actual latency contribution when you change the buffer size, 
  
  From the chart it might seem like a no brainer, just set the sample size to 32 samples and go.  But the smaller the sample size, the harder your computer needs to work (smaller bites of data repeated more often).  If that harder work causes the computer to fall behind in its processing work it can contribute to jitter.  Jitter (missing or mis-ordered data packets) results in unpleasant digital noise often described as crackling. 
  
  Your collaboration software can be set to try to correct for jitter, but that requires the computer to slow things down to complete the processing, and that results in latency.  So it's a tradeoff.  Jitter vs. Latency.
  
  ​I would recommend using 48kHz as your sample rate, and then starting at a buffer size of 256 samples.  Then try lowering it to 128.  If you don't hear crackling try lowering it to 64, and possibly even 32 samples.  But you may find that your limit is 256 or 128, so the contribution to latency from your computer will be 2.5-5 ms (twice that for both you and your collaborator).
   

• Your internet connection:  First things first - WiFi is a no go.  So are Bluetooth headphones.  You will need to use wired headphones plugged into the headphone jack of your interface and connect your interface with an ethernet cable either directly to your router or to a switch that is connected to your router via ethernet cable.  If you have a wireless mesh system, plugging your interface into the ethernet port of a wireless mesh point will generally not do the trick.  A fiber gigabit (1 gigabit per second) connection or faster is best.  Anything below 200 megabits will be too slow.  
  
  Slower or lower quality internet connections can also lead to jitter from missing data packets or packets that arrive in the wrong order.  As with the buffer size discussion, your software has two ways of dealing with this.  (a) Keep plowing ahead and ignore the missing or mis-ordered packets that are causing the digital noise, or (b) take a time-out to find the missing packets or re-order packets so that they can be played properly and thereby induce latency on purpose (auto jitter adjustment).  Again - a tradeoff between latency and digital noise. You can instruct the software the approach you would like to take using the auto-jitter adjustment, which will deal with jitter coming from all sources.
   

• Your computer:  Your audio drivers will contribute to latency.  Be sure to use an Asio driver such as the free Asio4All driver, not the standard Windows Audio driver, if you are using a Windows PC.  The more powerful the computer, the better it will be able to handle the load before resulting in jitter.  
  
  Another tradeoff is the degree of compression used on the audio being passed between collaborators.  Passing larger uncompressed audio will negatively impact users with narrower internet bandwidths, but smaller compressed files will put more load on the computer to compress and decompress the audio in real time.  If you are fortunate enough to have gigabit service, passing uncompressed audio is your best option. A user with slower internet and a more powerful computer might be better off with compressed files.
   

• Distance:  Here's the real show stopper.  We can't beat the laws of physics - distance matters.  At best your audio data packets will travel at the speed of light (186,282 miles/second) in a straight line from you to your collaborators, but that is unfortunately never the case.  Your data may zig and zag, and it may travel more slowly through a crowded network switch.  Even if conditions were perfect it would take your packet 0.013 seconds (13 ms) to travel 2,451 miles from NYC to LA as the crow flies, and another 13 ms to return, or 26 ms round-trip.  In real life it is closer to twice that, or 50+ ms.  So if we want to stay under 25 ms round trip between all of the contributors to latency, we get our best results when all of our collaborators are within 500 miles of each other, and the closer they are the better the results are. 
  
  So two collaborators 500 miles apart with gigabit service might have

  • 5 ms round-trip from distance (using 1/2 the crow's flight speed of light)​

  • 8 ms round-trip computer processing time

  • 8 ms round-trip from the need for the computers to correct for jitter
     

Which gets us to a workable 21 ms round-trip in theoretical latency, right about our target. In practice latency varies from session to session, and even during a session. Latency isn't just annoying, it often leads to something referred to as the "toilet bowl effect."  Especially with uptempo tunes, the drummer in NY might perceive that the bass player in Chicago is playing behind the beat because of latency.  She has two choices - either continue to play at a consistent tempo, which sounds like she is playing ahead of her bass player, or more instinctively trust her ears and slow slightly, perhaps unconsciously. 

The first is the correct choice for the sake of the session, but can lead to a far less musically enjoyable experience for the drummer.  The second will sound better, but then the bass player hears the drummer slowing, and he slows.  And then the drummer hears the bass player slowing ...  Thus, the toilet bowl effect.​​