While working on the BBC Sports app for Virgin TiVo, it would be safe to say that I encountered more problems than one would expect to on such a project. Some of them seemed to be bugs in Adobe’s StageCraft (Flash Lite 3.1), others seemed to be inconsistencies in the implementation of the Flash player on the box and still others came from unexpected content delivery network behaviour. As a result there were a number of feature-related tasks that turned out to be more difficult to implement than they perhaps should have been, and as a team we spent a number of days in total investigating causes of unexpected and/or undesirable behaviour in the app that resulted from these issues.
Still, this post isn’t about reflecting on the highs and lows of development or about detailing the inner workings of Virgin Media‘s flagship device: rather, it’s about a great little tool that I found while trying to diagnose one of the aforementioned problems: Connection Emulator by SoftPerfect.
The Euro 2012 tournament was about to begin and an updated version of the app was released to support it. During the opening match we received reports of stuttering video playback which we were able to replicate ourselves on some of the test devices in the office. What we were seeing came as quite a surprise because the first version of the app which was released for the start of the F1 season exhibited no such problems and played back its content without issue. All of the test content we had used during development had also played as intended. What was different about these new football streams that could cause such symptoms?
On the second or third day I was asked to take a look at the issue after different builds with different settings had failed to resolve it, and at the same time I was provided with some useful information including figures for the CDN (content delivery network) that was supplying the content. Some aspects of those figures were a little concerning and I wanted to run some tests on the TiVo under similar conditions to see how it responded.
I already had a tool that could simulate a limit in bandwidth, but in order to carry out the tests that I had in mind I needed something that could also simulate latency and packet loss. I develop on Windows because FlashDevelop (the greatest Flash IDE in the world) is only available for that OS, so Mac OSX’s built-in network simulation tools weren’t an option. After a few minutes on Google I discovered SoftPerfect’s Connection Emulator which claimed to offer precisely these features and so I downloaded the trial and took it for a spin.
Immediately the application instils confidence that you’re using a professional tool. The interface is well designed with separate tabs for each aspect of network simulation: transfer rate, latency, packet loss, duplication and reordering. The respective options on each tab are comprehensive and each option can be enabled and adjusted separately during or between tests. A graph at the bottom of the window gives a visual indication of the effect that your current settings are having on your machine’s throughput, with more precise statistics available in a separate pane.
Using the application I was able to test the TiVo under a variety of conditions with varying latency and packet loss, and along with throughput calculators (see here and here) was able to show conclusively why the football streams were stuttering: despite a dedicated connection of up to 10mbit on Virgin’s network, the TiVo simply wasn’t getting enough throughput from the CDNs because latency and packet loss were too high.
As one might imagine with stakeholders as large as projects like this often demand, in order to bring about a swift configuration change you need to be armed with significant evidence: vital settings will not be changed on the strength of gut feeling alone! Thanks to Connection Emulator I was able to provide that evidence in abundance which led to the necessary changes being made to deliver the content within the required parameters, and the rest of the tournament passed without any further problems.
One “gotcha” I discovered while using the tool however was that VirtualBox didn’t seem to like it one bit, blue-screening the host machine a few times before I was able to disable it. Switching over to VMware resolved the issue which would suggest that this is a problem with VirtualBox rather than with Connection Emulator itself, but I thought it worth mentioning anyway in case anyone else experiences the same issue.
VirtualBox issue aside, this is a great tool that performs its specialist tasks very well and very reliably, and it comes highly recommended to Windows-based developers who need to test their applications under varying network conditions.