The Internet is imperfect; data gets lost all the time. There are some methods to ensure that data always arrives intact, but these methods often introduce delay. As a result, video conferencing and other types of communications that require low delay/latency don’t use those methods, which means the decoder must then attempt to fill in the missing data. Sometimes this cover-up can be noticeable in the form of frozen and broken video.
SVC is able to solve this problem because of the different dependencies between layers – for example, in the case of temporal scalability.
In Illustration 1 depicting temporal scalability, if a blue frame is intentionally skipped, decoding can continue because the blue frame is not a dependency for any other frame (the same applies if that frame is somehow lost in transmission.)
The side effect, of course, is that the video will pause, but a pause for a single frame is almost imperceptible to the naked eye. Now if an orange frame is lost, where there exists a dependency for other frames, then the pause would last two frames. And if a gray frame is lost, where there are more dependencies, then the pause would last four frames. Pauses inevitably become more noticeable as more frames are lost, but fortunately, spatial scalability solves this.
In Illustration 2, if frame 4 from layer 2 is lost (marked with a red X). One option would be to skip it and pause the video for four frames, but a better alternative would be to drop down to layer 1 and display frames 4–7 (marked with red circles) at a lower resolution until we reach frame 8. Even though layer 1 is at a lower resolution, the average observer will find it more difficult to notice a slightly lower resolution image than to notice a long pause in the video.
In my final blog in this series, we’ll bring in LifeSize CTO Casey King for his take on SVC and what it means for our industry.