ELearning/Course building/Building for the web: Technical considerations

Using audio and video on the web can be problematic for a number of reasons:

• Some users want to listen or watch “live” while others want to download files to their device for later consumption.
• Different browsers support different formats.
• If a browser does not support a particular format, the user must have a plug-in installed on his or her device.
• Some formats can be streamed or progressively downloaded while others must first be downloaded prior to listening.
• HTML5 is changing the way we share media.

Audio Properties

There are currently three audio file formats for HTML5, though the mix is different for each browser.

Audio File Properties'
File Type	Streamed¹	Lossy²	Patented³	HTML5 Browsers
MP3 (MPEG)	Y	Y	Y	IE9+, Chrome6+, Safari5+
OGG-Vorbis	Y	Y	N	Chrome6+, Firefox3.6+, Opera10+
WAV	Y	N	Y	Chrome6+, Firefox3.6+, Safari5+, Opera10+
¹Plays immediately; not stored on the device. ²Degrades when copied. ³May be subject to royalties associated with its production.

Sound quality arises from a number of properties:

Sample Rate indicates the number of digital snapshots taken of an audio signal each second, with a range from 1.1 kHz (poor AM radio) to 96.0 kHz (Blu-ray DVD). The most common sampling frequency is 44.1 kHz (44,100 times per second) which, for technical reasons, produces sound at about 22 kHz.

Bit Depth - determines dynamic range. When a sound wave is sampled, each sample is assigned the amplitude value closest to the original wave’s amplitude. Higher bit depth provides more possible amplitude values, producing greater dynamic range, a lower noise floor, and higher fidelity. Bit depth ranges from 8-bit (telephone) to 32-bit (best).

Channels - separate streams of audio information. Mono contains a single channel while stereo contains two, left and right. Surround sound contains six channels.

Compression - reduces the dynamic range of the original signal, between the loudest and quietest parts, and is expressed as bit rate. This is accomplished by boosting the quieter signals and moderating the loudest. The higher the compression, the poorer the reproduction of the original low and high frequencies. Humans can detect sound frequencies between 20Hz and 20kHz, with levels outside this range beyond perception. Our most sensitive range, where we can detect the smallest differences is between 2kHz and 5kHz, and in increments of about 2Hz. Compression can reduce the original signal to within the limits of human hearing without any perceivable reduction in quality. Bit rates range from 96Kbps to 320Kbps; 128 is equivalent to FM radio and 160 or higher is considered CD quality.

Latency - the delay (usually milliseconds) between when an audio signal enters and when it emerges from a system. We experience latency in the long pause that occurs between the time a speaker on one end stops and the other responds. This is most likely to occur during live broadcasts over satellite. Latency can also occur when video and audio signals from the same broadcast are output or received at different rates.

Generally speaking, audio recording and conversion tools account for these factors in their default settings, with the exception of latency. It’s always a good idea to examine these settings when converting uncompressed WAV files to a compressed format. Experts advise recording in WAV or other uncompressed format (e.g., FLAC, APE, AIFF) and then converting to a compressed form. Latency is a bandwidth issue. Moving from a 1Mbps connection to 2Mbps decreases latency by half, for example. Furthermore, the “last mile” connection between your physical location and the ISP is often the most significant bottleneck. Wireless and mobile connections have their own latency issues. As end users we can only deal with latency issues through our choice of ISP. As designers, we produce the cleanest, leanest files we can that meet quality standards.

Video properties

As with audio, HTML5 currently supports three file types and the mix is different among the popular browsers.

Video File Properties'
File Type	Streamed¹	Lossy²	Patented³	HTML5 Browsers
MP4	Y	Y	Y	IE9+, Chrome6+, Safari5+
OGG-Theora	Y	Y	N	Chrome6+, Firefox3.6+, Opera10+
WebM (VP8)	Y	Y	N	Chrome6+, Firefox3.6+, Safari5+, Opera10+
¹Plays immediately; not stored on the device. ²Degrades when copied. ³May be subject to royalties associated with its production.

Video files contain three elements:

• Container (AVI, MOV, FLV, MP4, MEX, OGG, WebM). Container type does not significantly impact quality since its purpose is to simply organize and enclose the other elements.
• Video and Audio Signals (frame size, aspect ratio, frame rate, bitrate, audio sample rate). More below.
• Codec (H.264, WMA, DivX, Theora, VP8), small helper applications used by software to encode and decode the video and audio signals. Most codecs are proprietary and require licensing. OGG-Theora and WebM-VP8 are open source.

Video quality is affected by the signal and its constituent parts, and the codec.

Frame Size – the dimension of the frame, in pixels (320X420, 800X600).

Aspect Ratio – the ratio of width to height of the frame (4:3, 16:9).

Frame Rate – speed at which individual frames are captured and played back, expressed as frames per second (12fps, 30fps). The higher the frame rate, the higher the quality of the video and the larger the file size.

Bitrate – the amount of data used to describe the video and audio, measured in bytes per second. Note that bit rate and frame size increase in tandem (using the H.264 codec, 640X480 = 700Kbps, 1920X1080 = 5Mbps). Blu-ray discs are recorded at 40Mbps. Bit rate can be constant or variable. Variable produces more optimized files for web delivery.

Audio Sample Rate – described above.

Codec – the encoding and decoding mini-program used by software come in a wide variety, compressed and uncompressed, lossy and lossless. We will discuss just the three associated with HTML5: H.264 (most associated with MP4), VP8 (WebM), and Theora (OGG).

Objective quality comparisons have been made between the three codecs (artstechnia.com). The broad view is that H.264 is tops, followed closely by VP8, with Theora lagging. H.264 rules for now, with support for 3D, low to Blu-ray quality compression, and nearly universal mobile device support. However, the scene remains volatile. WebM-VP8 is backed by Google. MP4-H.264 is patented and free until at least December, 2015, but the future remains unclear. There are rumors that Google Android/Chrome may be “backing off” MP4 support. OGG-Theora has been most associated with the Linux community, which may be increasing in presence with the growth of the Chrome linux-based operating system. Stay tuned.

For now, the experts suggest the following video formats:

• For regular browsers, provide video in MP4 plus either WebM or OGG.
• For mobile devices, provide MP4 in 480X360 or 640X480.

Devices

Developing technology-based education and training must consider the delivery device(s) that allow learner access. For now, we have three categories to contend with:

• Desktop and laptop/notebook computers (13.3" diagonal to 30+") that use web browsers and wired or wifi wireless access
• Tablets (7" to 10.2") that typically use apps (self-contained narrowly-focused programs) and connect through wifi but may also use cellular technology for web access
• Smartphones and phablets (4" to 6.2") that use apps almost exclusively and connect with cellular technology almost exclusively

By using web browsers, desktops and laptops/notebooks generally work well with Flash technology. Mobile devices (tablets and smartphones/phablets), on the other hand, were built to favor apps and HTML5. Flash can be installed, with a fair amount of bother, but Adobe has abandoned its support for Android. Apple never allowed Flash, although third parties created apps for this purpose. All modern browsers also support HTML5, so it only makes sense to develop in HTML5, JavaScript, Java, and other programming languages. Newer authoring tools such as Captivate and Articulate Storyline provide an interface in which you design first for computers and then modify the design for tablets, and finally smartphones. Note that authoring tools are quickly evolving. Captivate 9, for example, now automatically rearranges content for the different screen sizes, after which you can add tweaks.

On a cautionary note, Jakob Nielsen (2011), the highly-regarded usability expert, found that when reading from a smartphone screen, comprehension scores for complex Web content were 48% of desktop monitor scores. His article, Mobile Content Is Twice as Difficult is highly recommended. It reinforces the imperative to design differently for mobile.

---

#top

⇑ Up to Course building

⇑ ⇑ Up to Course development

⇑ ⇑ ⇑ Up to Home