February 9, 2012

Embedding Audio and Video in eBooks on the Kindle Fire


Looking for a complete guide on eBook design and development? Please consider The eBook Design and Development Guide, which contains everything you need to know about HTML, CSS, EPUB, and MOBI/KF8 to make an eBook like a pro. Pick it up at Amazon for $6.99 today.

Previous Post: Working with Images

Embedded audio and video is defined as actually packaging the files into your eBook, while streaming audio and video is linking your HTML markup to a server (e.g. YouTube) where the eReading device downloads it for use by the reader. So, the good news is that embedded audio and video only works on the Kindle App for iOS devices (i.e. Kindle for iPad, iPhone, etc.), which is a small fraction of readers, and streaming audio and video is not allowed inside Kindle eBooks at all. This sort of makes for a difficult interactive experience. Since embedded audio and video files are massive in size, it is almost pointless to include them in an eBook you plan on selling at the Amazon Kindle store. Nevertheless, you should become familiar with the HTML5 elements that make embedded audio and video possible in the hopes that future Kindle hardware and software will be able to support them.

The Rise of HTML5 for Audio and Video


For a long time, audio and video on websites was only supported by a sloppy plug-in process with Adobe's Flash Player that was equally cumbersome no matter what browser you were using. Following Steve Jobs' quip that Flash was lousy for mobile devices, web standards have been shifting toward the much easier to code and consume HTML5 elements for multimedia. All major browsers support the new audio and video elements and provide automated, simple controls to play the media, adjust the volume, and mute.

The problem lies with the fact that different browsers recognize only specific formats. This is due to codec licensing fiascoes and the fact that all the major players (Apple, Microsoft, Mozilla, Google, etc.) having different ideas on the best way to encode multimedia. Amazon has chosen to go with the codecs that are supported by Microsoft and Apple, which is the mp4 format for videos and mp3 format for audio. If you have audio and video content that you want to embed that is not in the mp4 or mp3 format, you can convert it with open source software such as VLC.

Embedding Audio and Video in Your eBook


The HTML5 elements for embedding audio and video are fairly simple. Suppose you had a video of a baby hanging out with the relatives in northeastern Thailand in a file called baby.mp4, along with some amusing audio in a file called babyscreaming.mp3. Some sample HTML code would be as follows:
<p>I'm one of those annoying fathers that is so boring he only talks about his damn kids.</p>
<p> <audio src="babyscreaming.mp3" controls title="Audio of a Baby Screaming"></audio></p>
<p><video src="baby.mp4" controls poster="images/babyplaceholder.jpg" title="Video of a Thai Baby"></video></p>
<p>That's it for right now.</p>
Embedded Audio and Video in the Chrome Browser

The audio and video files will not play until the user clicks on them. Notice that, like the img element, the <audio> and <video> tags are nested within paragraph tags to give them block element behavior (i.e. on their own line). There are ways to make the audio and video play automatically in web browsers, but this is highly annoying for people goofing off at work and will not be discussed in this guide.

Let's break down each of these attributes within the HTML5 multimedia tags. The src attribute (required) is similar to the img element in that it specifies a relative path to the source file. The controls attribute (required) tells the browser or eReading device to use the automatic controls programmed into the software. The title attribute (optional) provides a description of the content that is generally displayed when the mouse pointer is rolled over the controls. The poster attribute (optional) in the video element provides an image to be shown in the video frame until the reader clicks the video to play.

Amazon imposes some additional requirements beyond the HTML5 specification to ensure a good reading experience across all Kindle devices. If the reader is not using the Kindle App for iOS, then there should be a way to at least inform them that there is multimedia accompanying the eBook. The additional requirements per Amazon are as follows:
  1. Audio and video files must be kept in a separate directory entitled audiovideo (e.g. audiovideo/baby.mp4)
  2. Metadata about the description of the multimedia and its length in minutes and seconds must be provided within the paragraph tags (e.g. A Video about Baby (00:20)</p>)
  3. Fallback text must be provided within the audio or video element if the Kindle device cannot render the mp3 or mp4 format (e.g. <videoThis device does not support the required audio or video format. Please view it at my website.</video>)
  4. Multimedia content requires a reference in the Table of Contents
Utilizing all of these requirements in the HTML markup would result in the following code about this hypothetical baby video in northeastern Thailand:
<p>I'm one of those annoying fathers that is so boring he only talks about his damn kids.</p>
<p> <audio src="babyscreaming.mp3" controls title="Audio of a Baby Screaming">This device does not support the required audio or video format. Please view it at my website.</audio> An Audio Clip of Baby Screaming (01:34)</p>
<p><video src="baby.mp4" controls poster="images/babyplaceholder.jpg" title="Video of a Thai Baby"> This device does not support the required audio or video format. Please view it at my website.</video> A Video about Baby (00:20)</p>
<p>That's it for right now.</p>
Important Note: The fallback text in between the <audio></audio> or <video></video> tags will only render if the multimedia files cannot be read by the eReading device. The content after the </audio> or </video> tag will always render.

The audio and video will be embedded properly on Kindle iOS apps; however, the fallback content of telling the reader to check out baby on a website will appear on older e-ink Kindles, Kindle App for Android, and the Kindle Fire. Below is an example of how the same HTML markup would render on the Kindle Fire and Kindle iPad app:
Example of Fallback Content and Embedded Content

Embedding of video and audio in Kindle eBooks leaves much to be desired due to the lack of support. However, future Kindles will most likely move toward supporting this content to provide a more interactive experience for the reader—one can only hope.

Next Post: Working with Tables
Share/Bookmark

9 comments:

Dan said...

I'm somewhat confused. The title of your post is "Embedding Audio and Video in eBooks on the Kindle Fire" yet in the first paragraph you state "the good news is that embedded audio and video only works on the Kindle App for iOS devices (i.e. Kindle for iPad, iPhone, etc.)". So, the Fire does not support embedded video and audio? I thought the Fire was going to bring all this interactivity to ebooks? What gives??

Paul Salvette said...

Dan,

Thanks for your comment. The title of this post is a bit tongue-in-cheek, because the Kindle Fire does not support embedded or streaming multimedia. Hopefully, if enough people gripe about it, Amazon will eventually support it.

Jeff said...

Ugh. For hours, I've been working on embedding a .mp3 (in Apple Pages) in my book for the Kindle. Every time, it just shows me the code I've copied/pasted/altered, not the result of the code.

I've tried starting out with html (including < >) and ending with /html, but that is about the extent of my programming skills. Please help me figure out what I'm doing wrong. Thanks.

Paul Salvette said...

Jeff,

You need to use the HTML in an actual HTML document in your text editor rather than in Pages.

Richard said...

Paul,

Your article is very informative and useful. That means embedding audio video in Kindle for iOS devises is now possible. As I added in the coding (as per your html5 sample), it showed up video in website. But when I convert it to Kindle Previewer, it asked for "ffmpeg path". Any idea what is it? Thanks.

Paul Salvette said...

Richard,

Video still doesn't work properly on Kindle devices (even Kindle Fire).

minimedi. de said...

Hi Paul,
I am trying to get audio and video to work in Kindle files in different ways ... I created a HTML-file that renders fine in my browser and gets converted perfectly with KindleGen ... then I tossed in your HTML-snippet from the article ... works well in browser, but after conversion, the Kindle app on my iPhone says: "Unable to play video. There was a problem downloading the video content of this title. Please remove the book form your device and re-download it." Same error as when creating a file with Legend Maker software. Any ideas? Thanks so much, Ulrich

Paul Salvette said...

Sorry, but video still doesn't work properly on Kindle devices (even Kindle Fire). I haven't had much luck with it, although the HTML snippets are okay. I guess we have to wait for Amazon to get off their duffs.

minimedi. de said...

Well, at least it's not *me*
How unfortunate, though. Thanks! Best, Ulrich