How to Display a Video on HTML5 Canvas

The following excerpt from the O'Reilly publication HTML5 Canvas will give help you work with video on HTML5 Canvas. First, we must learn the basics of displaying video on HTML5 Canvas. There are a few important things to note that are not immediately obvious when you start working with video and the canvas. We worked through them so you don’t have to do it yourself.

---

Video must still be embedded in HTML

Even though the video is only displayed on HTML5 Canvas, you still need a <video> tag in HTML. The key is to put the video in a <div> (or a similar construct), and set the display CSS style property of that <div> to none in HTML. This will ensure that while the video is loaded in the page, it is not displayed. If we wrote the code in HTML, it might look like this:

<div style="position: absolute; top: 50px; left: 600px; display:none">
<video loop controls id="thevideo" width="320" height="240" preload="auto">
 <source src="muirbeach.mp4" type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"' >
 <source src="muirbeach.webm"type='video/webm; codecs="vp8, vorbis"' >
 <source src="muirbeach.ogg" type='video/ogg; codecs="theora, vorbis"'>
</video>

However, we already know that we don’t want to use an HTML embed. As we stated at the end of the last section, video events do not appear to fire reliably when video elements are embedded in the HTML page. For this reason, we need a new strategy to load video dynamically—we’ll create the <div> and <video> elements in Javascript.

The first thing we do in our Javascript is add a couple variables to hold references to the dynamic HTML elements we will create. The videoElement variable will hold the dynamically created <video> tag, while videoDiv will hold the dynamically created <div>:

var videoElement;
var videoDiv;

Note: We use this method to create global variables throughout this chapter. There are many reasons not to use global variables, but for these simple applications, it’s the quickest way to get something on the canvas.

Next, we create our dynamic form elements in the eventWindowLoaded() function. First, we use the createElement() method of the document DOM object to create a <video> element and a <div> element, placing references to them in the variables we just created:

function eventWindowLoaded() {

   videoElement = document.createElement("video");
   videoDiv = document.createElement('div');
   document.body.appendChild(videoDiv);

Next, we add videoElement as a child of videoDiv, essentially putting it inside of that <div> on the HTML page. We then set the style attribute of <div> to display:none;, which will make it invisible on the HTML page. We do this because although we want the video to display on the canvas, we don’t want to show it on the HTML page:

   videoDiv.appendChild(videoElement);
   videoDiv.setAttribute("style", "display:none;");

We then create another new variable named videoType that holds the results of a new function we will create, supportedVideoFormat(). This function returns the file extension of the supported video format for the browser; otherwise, it returns “” (an empty string), which means we alert the user that there is no video support in the app for his browser:

   var videoType = supportedVideoFormat(videoElement);
   if (videoType == "") {
      alert("no video support");
      return;
   }

Finally, we set the src property of the video element using the file extension we just received from supportedVideoFormat(), and create the event handler for the canplaythrough event:

   videoElement.setAttribute("src", "muirbeach." + videoType);
   videoElement.addEventListener("canplaythrough",videoLoaded,false);

}

When the video has finished loading, the videoLoaded event handler is called, which in turn calls the canvasApp() function:

function videoLoaded(event) {

   canvasApp();

}

Before the code in the last section will work, we need to define the supportedVideoFormat() function. The reason for this function is simple: since we are adding video objects dynamically to the HTML page, we do not have a way to define multiple <source> tags. Instead, we are going to use the canPlayType() method of the video object to tell us which type of audio file to load.

The canPlayType() method takes a single parameter, a MIME type. It returns a text string of maybe, probably, or nothing (an empty string).


“” (nothing)



maybe



probably



We are going to use these values to determine which media type to load and play. For the sake of this exercise, we will assume that both maybe and probably equate to yes. If we encounter either result with any of our three MIME types (video/webm, video/mp4, video/ogg), we will return the extension associated with that MIME type so the sound file can be loaded.

In the function below, video represents the instance of HTMLVideoElement that we are going to test. The returnExtension variable represents that valid extension for the first MIME type found that has the value of maybe or probably returned from the call to canPlayType():

function supportedVideoFormat(video) {
   var returnExtension = "";
   if (video.canPlayType("video/webm") =="probably" || 
       video.canPlayType("video/webm") == "maybe") {
         returnExtension = "webm";
   } else if(video.canPlayType("video/mp4") == "probably" || 
       video.canPlayType("video/mp4") == "maybe") {
         returnExtension = "mp4";
   } else if(video.canPlayType("video/ogg") =="probably" || 
       video.canPlayType("video/ogg") == "maybe") {
         returnExtension = "ogg";
   }

   return returnExtension;

}

We do not check for a condition when no valid video format is found and the return value is “”. If that is the case, the code that has called this function might need to be written in a way to catch that condition and alter the program execution. We did that with the test of the return value and alert(), which we described previously.

Video is displayed like an image

When you write code to display a video on the canvas, you use the context.drawImage() function, as though you are displaying a static image. Don’t go looking for a drawVideo() function in the HTML5 Canvas spec because you won’t find it. The following code will display a video stored in a variable named videoElement, displayed at the x,y position of 85,30:

context.drawImage(videoElement , 85, 30);

However, when you draw a video for the first time, you will notice that it will not move—it stays on the first frame. At first you might think you have done something wrong, but you have not. You just need to add one more thing to make it work.

Set an interval to update the display

Just like when we discussed animation in the previous chapters, a video placed on HTML5 Canvas using drawImage() will not update itself. You need to call drawImage() in some sort of loop to continually update the image with new data from the playing video in the HTML page (hidden or not). To do this, we call the video’s play() method, and then use setInterval() to call the drawScreen() function every 33 milliseconds. This will give you about 30 frames per second (FPS). We put this code in our canvasApp() function, which is called after we know the video has loaded:

videoElement.play();
setInterval(drawScreen, 33);

In drawScreen(), we will call drawImage() to display the video, but since it will be called every 33 milliseconds, the video will be updated and play on the canvas:

function  drawScreen () {

   context.drawImage(videoElement , 85, 30);

   }

Example 6-6 gives the full code for displaying a video on the canvas and updating it using setInterval(). Figure 6-6 shows this code in the browser.

Example 6-6. Basic HTML5 loading video onto the canvas

<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>CH6EX6: Basic HTML5 Load Video Onto The Canvas</title>
<script src="modernizr-1.6.min.js"></script>
<script type="text/javascript">
window.addEventListener('load', eventWindowLoaded, false);
var videoElement;
var videoDiv;
function eventWindowLoaded() {

   videoElement = document.createElement("video");
   videoDiv = document.createElement('div');
   document.body.appendChild(videoDiv);
   videoDiv.appendChild(videoElement);
   videoDiv.setAttribute("style", "display:none;");
   var videoType = supportedVideoFormat(videoElement);
   if (videoType == "") {
      alert("no video support");
      return;
   }
   videoElement.setAttribute("src", "muirbeach." + videoType);
   videoElement.addEventListener("canplaythrough",videoLoaded,false);

}

function supportedVideoFormat(video) {
   var returnExtension = "";
   if (video.canPlayType("video/webm") =="probably" || 
       video.canPlayType("video/webm") == "maybe") {
         returnExtension = "webm";
   } else if(video.canPlayType("video/mp4") == "probably" || 
       video.canPlayType("video/mp4") == "maybe") {
         returnExtension = "mp4";
   } else if(video.canPlayType("video/ogg") =="probably" || 
       video.canPlayType("video/ogg") == "maybe") {
         returnExtension = "ogg";
   }

   return returnExtension;

}

function canvasSupport () {
     return Modernizr.canvas;
}

function videoLoaded(event) {

   canvasApp();

}

function canvasApp() {

   if (!canvasSupport()) {
          return;
        }

function  drawScreen () {

      //Background
      context.fillStyle = '#ffffaa';
      context.fillRect(0, 0, theCanvas.width, theCanvas.height);
      //Box
      context.strokeStyle = '#000000';
      context.strokeRect(5,  5, theCanvas.width−10, theCanvas.height−10);
      //video
      context.drawImage(videoElement , 85, 30);

   }

   var theCanvas = document.getElementById("canvasOne");
   var context = theCanvas.getContext("2d");
   videoElement.play();

   setInterval(drawScreen, 33);

}

</script>

</head>
<body>
<div style="position: absolute; top: 50px; left: 50px;">

<canvas id="canvasOne" width="500" height="300">
 Your browser does not support HTML5 Canvas.
</canvas>
</div>
</body>
</html>