-
Notifications
You must be signed in to change notification settings - Fork 10
x Modular tutorial for web app
This tutorial will build up a fully working task from simple modules, each meant to illustrate different concepts employed in a javascript web app that runs a behavioral task. Background reading and example code for each coding concept are included.
- Display an image (Canvases) -> mkturkimagedisplay.js
- Wait for user action (Promises & Generators) -> mkturk.html
- Save data (Dropbox) -> mkturkdropbox.js
- Control external hardware (Web Bluetooth) -> mkturkble.js
- Plot data (Googlecharts) -> liveplot.html & liveplot_googlecharts.js
- Load parameters/images externally (Dropbox) -> mkturkdropbox.js
- Make a Task (Task Parameters) -> mkturkparams.js & mkturk.html
Overview: In this module, we will examine how to display successive images rapidly. The speed of image display has come a long way in javascript with the introduction of performance.now() and window.requestAnimationFrame. In addition, we will buffer our two images on two canvases so that come display time, we only need to flip their canvas depth order to display one image or the other. In the future, this approach could be extended to a double buffering scenario where upcoming frames are pre-rendered on "offscreen" canvases during the display of the current frame's canvas.
Each .html contains a head and body section. For this module, we just need to create an html page with two canvases for displaying our two images in rapid succession and a paragraph element for showing the statistics text.
In the head section, we add a couple lines that will make our html page open in fullscreen, emulating app behavior.
<!doctype html>
<head>
<meta name="mobile-web-app-capable" content="yes"> <!-- full screen https://developer.chrome.com/multidevice/android/installtohomescreen -->
<meta name="viewport" content="width=device-width, user-scalable=no"> <!-- do not allow window rescaling. To avoid window rescaling in portrait mode, added with=device-width from http://stackoverflow.com/questions/22771523/ipad-w-retina-safari-reported-dimensions-in-landscape-make-no-sense -->
<script>
</script>
</head>Beginning of the body tag which contains our two canvases and paragraph element:
<body bgcolor=#7F7F7F>
<div id="canvasdiv" style="position:relative; width:100vw; height:100vh">
<p id="statstext" style="z-index:101; position: absolute; left: 1px; top: 1px; height: 40%; width: 80%; font-size: 18px; color: white; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif;"></p>
<canvas id="displaycanvas0" width="0" height="0" src="" style="z-index:0; position: absolute; left: 0px; top: 0px;"></canvas>
<canvas id="displaycanvas1" width="0" height="0" src="" style="z-index:1; position: absolute; left: 0px; top: 0px;"></canvas>
</div>Next, we initialize some variables that control how the images will display inside a script tag:
<script>
var canvas = {
framerate: 50, //fps
totaldur: 5, //seconds
sequence: [],
tsequence: [],
tsequenceActual: [0],
front: 0,
currentframe: 0,
ncanvas: 2,
scale: 1,
backingStoreRatio: 1,
devicePixelRatio: 1,
objs: [],
}
var images = {
image: [],
nums: [0,1],
scale: 1, //1=display on same # of pixels as image. <1 undersample, >1 large, oversampled
leftcorner: 500, //pixels
topcorner: 500, //pixels
filenameprefix: "imagetest",
}canvas.framerate will set how fast we want the images to display. Most tablets have a 60 fps refresh rate (16.667 ms between frames). This code is meant to test rendering at the specified frame rate and report back on how well javacript did rendering in time for the requested frame.
images.leftcorner & images.topcorner specify the x,y coordinate of the top left corner of where the images will be render. This is another parameter to vary. For example, you may want to move the images to flash underneath a photodiode you are using to test display times.
images.scale can be used to see how upscaling (image.scale>1) or downscaling (image.scale<1) of the original image (image.scale=1) affects rendering quality. Below, we'll discuss how to display the image most crisply given the advent of retina displays.
The sequence we'll use is simply to alternate which of two canvases is in front at rate canvas.framerate for canvas.duration seconds:
// 1: Create the canvas sequence (start at gray, then alternate between two images for canvas.totaldur seconds, then go back to gray canvas)
canvas.sequence.push(0)
canvas.tsequence.push(100)
var nframes = Math.round(canvas.totaldur*canvas.framerate/2)
for (var i=1; i<=nframes; i++){
canvas.sequence.push(0)
canvas.sequence.push(1)
canvas.tsequence.push(2*i * 1000/canvas.framerate + canvas.tsequence[0]) //in milliseconds
canvas.tsequence.push((2*i+1) * 1000/canvas.framerate + canvas.tsequence[0])
}Next, we'll setup the canvases. This is the most important part of this module to follow. We have to think about the concept of the logical pixel versus the device pixel. The device pixel is just what it says, it's how many pixels are on the actual, physical screen as you might find on the specs for your tablet. The logical pixel used to be the same as the device pixel up until the introduction of retina displays. In a typical retina display, there can be a devicePixelRatio of 2 so that each 1x1 logical pixel is rendered using 2x2 logical pixels. This upsampling requires interpolation and can lead to blurring over your image.
To work around this issue, you have to know an additional quantity, the webkitBackingStorePixelRatio. The backingStore is where canvas elements are rendered, and the BackingStoreRatio tells us the relative size of the backingStore to the canvas. For a simple example, say the backingStoreRatio = 2, then you are rendering your image in a space twice as large as the logical pixels of the canvas. While this results in a downscaling to go from the backingStore to the canvas, this is immediately compensated by the devicePixelRatio = 2 of a retina display, so you're image ends up being rendered pixel for physical pixel on the screen as your backingStore.
Unfortunately, the backingStoreRatio is browser dependent. In Chrome, the backingStoreRatio = 1, so now you are back to the situation where your image gets upsampled to the physical screen of a retina display. However, you can avoid this by doubling the size of your BackingStore through the canvasobj.width/height property then setting canvasobj.getContext("2d").scale(2,2). This creates a backingStore that is twice the size of the canvas. You render on this 2x backingStore, it gets scaled down to the logical pixels of the canvas and then scaled back up to the physical pixels of the device. You may be wondering why you had to manually rescale the canvas & backingStore. The idea was to give the user the option of deciding whether they want to create such large backingStores and take up memory or are ok with their image being upsampled. Other browsers automatically make that decision for you. Here's the code for setting up a standard canvas:
// 2: Setup canvas layout
for (var i=0; i<=canvas.ncanvas-1; i++){
canvas.objs.push(document.getElementById("displaycanvas" + i))
canvas.objs[i].style.top = "0px"
canvas.objs[i].style.left = "0px"
canvas.objs[i].width = window.innerWidth //get true window dimensions at last moment
canvas.objs[i].height = window.innerHeight
canvas.objs[i].style.margin = "0 auto"
canvas.objs[i].style.display = "block"
} //for canvas setupFollowed by code for updating the canvas BackingStore to work with HiDPI displays:
// 3: Scale for HiDPI display
canvas.devicePixelRatio = window.devicePixelRatio || 1
var visiblecontext = canvas.objs[0].getContext("2d");
canvas.backingStoreRatio = visiblecontext.webkitBackingStorePixelRatio || visiblecontext.mozBackingStorePixelRatio || visiblecontext.msBackingStorePixelRatio || visiblecontext.oBackingStorePixelRatio || visiblecontext.backingStorePixelRatio || 1;
canvas.scale = canvas.devicePixelRatio/canvas.backingStoreRatio
for (var i=0; i<=canvas.ncanvas-1; i++){
if (canvas.devicePixelRatio !== canvas.backingStoreRatio){
context = canvas.objs[i].getContext("2d")
canvas.objs[i].width = canvas.objs[i].width * canvas.scale
canvas.objs[i].height = canvas.objs[i].height * canvas.scale
canvas.objs[i].style.width = window.innerWidth + "px"
canvas.objs[i].style.height = window.innerHeight + "px"
canvas.objs[i].style.margin = "0 auto"
canvas.objs[i].getContext("2d").scale(canvas.scale,canvas.scale)
}
else{
//do nothing
} //if scaleCanvasforHiDPI
}//for HiDPIIn case you're wondering why you can't use the devicePixelRatio to determine the backing store size, the answer is that they aren't guaranteed to match. Despite presenting the same devicePixelRatio value, Chrome and Safari 6 can and do have entirely different approaches for the backing store size (and therefore the webkitBackingStorePixelRatio) on HiDPI devices. The net result is that we can't rely on devicePixelRatio to know how the browser is going to scale images that are written into the canvas. http://www.html5rocks.com/en/tutorials/canvas/hidpi/
Event logic of javascript Going Async with ES6 Generators, David Walsh blog
Chapter 4: Async flow control in You Don't Know JS, by Kyle Simpson
Thankfully, ES6 adds Promises to address one of the major shortcomings of callbacks: lack of trust in predictable behavior. Promises represent the future completion value from a potentially async task, normalizing behavior across sync and async boundaries.
But it's the combination of Promises with generators that fully realizes the benefits of rearranging our async flow control code to de-emphasize and abstract away that ugly callback soup (aka "hell").
Async callback stack in chrome devtools
Why async/await is a step in the wrong direction
Copied from mkturk.html comments section
<-- ************ COMMENT SECTION ************ --> This code uses generators & promises from ESM6 harmony to implement a state machine. This is experimental and only supported on modern browsers (see http://caniuse.com/#feat=promises for full support). The reasons for using this approach are twofold: (1) Solving the inversion of control with the old way of using async callbacks in javascript (http://blog.getify.com/promises-part-2/) (2) readability of the code (http://davidwalsh.name/async-generators) ->(1) makes exception handling much easier ->(2) makes the code easier to edit in the future: "The main strength of generators is that they provide a single-threaded, synchronous-looking code style, while allowing you to hide the asynchronicity away as an implementation detail. This lets us express in a very natural way what the flow of our program's steps/statements is without simultaneously having to navigate asynchronous syntax and gotchas." As of 2014.12.01, generators are not supported in safari and not in iOS (even Chrome for iOS is limited to apple webkit). Could transpile but better to use a native Chrome environment (i.e. android tablet). // Load audio webkit, see http://middleearmedia.com/controlling-web-audio-api-oscillators/ // var audiocontext = new webkitAudioContext(); // Create audio container with webkit prefix // In case you're wondering why you can't use the devicePixelRatio to determine the backing store size, the answer is that they aren't guaranteed to match. Despite presenting the same devicePixelRatio value, Chrome and Safari 6 can and do have entirely different approaches for the backing store size (and therefore the webkitBackingStorePixelRatio) on HiDPI devices. The net result is that we can't rely on devicePixelRatio to know how the browser is going to scale images that are written into the CANVAS. http://www.html5rocks.com/en/tutorials/canvas/hidpi/ <-- ************ /COMMENT SECTION ************ -->