Events

Implementing Air Mozilla’s Related Events Feature

Editor’s note: It’s not often that we hear from web developers and software engineers at the start their careers here on the Hacks blog. This post is a great reminder of what it’s like.

Mozilla participates in Outreachy, and offers internship opportunities to bring women and other under-represented groups into the world of free and open source software. The application process for the upcoming round will open on September 22, 2015. The application deadline is October 26. Internship dates will be December 7, 2015 to March 7, 2016. If you’ve been thinking of applying, read this, be like Gloria – and go for it!

How I became a Mozilla intern

It was a spring night in Greece when I got the news I’d been selected by Mozilla to be an Outreachy intern. Outreachy is a project of the Software Freedom Conservancy that helps people from groups underrepresented in FOSS (free and open source software) to get involved by offering focused internship opportunities with a number of open source software organizations. That night, I was the happiest person in my town, I believe. I remember rushing to say a big thank you to my mentor Peter Bengtsson, before I got too overwhelmed with happiness to the point that I forgot to be grateful. It was a two-way street. I chose Mozilla and Mozilla chose me.

I started contributing to Air Mozilla in March 2015, by solving beginner-friendly bugs. Initially I didn’t want to apply at all, because there were too many wonderful contributors, and in some way I felt overwhelmed by how rigorous the process was. I told several people I won’t apply because I am afraid I won’t be getting it, and sometimes it is okay to be afraid, but it is not okay to be afraid of something you are capable of achieving to the point of sabotaging yourself.

After several pep talks, I went to view the list of the companies and decided to apply only for Mozilla. It will be a hit or miss, I told myself. I noticed it listed “Python” and I was aware that I didn’t know Python that well as it was something I self-studied briefly in the past, but I told myself to go ahead and try. Sometimes you need to do hard things to get the experience you want. Every expert was once a newbie. I went to check Bugzilla to see what I could fix. One bug looked very easy, so I thought I should work on that. That’s how it all started. Feedback after feedback, perseverance, pull, merge, merge conflicts, and getting a deeper understanding of how the code base functions made me able to contribute a lot more. One thing I loved about interning with open source is the fact that everything is open. I was asked to blog about what I worked on, and I didn’t need to hide anything that concerned it. Well… that doesn’t mean I shared my passwords too though! Passwords need to be protected and secret for a reason.

Getting to work on Air Mozilla

The internship day officially started on May 25th. I was told that after discussions I am being assigned to add a new feature. That new feature will act as a recommendation system suggesting similar events to Air Mozilla viewers while they view a specific event. Air Mozilla is a platform for Mozilla’s online multimedia presence that provides live and pre-recorded shows, interviews, news snippets, tutorial videos, and features about the Mozilla community.

That sounded fascinating. I wasn’t nervous at all, it seemed pretty interesting and I always wondered how those things worked, that’s until I started getting really stuck down the road. Then I realized the complexity of it. I had to either build my own algorithms and computations to do the job, or find an open source tool we could use instead of reinventing the wheel. After several days of research we decided Elastic Search seemed like a good fit. Elastic Search, is a search server. It has a function called “More Like This” and that helps you search for similar events based on your set parameters. I went and read the documentation of Elastic Search but to be honest, I had a very hard time navigating it. Mostly because I couldn’t find enough examples on how to use it with Python. At some point we decided pyelasticsearch might make things easier; it’s a Python library version of Elastic Search that has the functionalities we were looking for.

For a short recap, Elastic Search creates a mapping and indexes all the events on Air Mozilla. When you visit an event page, with the “More Like This” feature, we ask it to search the indexed events that are similar to the event you are currently viewing and return back the title, id, tags and its similarity scores on the back-end. For events to be considered related, their tags, channel and titles must be very much alike, but that’s not where we stop.

We also need to be able to set someone’s level of access: a person who is logged in as a Mozilla employee will have more access than an anonymous guest would. A logged in volunteer contributor will have more access than an anonymous guest, but less than a Mozilla employee. You get the idea. Each event has its own access level setting. Some events are private and limited to employees, while the majority of Air Mozilla events are accessible to the public or to members of the entire Mozilla community. Regardless, the Related Events feature lets people see a list of similar events, filtered for their level of access, with the most similar “related events” appearing on top.

Implementing Elastic Search

In the code below we are using the more_like_this (mlt) query. We break the tags and the titles into different queries because we want to boost the relevance score of each one to a different degree. In this case we want a similar title to have a higher relevance score than the similar tags. The field parameter is used to determine which field we are trying to run mlt against, the docs parameter is the documents we are trying to find similar events for. You can read more about this at the More Like This Query page.

mlt_query1 = { 
    'more_like_this': { 
        'fields': ['title'], 
        'docs': [ 
            { 
                '_index': index, 
                '_type': doc_type, 
                '_id': event.id 
            }], 
        'min_term_freq': 1, 
        'max_query_terms': 20, 
        'min_doc_freq': 1, 
        'boost': 1.0, 
    } 
} 
mlt_query2 = { 
    'more_like_this': { 
        'fields': ['tags'], 
        'docs': [ 
            { 
                '_index': index, 
                '_type': doc_type, 
                '_id': event.id 
            }], 
        'min_term_freq': 1, 
        'max_query_terms': 20, 
        'min_doc_freq': 1, 
        'boost': -0.5, 
    }
} 

 


Below we are asking that one or more of the given mlt queries should match the documents, i.e., list of events:

query_ = { 
    'bool': { 
        'should': [mlt_query1, mlt_query2], 
    } 
} 

 


Below, with request.user.is_active we are checking if the user is logged in and what their access level is. If the user is logged in as a volunteer the events that appear must not contain events that are restricted to Mozilla employees only.

If the user is logged in and is not a volunteer, that means they are Mozilla paid staff members. Their “related events” results should contain all events that are similar to the event we are currently viewing.

In the case of an anonymous user, the “related events” results must contain only events that are accessible to everyone.

if request.user.is_active: 
  query = { 
    'fields': fields, 
    'query': query_ 
  } 
else: 
  query = { 
    'fields': fields, 
    'query': query_, 
    "filter": { 
      "bool": { 
        "must": { 
          "term": {"privacy": Event.PRIVACY_PUBLIC} 
        } 
      } 
    } 
  } 

In the code above you see we used the more_like_this query along with a filter. More_like_this does return similar events based on title, tags and channels but it doesn’t ensure that the events returned are filtered by access level. So this is where “filter” comes in. It makes sure to filter the relevant events and allow only the ones the user can view based on their level of access.

With the use of cron jobs I completely delete and re-index the index for the week, and every 10 minutes we re-index (without first deleting) all events that have changed in the last 10 minutes.

AJAX queries were used to get the thumbnails and links of the related events. AJAX loads asynchronously, so running those queries doesn’t slow the pageload. When I view an event, I want it to load the page fully without waiting for Elastic Search to finish running the queries. Elastic Search queries run pretty fast but it still takes some time. Sometimes, it can be slow. The relevant events section appears under the events’ details. Ideally, someone watching an event won’t mind if there is a slight delay in the loading of the related events while they view the event, however a page that loads slowly as a whole can be frustrating. Through AJAX our Elastic Search queries can run in the background after the page has loaded.

In the screenshots below, you can see how it works. I have added red arrows to the title of event we are on and the title of the first recommended event. If you compare both pictures you can see how the events we are viewing, and/or recommending, relate to each other.

Example of Related Events for volunteers and staff

A second example of Related Events

Examples of related events for signed-in users

 

4 key things that make you realize how much you enjoyed your internship:

  • Your mentor made sure you took the right steps to learn a lot.
  • You feel that every portion of it, even the frustrating moments, made you grow not only in skills but as a person.
  • You were extremely happy you got it.
  • You are extremely sad it is ending.

That’s exactly how it’s been for me. I want to take a moment to thank Outreachy, my mentor Peter, and the other Mozillians I met on IRC and in email during my internship. Thank you for being helpful, welcoming, and for contributing to my learning experience.

What made the pensive ending of my internship bright and merry is a motto Mozilla takes pride in: Once a Mozillian, Always a Mozillian.

View full post on Mozilla Hacks – the Web developer blog

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Keyboard events in Firefox OS TV: Part 2

Implementation details for keyboard events

In our introductory post, Keyboard events in Firefox OS TV, we described four keyboard event scenarios triggered by the Info key on a Smart TV remote: SYSTEM-ONLY, SYSTEM-FIRST, APP-CANCELLED, AND APP-FIRST. We explained how these keyboard events are activated, described the default sequence of events, and explored the iframe structure in Firefox OS and its embedded browser, in order to understand how developers can implement the four keyboard event scenarios.

Now we’ll take a closer look at each of the four scenarios, complete with example code for each event-handling scenario.

SYSTEM-ONLY

If a keyboard event is categorized as SYSTEM-ONLY, then the desired response is defined in mozbrowserbeforekey*’s event handler. Once this key is pressed, the system receives the mozbrowserbeforekey* event before the key* event is dispatched to an app. In addition, the key* events dispatch is cancelled once the system event handler is called. Now, we need to figure out a way to stop the event dispatch. Fig 5 – Keyboard events in Firefox OS TV shows that the keyboard events are dispatched to the system process, then also to the app process. To stop dispatching events to the the embedded page, event.preventDefault() is a straightforward solution. Event.preventDefault() is used to prevent the default action. The defined default action of the mozbrowserbeforekey* event is to dispatch the key* event. For this reason, by calling event.preventDefault() in mozbrowserbeforekey*’s event handler, key* events won’t be dispatched. The final result will be the same as shown in Fig 7.

f7

[Fig 7]

SYSTEM-FIRST

This is very similar to the implementation of SYSTEM-ONLY. The only difference — it’s not necessary to call event.preventDefault() in mozbrowserbeforekey*’s event handler. Apps are able to handle the key* event after the system finishes processing it.

f8

[Fig 8]

APP-CANCELLED
f9

[Fig 9]

If specific keyboard events are designated for use by apps only, such as those assigned to the four colored keys (shown in Fig 9) on smart TV remotes, then event.preventDefault() will be called in the app’s key* event handler. The event.preventDefault() cannot prevent the mozbrowserafterkey* event from being dispatched to the system, but the property embeddedCancelled of mozbrowserafterkey* will be set to true once the embedded app calls event.preventDefault(). The value of embeddedCancelled lets the system detect whether or not this event has been handled already. If this value is true, the system does nothing.

f10

[Fig 10]

APP-FIRST

The difference between APP-FIRST and APP-CANCELLED is that with APP-FIRST event.preventDefault() will not be called in the app’s event handler. Therefore, the value of embeddedCancelled is false and the system can take over the keyboard event.

f11

[Fig 11]

Sample code

Event handlers
function handleEvent(event) {
  dump("Receive event '" + event.type + "'.");
  // Handle event here.....
};

function handleEventAndPreventDefault(event) {
  dump("Receive event '" + event.type + "'.");
  // Handle event here.....

  // Call preventDefault() to stop the default action.
  // It means that the event is already handled.
  event.preventDefault();
};

function checkAttrAndHandleEvent(event) {
  dump("Receive event '" + event.type + 
       "' with embeddedCancelled equals to '" +
       event.embeddedCancelled + "'.");
  if (!event.embeddedCancelled) {
    // Do something if the event wasn't being handled before!
    // The following code should be executed in APP-FIRST scenario only!
  }
};
SYSTEM-ONLY
  • mozbrowser iframe host page
window.addEventListener('mozbrowserbeforekeydown', handleEventAndPreventDefault);
window.addEventListener('mozbrowserbeforekeyup', handleEventAndPreventDefault);
window.addEventListener('mozbrowserafterkeydown', function() { }); // no use
window.addEventListener('mozbrowserafterkeyup', function() { }); // no use
  • the embedded page
// This function will never be triggered because the preventDefault() is called in mozbrowserbeforekeyXXX's handler.
window.addEventListener('keydown', handleEvent);
window.addEventListener('keyup', handleEvent);
  • Results of keydown related events
Order the embedded page mozbrowser iframe host page Output
1 mozbrowserbeforekeydown Receive event ‘mozbrowserbeforekeydown’.
2 mozbrowserafterkeydown
  • Results of keyup related events
Order the embedded page the host page Output
1 mozbrowserbeforekeyup Receive event ‘mozbrowserbeforekeyup’.
2 mozbrowserafterkeyup
SYSTEM-FIRST
  • mozbrowser iframe host page
window.addEventListener('mozbrowserbeforekeydown', handleEvent);
window.addEventListener('mozbrowserbeforekeyup', handleEvent);
window.addEventListener('mozbrowserafterkeydown', function() { }); // no use
window.addEventListener('mozbrowserafterkeyup', function() { }); // no use
  • the embedded page
window.addEventListener('keydown', handleEvent);
window.addEventListener('keyup', handleEvent);
  • Received results of keydown related events
Order mozbrowser-embedded page mozbrowser iframe host page Output
1 mozbrowserbeforekeydown Receive event ‘mozbrowserbeforekeydown’.
2 keydown Receive event ‘keydown’.
3 mozbrowserafterkeydown
  • Received results of keyup related events
Order the embedded page mozbrowser iframe host page Output
1 mozbrowserbeforekeyup Receive event ‘mozbrowserbeforekeyup’.
2 keyup Receive event ‘keyup’.
3 mozbrowserafterkeyup Receive event ‘mozbrowserafterkeyup’ with embeddedCancelled equals to ‘true’.
APP-CANCELLED
  • mozbrowser iframe host page
window.addEventListener('mozbrowserbeforekeydown', function() { }); // no use
window.addEventListener('mozbrowserbeforekeyup', function() { }); // no use
window.addEventListener('mozbrowserafterkeydown', checkAttrAndHandleEvent);
window.addEventListener('mozbrowserafterkeyup', checkAttrAndHandleEvent);
  • mozbrowser iframe embedded page
window.addEventListener('keydown', handleEventAndPreventDefault);
window.addEventListener('keyup', handleEventAndPreventDefault);
  • Received results of keydown related events
Order the embedded page mozbrowser iframe host page Output
1 mozbrowserbeforekeydown
2 keydown Receive event ‘keydown’.
3 mozbrowserafterkeydown Receive event ‘mozbrowserafterkeydown’ with embeddedCancelled equals to ‘true’.
  • Received results of keyup related events
Order mozbrowser-embedded page mozbrowser iframe host page Output
1 mozbrowserbeforekeyup
2 keyup Receive event ‘keyup’.
3 mozbrowserafterkeyup Receive event ‘mozbrowserafterkeyup’ with embeddedCancelled equals to ‘true’.
APP-FIRST
  • mozbrowser iframe host page
window.addEventListener('mozbrowserbeforekeydown', function() { }); // no use
window.addEventListener('mozbrowserbeforekeyup', function() { }); // no use
// This will be trigger after keydown event is
// dispatched to mozbrowser iframe embedded page
window.addEventListener('mozbrowserafterkeydown', checkAttrAndHandleEvent);
window.addEventListener('mozbrowserafterkeyup', checkAttrAndHandleEvent);
  • mozbrowser iframe embedded page
window.addEventListener('keydown', handleEvent);
window.addEventListener('keyup', handleEvent);
  • Received results of keydown related events
Order mozbrowser-embedded page mozbrowser-iframe host page Output
1 mozbrowserbeforekeydown
2 keydown Receive event ‘keydown’.
3 mozbrowserafterkeydown Receive event ‘mozbrowserafterkeydown’ with embeddedCancelled equals to ‘false’.
  • Received results of keyup related events
Order mozbrowser-embedded page mozbrowser iframe host page Output
1 mozbrowserbeforekeyup
2 keyup Receive event ‘keyup’.
3 mozbrowserafterkeyup Receive event ‘mozbrowserafterkeyup’ with embeddedCancelled equals to ‘false’.

 

 

 

View full post on Mozilla Hacks – the Web developer blog

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Pointer Events now in Firefox Nightly

This past February Pointer Events became a W3C Recommendation. In the intervening time Microsoft Open Tech and Mozilla have been working together to implement the specification. As consumers continue to expand the range of devices that are used to explore the web with different input mechanisms such as touch, pen or mouse, it is important to provide a unified API that developers can use within their applications. In this effort we have just reached a major milestone: Pointer events are now available in Firefox Nightly. We are very excited about this effort which represents a great deal of cooperation across several browser vendors in an effort to produce a high quality industry standard API with growing support.

Be sure to download Firefox Nightly and give it a try and give us your feedback on the implementation either using the dev-platform mailing list or the mozilla.dev.platform group. If you have feedback on the specification please send those to public-pointer-events@w3.org.

The intent of this specification is to expand the open web to support a variety of input mechanisms beyond the mouse, while maintaining compatibility with most web-based content, which is built around mouse events. The API is designed to create one solution that will handle a variety of input devices, with a focus on pointing devices (mouse, pens, and touch). The pointer is defined in the spec as a hardware-agnostic device that can target a specific set of screen coordinates. Pointer events are intentionally similar to the current set of events associated with mouse events.

In the current Nightly build, pointer events for mouse input are now supported. Additionally, if you’re using Windows, once you’ve set two preferences, touch events can be enabled now. The first property, Async Pan & Zoom (APZ) is enabled by setting the layers.async-pan-zoom.enabled Firefox configuration preference to true. The dom.w3c_touch_events.enabled should also be enabled by setting this value to 1 in your preferences.

This post covers some of the basic features of the new API.

Using the Pointer API

Before getting started with the Pointer API, it’s important to test whether your current browser supports the API. This can be done with code similar to this example:

if (window.PointerEvent) {
.....
}else{
// use mouse events
}

The Pointer API provides support for pointerdown, pointerup, pointercancel, pointermove, pointerover, pointerout, gotpointercapture, and lostpointercapture events. Most of these should be familiar to you if you have coded event handling for mouse input before. For example, if you need a web app to move an image around a canvas when touched or clicked on, you can use the following code:

function DragImage() {
    var imageGrabbed = false;
    var ctx;
    var cnv;
    var myImage;
    var x = 0;
    var y = 0;
    var rect;
    this.imgMoveEvent = function(evt) {
        if (imageGrabbed) {
            ctx.clearRect(0, 0, cnv.width, cnv.height);
            x = evt.clientX - rect.left;
            y = evt.clientY - rect.top;
            ctx.drawImage(myImage, x, y, 30, 30);
 
        }
    }
    this.imgDownEvent = function(evt) {
        //Could use canvas hit regions
        var xcl = evt.clientX - rect.left;
        var ycl = evt.clientY - rect.top;
        if (xcl > x && xcl < x + 30 && ycl > y && ycl < y + 30) {
            imageGrabbed = true;
        }
    }
    this.imgUpEvent = function(evt) {
        imageGrabbed = false;
    }
    this.initDragExample = function() {
        if (window.PointerEvent) {
            cnv = document.getElementById("myCanvas");
            ctx = cnv.getContext('2d');
            rect = cnv.getBoundingClientRect();
            x = 0;
            y = 0;
            myImage = new Image();
            myImage.onload = function() {
                ctx.drawImage(myImage, 0, 0, 30, 30);
            };
            myImage.src = 'images/ff.jpg';
            cnv.addEventListener("pointermove", this.imgMoveEvent, false);
            cnv.addEventListener("pointerdown", this.imgDownEvent, false);
            cnv.addEventListener("pointerup", this.imgUpEvent, false);
        }
    }
}

 


PointerCapture events are used when there’s the possibility that a pointer device could leave the region of an existing element while tracking the event. For example, suppose you’re using a slider and your finger slips off the actual element –you’ll want to continue to track the pointer movements. You can set PointerCapture by using code similar to this:

var myElement = document.getElementById("myelement");
myelement.addEventListener("pointerdown", function(e) {
    if (this.setPointerCapture) {
    //specify the id of the point to capture
        this.setPointerCapture(e.pointerId);
    }
}, false);

 


This code guarantees that you still get pointermove events, even if you leave the region of myelement. If you do not set the PointerCapture, the pointer move events will not be called for the containing element once your pointer leaves its area. You can also release the capture by calling releasePointerCapture. The browser does this automatically when a pointerup or pointercancel event occurs.

The Pointer Event interface

The PointerEvent interface extends the MouseEvent interface and provides a few additional properties. These properties include pointerId, width, height, pressure, tiltX, tiltY, pointerType and isPrimary.

The pointerId property provides a unique id for the pointer that initiates the event. The height and width properties provide respective values in CSS pixels for the contact geometry of the pointer. When the pointer happens to be a mouse, these values are set to 0. The pressure property contains a floating point value from 0 to 1 to indicate the amount of pressure applied by the pointer, where 0 is the lowest and 1 is the highest. For pointers that do not support pressure, the value is set to 0.5.

The tiltY property contains the angle value between the X-Z planes of the pointer and the screen and ranges between -90 and 90 degrees. This property is most useful when using a stylus pen for pointer operations. A value of 0 degrees would indicate the pointer touched the surface at an exact perpendicular angle with respect to the Y-axis. Likewise the tiltX property contains the angle between the Y-Z planes.

The pointType property contains the device type represented by the pointer. Currently this value will be set to mouse, touch, pen, unknown or an empty string.

var myElement = document.getElementById("myelement");
myElement.addEventListener("pointerdown", function(e) {
    switch(e.pointerType) {
        case "mouse":
            console.log("Mouse Pointer");
            break;
        case "pen":
            console.log("Pen Pointer");
            break;
        case "touch":
            console.log("Touch Pointer");
            break;
        default:
            console.log("Unknown Pointer");
    }
 
}, false);

The isPrimary property is either true or false and indicates whether the pointer is the primary pointer. A primary pointer property is required when supporting multiple touch points to provide multi-touch input and gesture support. Currently this property will be set to true for each specific pointer type (mouse, touch, pen) when the pointer first makes contact with an element that is tracking pointer events. If you are using one touch point and a mouse pointer simultaneously both will be set to true. The isPrimary property will be set to false for an event if a different pointer is already active with the same pointerType.

var myElement = document.getElementById("myelement");
myelement.addEventListener("pointerdown", function(e) {
    if( e.pointerType == "touch" ){
         if( e.isPrimary ){
             //first touch
         }else{
             //handle multi-touch
         }
    }
 
}, false);

Handling multi-touch

As stated earlier, touch pointers are currently implemented only for Firefox Nightly running on Windows with layers.async-pan-zoom.enabled and dom.w3c_touch_events.enabled preferences enabled. You can check to see whether multi-touch is supported with the following code.

if( window.maxTouchPoints && window.maxTouchPoints > 1 ){
//supports multi-touch
}

Some browsers provide default functionality for certain touch interactions such as scrolling with a swipe gesture, or using a pinch gesture for zoom control. When these default actions are used, the events for the pointer will not be fired. To better support different applications, Firefox Nightly supports the CSS property touch-action. This property can be set to auto, none, pan-x, pan-y, and manipulation. Setting this property to auto will not change any default behaviors of the browser when using touch events. To disable all of the default behaviors and allow your content to handle all touch input using pointer events instead, you can set this value to none. Setting this value to either pan-x or pan-y invokes all pointer events when not panning/scrolling in a given direction. For instance, pan-x will invoke pointer event handlers when not panning/scrolling in the horizontal direction. When the property is set to manipulation, pointer events are fired if panning/scrolling or manipulating the zoom are not occurring.

This element receives pointer events when not panning in the horizontal direction.

 


// Very Simplistic pinch detector with little error detection,
// using only x coordinates of a pointer event
 
// Currently active pointers
var myPointers = [];
var lastDif = -1;
 
function myPointerDown(evt) {
    myPointers.push(evt);
    this.setPointerCapture(evt.pointerId);
    console.log("current pointers down = " + myPointers.length);
}
 
//remove touch point from array when touch is released
function myPointerUp(evt) {
    // Remove pointer from array
    for (var i = 0; i < myPointers.length; i++) {
        if (myPointers[i].pointerId == evt.pointerId) {
            myPointers.splice(i, 1);
            break;
        }
    }
    console.log("current pointers down = " + myPointers.length);
 
    if (myPointers.length < 2) {
        lastDif = -1;
    }
}
 
//check for a pinch using only the first two touchpoints
function myPointerMove(evt) {
    // Update pointer position.
    for (var i = 0; i < myPointers.length; i++) {
        if (evt.pointerId = myPointers[i].pointerId) {
            myPointers[i] = evt;
            break;
        }
    }
 
    if (myPointers.length >= 2) {
        // Detect pinch gesture.
        var curDif = Math.abs(myPointers[0].clientX - myPointers[1].clientX);
        if (lastDif > 0) {
            if (curDif > lastDif) { console.log("Zoom in"); }
            if (curDif < lastDif) { console.log("Zoom out"); }
        }
        lastDif = curDif;
    }
}

You can test the example code here. For some great examples of the Pointer Events API in action, see Patrick H. Lauke’s collection of Touch and Pointer Events experiments on GitHub. Patrick is a member of the W3C Pointer Events Working Group, the W3C Touch Events Community Group, and Senior Accessibility Consultant for The Paciello Group.

Conclusion

In this post we covered some of the basics that are currently implemented in Firefox Nightly. To track the progress of this API, check out the Gecko Touch Wiki page. You can also follow along on the main feature bug and be sure to report any issues you find while testing the new Pointer API.

View full post on Mozilla Hacks – the Web developer blog

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Keyboard events in Firefox OS TV

Getting started

The behavior of input events via hardware keys in Firefox OS varies widely from app to app. Early smartphones came with a limited number of keys — Power, Home, Volume up, Volume down — so it was easy for the software to determine an appropriate response for each keypress event. However, Smart TV remotes now come with many hardware keys, and defining the appropriate behavior when a key is pressed has become an important issue on the Firefox OS TV platform. If a hardware key on a smart remote can be used both by apps and by the system, it’s important to determine which response is triggered when the key is pressed.

This post will introduce the challenges of programming a TV remote to manage keyboard events on the Firefox OS Smart TV platform. We’ll classify keyboard events into four scenarios and describe these dispatch scenarios and how they interact. This is the first of two posts about keyboard events for Firefox OS Smart TV. Next week, in Part 2, we’ll drill down into the implementation details and and take a look at sample code for event handling on TV remote.

f1

[Fig 1]

We begin with the ‘Info’ key on a TV remote. Often, it’s used by the hardware to display system information, although it’s possible for an application to use the same key to display app information. When a user presses the key, what action will be shown on screen — system info or app info?

Four keyboard event scenarios

To determine the appropriate behavior when hardware keys are pressed, we start by describing four scenarios for keyboard events.

Scenario Description Event order
SYSTEM-ONLY For keys which should be handled by mozbrowser-iframe-host-page only. system
SYSTEM-FIRST For keys which can be handled by mozbrowser-iframe-host-page first and can then also be handled by mozbrowser-iframe-embedded-page. system->app
APP-CANCELLED For keys which should be handled by the mozbrowser-iframe-embedded-page only. app
APP-FIRST For keys which can be handled by mozbrowser-iframe-embedded-page first and can then also be handled by mozbrowser-iframe-host-page. app->system

[Table 1]

[Fig 2]

[Fig 2]

The mozbrowser-iframe-host-page and mozbrowser-iframe-embedded-page mentioned in Table 1 are illustrated in Fig.2 above. If A.html represents a host page whose source is B.html, then A.html is the mozbrowser-iframe-host-page, and B.html is mozbrowser-iframe-embedded page. mozbrowser uses the non-standard Firefox Browser API, built for the implementation of key features and content experiences in Firefox OS apps. Learn more about mozbrowser on MDN.

Suitable responses for any given keyboard event depend on the scenario. In the case illustrated above, let’s suppose that the ‘Info’ keyboard event is categorized as APP-FIRST and the default action set by system is to show system information. Thus, when we press the ‘Info’ key with app Z in the foreground, there are two possible results:

    1. If app Z has an event handler that tells the ‘Info’ key to show app information, then app information will appear on screen when the user presses the ‘Info’ key on the remote.
    2. If app Z doesn’t set an event handler for the ‘Info’ key, the default action is triggered. That is, the screen will show the system information.

How to implement the four scenarios

To implement the four keyboard event scenarios described above, we’ve introduced four new keyboard events:

      • mozbrowserbeforekeydown – fired before the keydown event
      • mozbrowserafterkeydown – fired after the keydown event
      • mozbrowserbeforekeyup – fired before the keyup event
      • mozbrowserafterkeyup – fired after the keyup event

These four keyboard events are only received by the window that embeds a mozbrowser-iframe.

The keyboard events occur in a specific sequence over time: mozbrowserbeforekeydown, mozbrowserafterkeydown, mozbrowserbeforekeyup, keyup, mozbrowserafterkeyup.

This gives developers a way to implement the four scenarios mentioned above. Conceptually, the scenarios SYSTEM-ONLY, SYSTEM-FIRST and APP-CANCELLED, APP-FIRST can be implemented by setting proper handlers for the mozbrowserbeforekey* and mozbrowserafterkey* events. The SYSTEM-ONLY and SYSTEM-FIRST scenarios can be implemented by setting proper handlers for mozbrowserbeforekey* events and the APP-CANCELLED and APP-FIRST scenarios can be implemented via mozbrowserafterkey* events.

iframe structure in Firefox OS

[Fig 3]

[Fig 3]

To understand how to implement the four scenarios, let’s first take a look at iframe structure in Firefox OS. The outermost iframe in Firefox OS is shell.html. It embeds an in-process iframe which is sourced from system/index.html. The system app (system/index.html) contains several web apps (essentially iframes) which can be in-process (remote=”false”) or out-of-process (remote=”true”). As a result, the relationship of these three layers can be displayed in the following table:

mozbrowser iframe host page mozbrowser iframe embedded page
shell.html system/index.html
system/index.html web apps(essentially iframes)

[Table 2]

Dispatch order of keyboard events

[Fig 4]

[Fig 4]

When a keydown event is sent to some element in a mozbrowser-iframe-embedded-page, the owner of the embedded iframe, i.e., the mozbrowser-iframe-host-page, will receive the mozbrowserbeforekeydown event before the keydown event is sent and the mozbrowserafterkeydown event after the event is sent to the mozbrowser-iframe-embedded-page.

In Gecko, once there is one keydown event with the target in an out-of-process iframe embedded in an HTML document, the keydown event is duplicated to the HTML document as well. The target of this duplicated event is set as the embedded iframe element.

This results in the keyboard event sequence shown in Fig 4. It illustrates all related keydown events and their relationship when a keydown event with a target in a mozbrowser-iframe-embedded-page needs to be dispatched. In brief, events follow this sequence:

      1. Before dispatching any keydown event, the mozbrowserbeforekeydown event is first dispatched to the window of mozbrowser-iframe-host-page.
      2. Then, the original keydown event (with a target in a mozbrowser-iframe-embedded-page) will be duplicated to the mozbrowser-iframe-host-page HTML document. Its target will be set to be the iframe that contains the mozbrowser-iframe-embedded-page.
      3. Next, the original keydown event will be dispatched to its target.
      4. After the original keydown event dispatch is complete, the mozbrowserafterkeydown event will be dispatched to the window of mozbrowser-iframe-host-page.

Notice that the event dispatch process described above follows the DOM tree event flow. Event sequence and event targets can be organized into the following table:

Order event target
1 mozbrowserbeforekeydown window in mozbrowser-iframe-host-page
2 keydown iframe that contains the mozbrowser-iframe-embedded-page in mozbrowser-iframe-host-page
3 keydown original one in mozbrowser-iframe-embedded-page
4 mozbrowserafterkeydown window in mozbrowser-iframe-host-page

[Table 3]

f5

[Fig 5]

The keyboard events mozbrowserbeforekeydown, keydown, and mozbrowserafterkeydown, can be extended to nested mozbrowser iframes, like the iframe structure in Firefox OS described in Table 2. In this case, the mozbrowserbeforekeydown and mozbrowserafterkeydown events will be dispatched to the innermost mozbrowser-iframe-host-page as well as the outer one. Thus, in Firefox OS, mozbrowserkeydown and mozbrowserafterkeydown will be dispatched to the window of system/index.html and the window of shell.html. Fig. 5 illustrates the whole dispatch sequence of related events when a keydown event is dispatched to a web app. The sequence of events is demonstrated in Table 4.

Order event target
1 mozbrowserbeforekeydown window in shell.html
2 mozbrowserbeforekeydown window in system/index.html
3 keydown iframe that contains the web app in system/index.html
4 keydown original one in web app
5 mozbrowserafterkeydown window in system/index.html
6 mozbrowserafterkeydown window in shell.html

[Table 4]

f6

[Fig 6]

Although the keyup event must be fired after keydown, the keydown event and the keyup event are independent of each other. Moreover, the path mozbrowserbeforekeyup, keyup, mozbrowserafterkeyup is independent of the path mozbrowserbeforekeydown, keydown, mozbrowserafterkeydown. Therefore, it’s possible for these two paths to cross each other. The mozbrowserbeforekeyup may arrive before the keydown event.

In Firefox OS, most apps run out-of-process. This means that the app runs on its own process, not on the main process. After dispatching a given key* event (the duplicate) to the system app, it takes time to send the original key* event to the process where the mozbrowser-iframe-embedded-page is located. In a similar manner, after a given key* event is dispatched to the mozbrowser-iframe-embedded-page’s process, time is required to send mozbrowserafterkey* back to the process where the mozbrowser-iframe-host-page is located.

Consequently, the mozbrowserbeforekeyup event may arrive in the main Firefox OS process (where the system app lives), before the keydown event is dispatched to the app’s own process. Common results of the order of the key* events are demonstrated above in Fig. 6. The yellow series represents the keydown path, and the blue series show the keyup path. And yes, these two paths may cross each other.

Next steps

If you’re looking for implementation details for the four scenarios we’ve described, as well as event handler sample code to help with your implementation of keyboard events for smart TV remotes, keep an eye for next week’s post, and stay tuned for more complete documentation arriving very soon on MDN.

View full post on Mozilla Hacks – the Web developer blog

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Ambient Light Events and JavaScript detection

I think that one of the most interesting things with all WebAPIs we’re working on, is to interact directly with the hardware through JavaScript, but also, as an extension to that, with the environment around us. Enter Ambient Light Events.

The idea with an API for ambient light is to be able to detect the light level around the device – especially since there’s a vast difference between being outside in sunlight and sitting in a dim living room – and adapt the user experience based on that.

One use case could be to change the CSS file/values for the page, offering a nicer reading experience under low light conditions, reducing the strong white of a background, and then something with more/better contrast for bright ambient light. Another could be to play certain music depending on the light available.

Accessing device light

Working with ambient light is quite simple. What you need to do is apply a listener for a devicelight event, and then read out the brightness value.

It comes returned in the lux unity. The lux value ranges between low and high values, but a good point of reference is that dim values are under 30 lux, whereas really bright ones are 10,000 and over.

window.addEventListener("devicelight", function (event) {
    // Read out the lux value
    var lux = event.value;
    console.log(lux);?    
});

Web browser support

Ambient Light Events are currently supported in Firefox on Android, meaning both mobile phones and tablets, and it’s also supported in Firefox OS. On Android devices (the ones I’ve tested), the sensor is located just right to the camera facing the user.

It is also a W3C Working Draft, following the type of other similar events, such as devicemotion, so we hope to see more implementations of this soon!

Demo

Dmitry Dragilev and Tim Wright recently wrote a blog post about the Ambient Light API, with this nice demo video:

You can also access the demo example directly, and if you test in low light conditions, you’ll get a little music. Remember to try it out on a supported device/web browser.

View full post on Mozilla Hacks – the Web developer blog

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Aurora 18: HiDPI & Touch Events

In this Aurora cycle, Firefox gained several new interesting features:

  • HiDPI displays are now supported (like Mac’s Retina)
  • Performance and Memory improvements (with IonMonkey and a huge improvement for people using HTTP proxies)
  • Web standard support: HTML5, JS & CSS new features, with the full support of the W3C Touch events.

Let’s look at this in more details!

Some new HiDPI iconsMac users with a Retina displays will be pleased to have an adapted Firefox: support for HiDPI screens has landed and it is gorgeous. Not only the text is using the whole resolution, but all icons have been redone.

A lot of performance improvements have landed during this cycle:

  • IonMonkey is the codename for the latest JavaScript improvements, and that’s a huge boost in our performance.
  • CSS Gradients use a new caching mechanism that leads to speed gains on page heavily using them.
  • Also work on reactivity led us to lower our start-up times: an unnecessary certificate validation has been removed, but the most significant gains will be seen by people using Firefox behind a proxy, like a lot of business users.
  • Finally, tab switching is quicker as Jared Wein replaced a costly regular expression by more efficient code.

Memory management continues to get some significant work, though most progress is seen on special cases:

  • We introduced coalescing of inline style rules across nodes, when possible, leading to a significant reduction (up to 20%) of memory on pages with a lot of nodes.
  • Optimizations of our event queues led to less memory used by pages with a lot of HTML and CSS errors (when the console is closed).
  • Also tabs not yet restored are using less memory.

As expected DThe new preset of the Responsive Design Vieweveloper Tools see steady improvements. Paul Rouget made some nice videos of these features. Of special notice we have:

  • The live preview of the DOM Tree is available.
  • The Responsive design view has now presets and the scroll bars behave more similarly of how they do on mobile phones.
  • On Mac OS, the Scratchpad now supports the native fullscreen feature.
  • New commands, dbg open & dbg close, have been added to our Command line.

Implementation of CSS3, HTML5 & JavaScript features continues at a quick pace:

  • The W3C Touch events are now implemented on Windows too. That means that our experimental MozTouch event is no more necessary on any platform and should be replaced by the standard way of doing them. We plan to remove support of these deprecated events very soon.
  • The HTML5 reversed attribute of the <ol> element is implemented.
  • The EcmaScript 6 proxies landed.
  • WebWorkers now supports transferable objects.
  • CSS Animations have been adapted to the latest evolution of the CSS cascade.
  • SVG Glyphs can now be integrated into OpenType fonts.

Like always, this cycle brought its fair share of novelties. And more to come in the future: CSS3 Flexible boxes layout landed and is in Aurora, but the implementation is still experimental and behind a pref, layout.css.flexbox.enable. Not yet ready for prime time, but we are getting close. Similarly WebRTC is available behind a pref, and feedback is welcomed.

Exciting time ahead for Web developers.

View full post on Mozilla Hacks – the Web developer blog

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Dev Derby February – working with touch events

It’s February and time for our next Dev Derby! Dev Derby is a part of Mozilla Developer Network (MDN), and each of them are focused on a certain technology where people can submit their demos.

This month, we want to see what you can do with touch events! If you need help to get started, we recommend reading more about touch events on MDN to know how they work and what you have available.

Prizes

Android mobile device
Winner gets an Android mobile device from Motorola or Samsung.
Rickshaw laptop bag
Runner-up gets a hand-crafted laptop messenger bag from Rickshaw.
MDN t-shirt
3rd place gets a limited edition MDN t-shirt to show off their geek cred.

And as if that wasn’t enough, your demo will be showcased in Mozilla’s Demo Studio, and we would like to feature you in an article here on Mozilla Hacks as well!

Go for it!

Submit your demo now!

View full post on Mozilla Hacks – the Web developer blog

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

TTMMHTM: Fiddle with JSFiddle, stop looking for ‘Likes’, conference book and Mozilla Events

Things that made me happy this morning:


View full post on Christian Heilmann’s blog – Wait till I come!

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)