A Social-aware Dashboard Experience with Gecko in Walls

This article covers a Web project that rethinks how TV appliances can be used in public spaces for real-time content and interaction with social networks. Tela Social is a powered by Mozilla system application that runs in Linux appliances and creates a visual experience that presents custom and interactive content with real time data using the Web. It’s different of a standard kiosk and far from the personal browser experience because it’s not tailored for an individual. Instead, it explores how online data can be used to provide impact to users in common areas. In the following sections, you will read about the general system’s architecture; a special focus is given to how CSS3 and JavaScript is used to support the user experience with engaging animated effects.

First, boot to the Web

The application starts following the Linux system’s reboot. A monitor process is launched and starts to check whether the Gecko-based client is running or not. If not running, it starts the main process: a Gecko-based single window browsing engine with no toolbars in full screen.

The following image shows an overview of the web stack infrastructure that is launched. The web application sits above the engine in a similar way to applications such as Firefox or other Powered by Mozilla apps using Gecko.

Gecko architecture overview

Online and Offline

Instead of loading a remote web page, most of the resources are loaded from a local web server written in JavaScript using NodeJS — TelaSocial Mediator. This system is also monitored in order to reduce problems in the case of script execution failures.

Once the web app comes from a local web server, a main HTML layout may bring inner HTML components that are loaded within iframes. This degree of separation exists to help developers to create isolated components in a standard test environment: all the components are made of plain pages. As part of this project, an experimental project script was created to help developers to quickly deploy new grid layouts — Gridtype.JS can take strings as arguments and can generate a grid layout that is tableless. The following example shows how to generate a grid of DIVs:


The above “string” specification will generate the following layout:

Layout design

Free of pop-ups or unwanted desktop interfaces

There are many systems using online data to display information in TV appliances. Some of these systems are applications launched on top of the operating system graphic user interface. The following photos were taken from airports in 2012 in Brazil. These are a few of many examples of problems with unwanted user interface elements that are displayed on top of systems using operating system desktops. For this reason, and also the principle of simplicity and cost reduction, the designed approach was to launch Gecko from the basic X Window infra structure. Such approach ensures that no other visual applications are running at the same time, thus creating an improved quality experience without those unwanted pop-ups.

Picture shows an Operating System update pop up which is common in airports in Brazil

Picture shows an Operating System update pop up which is common in airports in Brazil

Routing Gecko pop-ups into frames

While OS desktop pop-ups are out of the scene, yet there are some conditions where Gecko used to display pop-ups to alert the user. The solution to this problem was to use Gecko’s preferences API and ask Gecko to display network error messages inline. With this, all errors are displayed in the iframe which is great because it is possible to track the errors and signal application logic outside the Gecko process. As an example, we can monitor the output console and change the layout depending on network error conditions. So, for example, if a weather channel frames brings a network error, the system should be able to launch an alternate layout.

Animation library with zoom and pan

The above elements are essential to keep the level of quality that is expected in enterprise environments. With this functional base, it’s when things can be creative and the web developer mindset rules. HTML5 and modern effects, provided by CSS3, can be used to help understand and improve the user experience in this scenario that is not the same of a desktop or a mobile.

To help the creation of content, a small JavaScript library was also produced as part of this project. The main goal of this library is to help the creation of visual narratives out of HTML pages. The TagVisor library reads a list of <li> elements which tells how CSS3 and other transformation effects are applied with time-referenced data:

<ul id="animation" style="display: none;">
   <li data-target="slidea" data-time="1s"
      data-effect="scalefit" data-duration="3s">
   <li data-target="slideb" data-time="6s" 
      data-effect="scalefit" data-duration="3s">
   <li data-target="slidec" data-time="12s"
      data-effect="scalefit" data-duration="3s">

The above script dispatches transformation functions that can modify the document using JavaScript for DOM transformations and DOM style changes that will take care of the visual effects. These effects are common in many systems, including well known JS-based solutions such as Impress.JS. However TagVisor comes with tailored functions that are very important for the scenarios in digital signage. One difference is that the script allows modifications to be performed within iframes which is good for cases where many separated HTML resources are loaded in a grid arrangement.

The zoom and animation metaphor does a lot of the heavy work to ensure a level of smoothness in animation which is expected by users in front of a TV panel. Some effects can also be combined in time in order to produce engaging visual narratives as shown in the TagVisor video demonstration.

Web-driven, for adaptation

User experience is of major importance and a main reason for the use of a web-based infrastructure. Such model brings a level of dynamism supporting organisations who can tailor the system with custom data-sources using web services formats such as JSON or RSS. This approach has allowed us to test the screens with a variety of online sources; and to learn how the stream of web events can impact users in local organisations.

The following videos are examples of systems with Tela Social. The first video refers to the system at a major science park in Brazil (Press release in Portuguese). It’s an example with a variety of content areas that are tied to a variety of sources: from journalists to editors plus open channels using social networks. The second video shows an example where the TV appliance is set in vertical position.

People recognise that using data from events plus local interest content can build local participation. For most people, TV content is something far away and not tailored to their local community interests. To make things different, we have even used TVs in the vertical position to make a point that the whole experience was flipped and that content and interaction is bottom up.

The objective of the project is to develop web-based solutions that can be useful in spaces and bring social-aware experiences that are positive to local communities . While social features can be used in the mobile and on desktop, we believe that our approach can help communities because it extends the physical environment — it creates a social reflection to the environment. It’s a chance for custom web experiences that build awareness about contributions made by local communities and real-time data of interest. This approach also explores challenges in user interfaces since it engages people to interact in new ways: it’s not a direct touch like standard kiosks. It’s the web in the first place and opportunities for interaction are explored based on live data.

View full post on Mozilla Hacks – the Web developer blog

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Mobile World Congress, Boot to Gecko and the unknown beast called HTML5

Last week I was a bit more silent than usual (some said I appeared to be human rather than a news/tech/kittens/puppies feed). The reason was that I was at mobile world congress in Barcelona, Spain and got roped into doing booth duty on top of my presentation there.

Mothereffing Boot to Gecko!

The main reason though was that I finally got my hand on a boot to gecko phone and not the best horses or king’s men could stop me from showing it to anyone who was even marginally interested.

The presentation that made the rounds was the one on Engadget:

So before going back to my MWC experience, here are the quick points about this:

  • Boot to Gecko is a fully open phone stack
  • It has a Linux core and its Gaia interface is written in HTML5 – so there is no need to learn any Java or C++ to write apps for it or customise the OS itself
  • The phone features a full telephony and messaging stack
  • It also runs WebGL with 60fps smoothly
  • By the end of the year Telefonica want to release a phone running it and we are in talks with others to do the same (I am off to Hannover on Wednesday for Cebit for that)
  • As the OS is incredibly light-weight it boots in 12 seconds:

In other words: it kicks ass and it allows me as a web developer to build things for phones with the same technologies that I build web sites in. The interesting thing was that this option doesn’t transpire that easily to mobile developers.

HTML5 concerns of mobile developers and entrepreneurs

Whilst showing off mobile Firefox on various tablets and mobile phones I got a lot of feedback and questions about HTML5 from people. They were incredibly interesting as we assume as web developers that mobile HTML5 development is easier – as you know the browser that is in use if you target a certain phone. However, I realised that web development and mobile development are still very much different worlds. We had the same experience during a discussion round at the WIP Jam. I was lucky enough to partner with Wolfram Kriesing of uxebu to answer the concerns of mobile developers. Some of those made it into my talk.

  • When is HTML5 ready for use? This was a very big one. Coming from a world of IDEs, SDKs and builds this is a predictable question but as web developers we know that our tech is constantly moving (I am writing a big post/talk on that soon)
  • How can I protect my code? A common concern was that HTML5/CSS and JS are open and can be simply copied. So a lot of entrepreneurs wondered how they can stop people from doing that. My advice was to go to the Android store and search for “Angry” and see that the same happens in native code. The energy you put into protecting your code just to find it ripped you might as well put into creating a great user experience and let your lawyers worry about plagiarism.
  • Do you need to be online to use HTML5 games? It was amazing how unknown the local storage options of HTML5 are. Again, I showed the native Angry Birds seasons which loads each and every level and thus is pointless offline. It is not a problem of the technology, it means you implemented it wrongly.
  • What tools do I use to build HTML5? Another big one. Whilst as web developers we are very happy to go into the code in a text editor, it seems alien to a lot of developers out there. Which means that tooling is a very necessary task for us HTML5 aficionados right now and it is cool to see uxebu and cloud9ide going there.

Reacting to question and presenting on HTML5

During the WIP Jam at MWC Joe Stagner and me gave a 1.5 hour “workshop” on HTML5 and sat on the table all day answering questions (and handing out a lot of shirts and stickers). I recorded our talk and – despite a few technical issues with the projector and the room giving a good impression of the boiler room of the Titanic (32 degrees) it went really well:

Slides for HTML5 and friends

Noteworthy is that by using Mortadelo y Filemón in my slides, I made quite a few people happy, so happy actually that @martuishere got me a copy of one of their books (which I had in German when I grew up).

The rocky road ahead

All in all we have quite a rocky road ahead of us if we want to get the mobile world to embrace HTML5. A lot of it is not really technical issues but have to do with attitude and environments. In the past we were lucky as web developers as nobody understood and cared what we did. Now everybody wants to play in our sandbox but they expect laser guided shovels and shiny clean sand that smells of roses rather than getting their fingers dirty diving deep. A big one was comparing HTML5 to native code – something that is limiting web development a lot. Of course a native app is snappier. A formula one car is faster than a hovercraft, but it fails miserably on water. Great web apps should change and adapt to their environments – something that is very uncommon in native code. Funnily enough only developers played with B2G and said it is a bit laggy. Users of phones and managers didn’t realise any difference – actually they asked if it is a windows phone as our Gaia app icons are rectangular rather than rounded squares.

The conference as a whole

All in all I dreaded the conference. It is an overpriced, hyped sales show complete with everything I hate about conferences: booth babes who smile and show a lot of flesh rather than knowledge, tons of printed out marketing material to throw away a day later, amazingly stupid taglines (as reported by The Register with my favourite being “Applying Thought”), bad catering (I more or less lived on omelette sandwiches), the whiff of lots and lots of money being spent all over the place, and a big pile of business cards for me to wade through which could have been quick emails we could have sent each other from our fancy phones (they do have that feature).

That said, some of the people there really made it worth while – most of them I knew beforehand, but I met a lot of locals that I will so come back to.

I love working for Mozilla

The main thing that made it worth while for me to go were my colleagues in Mozilla. The B2G and Mobile Firefox team did not sleep at all getting the demos and the system ready, everybody was switched on 100% of the time and happy to help each other and the people not at the conference did a grand job blogging our exploits (or linking to the open tech we talked about), monitoring the press and answering comments. It was exhausting, taxing and a lot of work to do, but it worked out and it is incredible to see that a conference I thought Mozilla would be very alien at and belittled as “those web guys” got rocked to its core by Boot to Gecko and suddenly a lot more people start believing when we say “the web is the platform”.

View full post on Christian Heilmann

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Announcing Boot to Gecko (B2G) – Booting to the Web

Mozilla recently announced the Boot to Gecko (B2G) Project which is a project towards the goal of building a complete, standalone operating system for the open web.

The aim of B2G

The idea is that open web technologies can supersede single vendor control over application environments, and instead use something that will be open for all and consistent across the board. The first main aim is mobile/tablet devices and to be able in such an environment to give access through web technologies to all the capabilities native applications have.

The project is in a very early stage right now, but we believe in communicating this early and getting as much valuable input, help and suggestions as possible to make this out to be all it can be.

Areas of work

The main areas we see right now that will need a lot of work and input are:

New web APIs
Build prototype APIs for exposing device and OS capabilities to content (Telephony, SMS, Camera, USB, Bluetooth, NFC, etc.)
Privilege model
Making sure that these new capabilities are safely exposed to pages and applications
Prototype a low-level substrate for an Android-compatible device
Choose and port or build apps to prove out and prioritize the power of the system.

Helping out with B2G

Let me re-emphasize that the work with B2G has just begun. As you understand, the possibilities, work and collaboration being needed here is huge, and being open about the whole process, we would love to get your thoughts, suggestions, ideas, practical help or anything else you can think of to make this happen!

Feel free to comment below or add your takes to the Booting to the Web thread. Please read more in the B2G page about what you can do to contribute.

FAQ about B2G

To cover some common ground and questions, here are the Frequently Asked Questions for B2G:

What is Boot to Gecko?

Boot to Gecko (B2G) is an early-stage, exploratory project with the goal of building a complete, standalone operating system for the open web. It is not a product offering, but if successful, could form the basis for one.

When can we expect to see something?

We’re very early in the project, soliciting suggestions and contributions from a lot of people. As we have more specific estimates for different pieces of functionality, they’ll be shared widely.

What is the size of the team working on this project?

It’s very small right now: just 3 people working part time, but we’re looking to ramp up and as an open project we are actively inviting participation of developers, designers, and others from across the Web. We’re seeing lots of excitement and offers of help already, and we’re also obviously leaning heavily on the existing Gecko and Firefox mobile work.

Why are you doing this now?

We believe that the next frontier for Web applications is full device integration, so that Web developers have the same capabilities as those building for OS-specific stacks.

What does it mean for your relationships with Apple, Google, Microsoft?

We don’t expect that it will affect our relationships with other organizations.

Does this replace work that’s already being done on Web APIs for desktop and mobile?

We are already pushing hard on new Web APIs, and have been for some time. We’ll continue to implement and standardize new APIs for Web content while the B2G project ramps up.

How is this different than the Webian Shell project?

The Webian shell is an impressive project even in its early stages. Where Webian is focused on a Web-centric desktop experience, we’re focused on extending the Web to include more of what is traditionally the domain of OS-specific code. We think we can work together on a bunch of things, and we’re looking forward to it.

How is this different from Chrome OS?

We’re aiming at mobile/tablet devices rather than a notebook form factor. This is an early-stage project to expose all device capabilities such that infrastructure like phone dialers can be built with Web APIs, and not only “high level” apps like word processors and presentation software. We will of course be happy to work with the Chrome OS team on standards activities, and indeed to share source code where appropriate.

Are OEMs interested in B2G?

This is an early-stage project. We just got started, so we haven’t had any of those discussions yet. If an OEM shares our vision of a standard and open platform from top to bottom, we’d be happy to work together to get such a platform into the hands of users.

Whose hardware will you support?

We’ll be selecting initial hardware for hackability and general availability, but we haven’t settled on that yet. A Tegra 2 device is likely to be selected, due to its support for VP8 hardware acceleration. Over time we expect that B2G will work on the majority of devices that support modern Android versions.

Will this mean a Firefox Phone?

We don’t have any plans to build or distribute a custom device.

View full post on Mozilla Hacks – the Web developer blog

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)