live

Decoded Chats – first edition live on the Decoded Blog

Over the last few weeks I was busy recording interviews with different exciting people of the web. Now I am happy to announce that the first edition of Decoded Chats is live on the new Decoded Blog.

Decoded Chats - Chris interviewing Rob Conery

In this first edition, I’m interviewing Rob Conery about his “Imposter Handbook“. We cover the issues of teaching development, how to deal with a constantly changing work environment and how to tackle diversity and integration.

We’ve got eight more interviews ready and more lined up. Amongst the people I talked to are Sarah Drasner, Monica Dinculescu, Ada-Rose Edwards, Una Kravets and Chris Wilson. The format of Decoded Chats is pretty open: interviews ranging from 15 minutes to 50 minutes about current topics on the web, trends and ideas with the people who came up with them.

Some are recorded in a studio (when I am in Seattle), others are Skype calls and yet others are off-the-cuff recordings at conferences.

Do you know anyone you’d like me to interview? Drop me a line on Twitter @codepo8 and I see what I can do 🙂

View full post on Christian Heilmann

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Inside the Party Bus: Building a Web App with Multiple Live Video Streams + Interactive Graphics

Gearcloud Labs is exploring the use of open technologies to build new kinds of shared video experiences. Party Bus is a demo app that mixes multiple live video streams together with interactive graphics and synchronized audio. We built it using a combination of node.js, WebSockets, WebRTC, WebGL, and Web Audio. This article shares a few things we learned along the way.

User experience

First, take a ride on the Party Bus app to see what it does. You need Firefox or Chrome plus a decent GPU, but if that’s not handy you can get an idea of the app by watching the example video on YouTube.

Since the app uses WebRTC getUserMedia(), you have to give permission for the browser to use your camera. After it starts, the first thing you’ll notice is your own video stream mounted to a seat on the 3D bus (along with live streams from any other concurrent riders). In most scenes, you can manipulate the bus in 3D using the left mouse (change camera angle), scroll wheel (zoom in/out), and right mouse (change camera position). Also try the buttons in the bottom control bar to apply effects to your own video stream: from changing your color, to flipping yourself upside down, bouncing in your seat, etc.

How party bus uses WebRTC

Party Bus uses WebRTC to set up P2P video streams needed for the experience. WebRTC does a great job supporting native video in the browser, and punching out firewalls to enable peer connections (via STUN). But with WebRTC, you also need to provide your own signaler to coordinate which endpoints will participate in a given application session.

The Party Bus app uses a prototype platform we built called Mixology to handle signaling and support the use of dynamic peer topologies. Note that many apps can simply use peer.js, but we are using Mixology to explore new and scalable approaches for combining large numbers of streams in a variety of different connection graphs.

For example, if a rider joins a bus that already has other riders, the system takes care of building the necessary connection paths between the new rider and peers on the same bus, and then notifying all peers through a WebSocket that the new rider needs to be assigned a seat.

Specifically, clients interact with the Mixology signaling server by instantiating a Mixology object

var m = new Mixology(signalerURL);

and then using it to register with the signaler

m.register(['mix-in'], ['mix-out']);

The two arguments give specific input and output stream types supported by the client. Typing inputs and outputs in this way allows Mixology to assemble arbitrary graphs of stream connections, which may vary depending on application requirements. In the case of Party Bus, we’re just using a fully connected mesh among all peers. That is, all clients register with the same input and output types.

The signaler is implemented as a node.js application that maintains a table of registered peers and the connections among them. The signaler can thus take care of handling peer arrivals, departures, and other events — updating other peers as necessary via callback functions. All communications between peers and the signaler are implemented internally using WebSockets, using socket.io.

For example, when a new peer is registered, the server updates the topology table, and uses a callback function to notify other peers that need to know about the new connection.

m.onPeerRegistered = function(peer) { ... }

In this function, peers designated to send streams initiate the WebRTC offer code. Peers designated to receive streams initiate the WebRTC answer code (as well as provide a callback function onAddStream() to be used when the new input stream is ready).

In the case of Party Bus, it’s then up to the app to map the new video stream to the right seat in the 3D bus model, and from then on, apply the necessary 3D transforms using three.js. Similarly, if a rider leaves the bus, the system takes care of notifying other clients that a peer has exited, so they can take appropriate action to remove what would otherwise be a dead video stream in the display.

Party Bus organizes the “riders” on a bus using an array of virtual screens:

var vsArray = new Array(NUM_SEATS);

After registering itself with Mixology, the app receives a callback whenever a new peer video stream becomes available for its bus instance:

function onAddStream(stream, peerId) {
    var i = getNextOpenSeat();
    vsArray[i] = new VScreen(stream, peerId);
}

The Party Bus app creates a virtual screen object for every video stream on the current bus. The incoming streams are associated with DOM video objects in the virtual screen constructor:

function VScreen(stream, id) {
    var v = document.createElement(‘video’);
    v.setAttribute(“id”, “monitor:+ id);
    v.style.visibility = “hidden”;
    v.src = window.URL.createObjectURL(stream);  // binds stream to dom video object
    v.autoplay =true;
    document.body.appendChild(v);
}

Movie or app?

Party Bus uses three.js to draw a double decker bus, along with virtual screens “riding” in the seats. The animation loop runs about two minutes, and consists of about a dozen director “shots”. Throughout the demo, the individual video screens are live, and can be manipulated by each rider. The overall sequence of shots is designed to change scene lighting and present other visual effects, such as bus thrusters which were created with the particle engine of Stemkoski.

Party Bus is a web app, but the animation is programmed so the user can just let it run like a movie. The curious user may try to interact with it, and find that in most scenes it’s also possible to change the 3D view. However, in shots with a moving camera or bus, we found it necessary to block certain camera translations (movements in x, y, z position), or rotations (turning on x, y, z axis) — otherwise, the mouse will “fight” the program, resulting in a jerky presentation.

But most of the fun in Party Bus is just hamming it up for the camera, applying visual effects to your own stream, and looking for other live riders on the bus.

More info

For more information on the Party Bus app, or to stay in the loop on development of the Mixology platform, please check out www.gearcloudlabs.com.

View full post on Mozilla Hacks – the Web developer blog

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Live Editing Sass and Less in the Firefox Developer Tools

Sass and Less are expressive languages that compile into CSS. If you’re using Sass or Less to generate your CSS, you might want to debug the source that you authored and not the generated CSS. Luckily you can now do this in the Firefox 29 developer tools using source maps.

The Firefox developer tools use source maps to show the line number of rules in the original source, and let you edit original sources in the Style Editor. Here’s how to use the feature:

1. Generate the source map

When compiling a source to CSS, use the option to generate a sourcemap for each style sheet. To do this you’ll need Sass 3.3 (Currently in pre-release, gem install sass --pre) or Less 1.5+.

Sass

sass index.scss:index.css --sourcemap

Less

lessc index.less index.css --source-map

This will create a .css.map source map file for each CSS file, and add a comment to the end of your CSS file with the location of the sourcemap: /*# sourceMappingURL=index.css.map */. The devtools will use this source map to map locations in the CSS style sheet to locations in the original source.

2. Enable source maps in developer tools

Right-click anywhere on the inspector’s rule view or in the Style Editor to get a context menu. Check off the Show original sources option:

Enabling source maps in devtools

Now CSS rule links will show the location in the original file, and clicking these links will take you to the source in the Style Editor:

Original source showing in Style Editor

3. Set up file watching

You can edit original source files in Style Editor tool, but order to see the changes apply to the page, you’ll have to watch for changes to your preprocessed source and regenerate the CSS file each time it changes. To set watching up:

Sass

sass index.scss:index.css --sourcemap --watch

Less

For Less, you’ll have to set up another service to do the watching, like grunt.

4. Save the original source

Save the original source to your local file system by hitting the Save link or Cmd/Ctrl-S:

Saving source to disk

Saving source to disk

The devtools will infer the location of the generated CSS file locally and watch that file for changes to update the live style sheet on the page.

Now when you edit an original source and save it, the page’s style will update and you’ll get immediate feedback on your Sass or Less changes.

The source has to be saved to disk and file watching set up in order for style changes to take effect.

View full post on Mozilla Hacks – the Web developer blog

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Live editing WebGL shaders with Firefox Developer Tools

If you’ve seen Epic Games’ HTML5 port of ‘Epic Citadel’, you have no doubt been impressed by the amazing performance and level of detail. A lot of the code that creates the cool visual effects you see on screen are written as shaders linked together in programs – these are specialized programs that are evaluated directly on the GPU to provide high performance real-time visual effects.

Writing Vertex and Fragment shaders are an essential part of developing 3D on the web even if you are using a library, in fact the Epic Citadel demo includes over 200 shader programs. This is because most rendering is customised and optimised to fit a game’s needs. Shader development is currently awkward for a few reasons:

  • Seeing your changes requires a refresh
  • Some shaders are applied under very specific conditions

Here is a screencast that shows a how to manipulate shader code using a relatively simple WebGL demo:

Starting in Firefox 27 we’ve introduced a new tool called the ‘Shader Editor’ that makes working with shader programs much simpler: the editor lists all shader programs running in the WebGL context, and you can live-edit shaders and see immediate results without interrupting any animations or state. Additionally editing shaders should not impact WebGL performance.

Enabling the Shader Editor

The Shader Editor is not shown by default, because not all the web pages out there contain WebGL, but you can easily enable it:

  1. Open the Toolbox by pressing either F12 or Ctrl/Cmd + Shift + I.
  2. Click on the ‘gear’ icon near the top edge of the Toolbox to open the ‘Toolbox Options’.
  3. On the left-hand side under ‘Default Firefox Developer Tools’ make sure ‘Shader Editor’ is checked. You should immediately see a new ‘Shader Editor’ Tool tab.

Using the Shader Editor

To see the Shader Editor in action, just go to a WebGL demo such as this one and open the toolbox. When you click on the shader editor tab, you’ll see a reload button you will need to click in order to get the editor attached to the WebGL context. Once you’ve done this you’ll see the Shader Editor UI:

The WebGL Shader Editor

  • On the left you have a list of programs, a vertex and fragment shader corresponds to each program and their source is displayed and syntax highlighted in the editors on the right.
  • The shader type is displayed underneath each editor.
  • Hovering a program highlights the geometry drawn by its corresponding shaders in red – this is useful for finding the right program to work on.
  • Clicking on the eyeball right next to each program hides the rendered geometry (useful in the likely case an author wants to focus solely on some geometry but not other, or to hide overlapping geometry).
  • The tool is responsive when docked to the side.

Editing Shader Programs

The first thing you’ll notice about Shader program code is that it is not JavaScript. For more information on how Shader programs work, I highly recommend you start with the WebGL demo on the Khronos wiki and/or Paul Lewis’ excellent HTML5 Rocks post. There also some great long standing tutorials on the Learning WebGL blog. The Shader Editor gives you direct access to the programs so you can play around with how they work:

  • Editing code in any of the editors will compile the source and apply it as soon as the user stops typing;
  • If an error was made in the code, the rendering won’t be affected, but an error will be displayed in the editor, highlighting the faulty line of code; hovering the icon gutter will display a tooltip describing the error.

Errors in shaders

Learn more about the Shader Editor on the Mozilla Developer Network.

Here is a second screencast showing how you could directly edit the shader programs in the Epic Citadel demo:

View full post on Mozilla Hacks – the Web developer blog

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

First video of a Firefox OS series is live

The last weeks I have been busy scripting (and then improvising as always) a series of videos explaining Firefox OS. These are now going live on a weekly basis.

be-the-future

Over on the Mozilla hacks blog, you can now find the first in a series of six videos explaining what Firefox OS is about. Under the description “Firefox OS – the platform HTML5 deserves” (a slogan I used in a few talks and interviews already) these videos are meant to explain a few things:

  • What Firefox OS is
  • How it is different to any other mobile platform
  • What it means for HTML5 as a movement
  • How you can be part of it
  • What its benefits are to you (a stable HTML5 platform with full hardware access aimed at a completely new and huge market of end users)
  • How to get started
  • Where to find documentation and file complaints and enhancement ideas

All in all, we thought a series of videos would be a good way to get the message out that scales better than talks and posts. Each of the videos will be about five minutes long and an interview/conversation between experts and me, namely in this case Daniel Appelquist ( @torgo) from Téléfonica Digital/ W3C and Desigan Chinniah ( @cyberdees) from Mozilla. The videos were shot over a period of two days in the London Mozilla office by Rainer Cvillink who did an amazing job.

Get over to the hacks blog and see the first video now, and feel free to spread it as far and wide as you can. Cheers.

video

View full post on Christian Heilmann

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)