Debugging

Debugging JavaScript – console.loggerheads?

The last two days I ran a poll on Twitter asking people what they use to debug JavaScript.

  • console.log() which means you debug in your editor and add and remove debugging steps there
  • watches which means you instruct the (browser) developer tools to log automatically when changes happen
  • debugger; which means you debug in your editor but jump into the (browser) developer tools
  • breakpoints which means you debug in your (browser) developer tools

The reason was that having worked with editors and developer tools in browsers, I was curious how much either are used. I also wanted to challenge my own impression of being a terrible developer for not using the great tools we have to the fullest. Frankly, I feel overwhelmed with the offerings and choices we have and I felt that I am pretty much set in my ways of developing.

Developer tools for the web have been going leaps and bounds in the last years and a lot of effort of browser makers goes into them. They are seen as a sign of how important the browser is. The overall impression is that when you get the inner circle of technically savvy people excited about your product, the others will follow. Furthermore, making it easier to build for your browser and giving developers insights as to what is going on should lead to better products running in your browser.

I love the offerings we have in browser developer tools these days, but I don’t quite find myself using all the functionality. Turns out, I am not alone:

The results of 3970 votes in my survey where overwhelmingly in favour of console.log() as a debugging mechanism.

Twitter poll
Poll results: 67% console, 2% watches, 15% debugger and 16% breakpoints.

Both the Twitter poll and its correlating Facebook post had some interesting reactions.

  • As with any too simple poll about programming, a lot of them argued with the questioning and rightfully pointed out that people use a combination of all of them.
  • There was also a lot of questioning why alert() wasn’t an option as this is even easier than console().
  • There was quite some confusion about debugger; – seems it isn’t that common
  • There was only a small amount of trolling – thanks.
  • There was also quite a few mentions of how tests and test driven development makes debugging unimportant.

There is no doubt that TDD and tests make for less surprises and are good development practice, but this wasn’t quite the question here. I also happily discard the numerous mentions of “I don’t make mistakes”. I was pretty happy to have had only one mention of document.write() although you do still see it a lot in JavaScript introduction courses.

What this shows me is a few things I’ve encountered myself doing:

  • Developers who’ve been developing in a browser world have largely been conditioned to use simple editors, not IDEs. We’ve been conditioned to use a simple alert() or console.log() in our code to find out that something went wrong. In a lot of cases, this is “good enough”
  • With browser developer tools becoming more sophisticated, we use breakpoints and step-by-step debugging when there are more baffling things to figure out. After all, console.log() doesn’t scale when you need to track various changes. It is, however, not our first go-to. This is still adding something in our code, rather than moving away from the editor to the debugger
  • I sincerely hope that most of the demands for alert() were in a joking fashion. Alert had its merits, as it halted the execution of JavaScript in a browser. But all it gives you is a string and a display of [object object] is not the most helpful.

Why aren’t we using breakpoint debugging?

There should not be any question that breakpoint debugging in vastly superior to simply writing values into the console from our code:

  • You get proper inspection of the whole state and environment instead of one value
  • You get all the other insights proper debuggers give you like memory consumption, performance and so on
  • It is a cleaner way of development. All that goes in your code is what is needed for execution. You don’t mix debugging and functionality. A stray console.log() can give out information that could be used as an attack vector. A forgotten alert() is a terrible experience for our end users. A forgotten “debugger;” or breakpoint is a lot less likely to happen as it does pause execution of our code. I also remember that in the past, console.log() in loops had quite a performance impact of our code.

Developers who are used to an IDE to create their work are much more likely to know their way around breakpoint debugging and use it instead of extra code. I’ve been encountering a lot of people in my job that would never touch a console.log() or an alert() since I started working in Microsoft. As one response of the poll rightfully pointed out it is simpler:

So, why do we then keep using console logging in our code rather than the much more superior way of debugging code that our browser tooling gives us?

I think it boils down to a few things:

  • Convenience and conditioning – we’ve been doing this for years, and it is easy. We don’t need to change and we feel familiar with this kind of back and forth between editor and browser
  • Staying in one context – we write our code in our editors, and we spent a lot of time customising and understanding that one. We don’t want to spend the same amount of work on learning debuggers when logging is good enough
  • Inconvenience of differences in implementation – whilst most debuggers work the same there are differences in their interfaces. It feels taxing to start finding your way around these.
  • Simplicity and low barrier of entry – the web became the big success it is by being independent of platform and development environment. It is simple to show a person how to use a text editor and debug by putting console.log() statements in your JavaScript. We don’t want to overwhelm new developers by overloading them with debugger information or tell them that they need a certain debugging environment to start developing for the web.

The latter is the big one that stops people embracing the concept of more sophisticated debugging workflows. Developers who are used to start with IDEs are much more used to breakpoint debugging. The reason is that it is built into their development tools rather than requiring a switch of context. The downsides of IDEs is that they have a high barrier to entry. They are much more complex tools than text editors, many are expensive and above all they are huge. It is not fun to download a few Gigabyte for each update and frankly for some developers it is not even possible.

How I started embracing breakpoint debugging

One thing that made it much easier for me to embrace breakpoint debugging is switching to Visual Studio Code as my main editor. It is still a light-weight editor and not a full IDE (Visual Studio, Android Studio and XCode are also on my machine, but I dread using them as my main development tool) but it has in-built breakpoint debugging. That way I have the convenience of staying in my editor and I get the insights right where I code.

For a node.js environment, you can see this in action in this video:

Are hackable editors, linters and headless browsers the answer?

I get the feeling that this is the future and it is great that we have tools like Electron that allow us to write light-weight, hackable and extensible editors in TypeScript or just plain JavaScript. Whilst in the past the editor you use was black arts for web developers we can now actively take part in adding features to them.

I’m even more a fan of linters in editors. I like that Word tells me I wrote terrible grammar by showing me squiggly green or red underlines. I like that an editor flags up problems with your code whilst you code it. It seems a better way to teach than having people make mistakes, load the results in a browser and then see what went wrong in the browser tools. It is true that it is a good way to get accustomed to using those and – let’s be honest – our work is much more debugging than coding. But by teaching new developers about environments that tell them things are wrong before they even save them we might turn this ratio around.

I’m looking forward to more movement in the editor space and I love that we are able to run code in a browser and get results back without having to switch the user context to that browser. There’s a lot of good things happening and I want to embrace them more.

We build more complex products these days – for good or for worse. It may be time to reconsider our development practices and – more importantly – how we condition newcomers when we tell them to work like we learned it.

View full post on Christian Heilmann

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Developer Edition 47 – User agent emulation, popup debugging and more

This week marks the release of Firefox Developer Edition 47! In recent weeks, we’ve covered the DevTools reload add-on and service worker tooling, so be sure to check out those posts. In this post, we’ll cover the rest of the updates and changes in this Developer Edition release.

User Agent emulation

We have added the ability to emulate a custom user agent on any webpage from responsive mode. You can now simply type your new user agent string inside the “Custom User Agent” field. You can use this to check whether a site uses user agent sniffing. For example, you can type in a mobile browser user agent to see how a website would look on your phone.

Here is a screencast of the feature:
User Agent Emulation

Retaining paths as graph

In the previous release, we added a dominator view to help you debug memory-intensive applications. In this version, we’ve improved it by adding a retaining paths panel, which gives you a graph displaying all the nodes that are preventing a selected node from being garbage collected. This is particularly useful when you are debugging a possible memory leak. You can read more about this feature in the MDN documentation here.

Here is a screenshot of the graph:

Console multi-line input

The way multi-line input in the console is handled has been improved. When pressing the “Enter” key, the console will now detect whether your input is complete or not. If it is, pressing “Enter” will simply execute your command. If not, pressing “Enter” will add a new line to your input, so you can seamlessly continue writing your command.

Storage Inspector

The storage inspector now has support for cache storage, which is very useful if you’re debugging a service worker. Be sure to check Sole Penadés’ blog post that dives into the details of service worker debugging.

In addition to cache storage support, you can now filter the contents of the table using the search box located on the top toolbar. Here’s a screenshot of the feature:

Theme changes

In this release, we gave a visual tune-up to the toolbox. We have made small tweaks such as reducing the default tab width and adding new icons inside the memory tool, but we have also made some major changes. For example, we gave the light theme a complete makeover, to give it a lighter and more polished look and feel.

Here is a screenshot of the new light theme:
Refreshed light theme

We have also updated the debugger gutter style—conditional breakpoints are now highlighted in the gutter in orange. Here’s a screenshot:
New debugger gutter

Finally, we have moved the network monitor toolbar to the top to be more accessible and more consistent with the other tools.
Network monitor toolbar

Debugging popups for addons

In preparation for the release of WebExtensions, we are adding new features that will make add-on debugging easier. In this release, we’ve made inspecting popups much easier. You can now lock popups so they don’t disappear once you click away. To use this feature, you need to launch the Browser Toolbox and click the icon with the four squares located in the top-right corner of the toolbox. You can read more about how to debug your extensions here.

Here is a screencast showing this feature in action:

Other notable changes

In addition to the improvements above, we have polished various areas throughout the toolbox, in particular:

We have also removed the 3D view, since this feature conflicts with the multi-process version of Firefox. If you’d like to use the feature, you can install this add-on instead.

Finally, the font inspector has been disabled by default, since it will be reworked for future releases. You can re-enable the tool by toggling the devtools.fontinspector.enabled preference in about:config.

Thanks to everyone that has contributed to this Developer Edition release! Grab a copy of the latest Developer Edition now and let us know what you think.

View full post on Mozilla Hacks – the Web developer blog

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Debugging Service Workers and Push with Firefox DevTools

Following the announcement of Web Push in Firefox 44, we’re now delivering the capability to develop and debug service workers and push notifications code with DevTools using Firefox Developer Edition 47.

Here’s a screencast that demonstrates the features described in this post:

Or if you prefer text, keep reading!

about:debugging

Service workers do not behave exactly as normal web workers, and their lifecycle is different, so we can’t show them alongside normal scripts in the Debugger tab of DevTools.

Instead, we’ve added a new dashboard that will collect all registered service workers and shared workers, amongst other debuggable items such as Add-ons.

Therefore, our debugging adventure starts by going to about:debugging in a new tab, and clicking on the Workers tab on the left.

about:debugging interface

Alternatively, you can access this dashboard by going to the Tools ? Web Developer ? Service Workers menu, or by clicking on the toolbar menu, then Developer, and finally Service Workers.

Accessing about:debugging using the application menuAccessing about:debugging with toolbar menu

Dashboard instant updates

The first time we access the dashboard “nothing yet” will be displayed under the Service Workers and Shared Workers sections. These sections will be updated automatically as workers get registered. The displayed buttons will change accordingly, showing Push and Debug if the worker is running, or just a Start button if the worker is registered, but inactive.

Try it! Open about:debugging in one window, and navigate to this simple service worker demo in another window. The service worker will be registered and displayed under the Service Workers section. No need for you to reload the dashboard!

Debugging service workers

To debug a service worker, the worker must already be running. Click on the associated Debug button, or Start the worker if it’s not running yet (as long as it has been registered, and thus is in the about:debugging Dashboard).

This will pop up a new window with the code of the service worker. Here you can do all the usual debugging you would expect: setting breakpoints, step-by-step execution, inspecting variables, etc.

Service Worker debugger pop up window

Push notifications

Code that uses the Web Push API can now be debugged as well, by setting a breakpoint in the listener for the push event of the service worker. When the push notification is received, the debugger will stop at the breakpoint.

Debugger stopped at the push event listener

This is very handy, but sometimes notifications can be delayed for reasons outside of our control, or the network might be temporarily unreachable. Luckily, you can still test code that relies on push events, by pressing the Push button on the worker.

This will send a push payload, and in turn, it will trigger the push event pretty much instantly. You can reduce your development time as you won’t have to wait for the server to deliver the push.

Debugging shared workers

There’s also support for debugging shared workers. The most important difference is that they will show up in their own dedicated section in about:debugging.

Debugging requests (and cached requests)

You can also now distinguish normal network requests from requests cached by the worker. These cached requests are displayed as Service Worker in the Transferred column, instead of displaying the amount of transferred data.

Network panel showing cached requests

Requests initiated by service workers can be intercepted and debugged by setting a breakpoint on the fetch event listener.

Stopping at the fetch event

We can inspect data such as the requested url, http headers, etc., by looking at the event object in the variables list when the debugger stops at the breakpoint.

Wrap up

Hopefully, this provides a good overview of the new features we’re working on.

The reference documentation for about:debugging is on MDN. If you want to learn more about service workers, you should check out the guide to Using Service Workers, and, of course, the Service Workers cookbook, which is loaded with great demos and examples.

View full post on Mozilla Hacks – the Web developer blog

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Developer Edition 42: Wifi Debugging, Win10, Multiprocess Firefox, ReactJS tools, and more

Firefox 42 has arrived! In this release, we put a lot of effort into the quality and polish of the Developer Edition browser. Although many of the bugs resolved this release don’t feature in the Release Notes, these small fixes make the tools faster and more stable. But there’s still a lot to report, including a major change to how Firefox works.

Debugging over wifi

Now, with remote website debugging, you can debug Firefox for Android devices over wifi – no USB cable or ADB needed.

Multiprocess is enabled by default

Multiprocess Firefox (aka E10s) has been enabled by default in Developer Edition. When it’s enabled, Firefox renders and executes web-related content in a single background content process. If you experience any issues with addons after updating to Developer Edition 42, try disabling incompatible addons or reverting to a single process mode using about:preferences.

Windows 10 theme support

The Developer Edition theme has a new look in Windows 10 to match the OS styling. Take a look:

Screenshot of the dark Developer Edition theme in Windows 10

Dark Developer Edition theme – Windows 10

Screenshot of the light Developer Edition theme in Windows 10

Light Developer Edition theme – Windows 10

React Developer Tools support for Firefox

If you’re developing with ReactJS, you may have noticed that the React project recently released a beta for their developer tools extension, including initial support for Firefox. While there are no official builds yet of the Firefox version, the source is available on github.

Other notable changes

  • Asynchronous call stacks now allow you to follow the code flow through setTimeout, DOM event handlers, and Promise handlers. (Bug 981514)
  • There is a new configurable Firefox OS simulator page in WebIDE. From here, you can change a simulator to run with a custom profile and screen size, using a list of presets from reference devices. (Bug 1156834)
  • CSS filter presets are now available in the inspector. (Bug 1153184)
  • The MDN tooltip now uses syntax highlighting for code samples. (Bug 1154469)
  • When using the “copy” keyboard shortcut in the inspector, the outerHTML of the selected node is now copied onto the clipboard. (Bug 968241)
  • New UX improvements have landed in the style editor’s search feature. (Bug 1159001, Bug 1153474)
  • CSS variables are now treated as normal declarations in the inspector. (Bug 1142206)
  • CSS autocomplete popup now supports pressing ‘down’ to list all results in an empty value field (Bug 1142206)

Thanks to everyone who contributed time and energy to help the DevTools team in this release of Firefox Developer Edition 42! Each release takes a lot of effort from people writing patches, testing, documenting, reporting bugs, sending feedback, discussing features, etc. You can help set our priorities by sharing constructive feedback and letting us know what you’d like from Firefox Developer Tools.

You can download Firefox Developer Edition now, for free.

View full post on Mozilla Hacks – the Web developer blog

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Compiling to JavaScript, and Debugging with Source Maps

This is a tutorial on how to write a compiler which generates JavaScript as its target language, and maintains line and column meta-data in source maps for debugging. Storing line and column coordinates in a source map allows the end-user of the compiler to debug the source code that they wrote, rather than the ugly, generated JavaScript they are not familiar with.

In this tutorial, we will be compiling a small Reverse Polish Notation, or RPN, language to JavaScript. The language is super simple, and is nothing more than simple arithmetic with variable storage and output capabilities. We are keeping the language simple so that we can focus on integrating source maps with the compiler, rather than language implementation details.

Availability

Initial support for source maps in the debugger is available in Firefox 23 (Aurora at time of writing) with more improvements coming in Firefox 24 (Nightly at time of writing). Chrome DevTools also have support for source maps.

Overview of the Source Language

RPN uses postfix notation, meaning that the operator follows its two operands. One of the benefits of RPN is that as long as we limit ourselves to binary operators, we do not need any parentheses, and do not need to worry about operator precedence.

Here is an example program in our source language:

a 5 =;
b 3 =;
c a b + 4 * =;

This is an equivalent program written in a language which uses infix notation for its arithmetic operators:

a = 5;
b = 3;
c = (a + b) * 4;

Our language will support addition, subtraction, multiplication, division, assignment, and printing. The print operator’s first operand is the value to print, the second operand is how many times to print the value and must be greater than or equal to one:

5 1 print;
# Output:
# 5
 
3 4 print;
# Output:
# 3
# 3
# 3
# 3
 
4 print;
# Syntax error
 
n -1 =;
4 n print;
# Runtime error

Lastly, division by zero should throw an error:

5 0 /;
# Runtime error

Getting Setup

We will be writing our compiler on Node.js, using Jison to generate the parser for our language from a grammar, and using the source-map library to help generate source maps.

The first step is to download and install Node.js if you don’t already have it on your system.

After you have installed Node.js, use its package manager npm to create a new project for the compiler:

$ mkdir rpn
$ cd rpn/
$ npm init .

After the last command, npm will prompt you with a bunch of questions. Enter your name and email, answer ./lib/rpn.js for the main module/entry point, and just let npm use the defaults that it supplies for the rest of the questions.

Once you have finished answering the prompts, create the directory layout for the project:

$ mkdir lib
$ touch lib/rpn.js
$ mkdir -p lib/rpn

The public API for the compiler will reside within lib/rpn.js, while the submodules we use to implement various things such as the lexer and abstract syntax tree will live in lib/rpn/*.js.

Next, open up the package.json file and add jison and source-map to the project’s dependencies:

...
"dependencies": {
  "jison": ">=0.4.4",
  "source-map": ">=0.1.22"
},
...

Now we will install a link to our package in Node.js’s globally installed packages directory. This allows us to import our package from the Node.js shell:

$ npm link .

Make sure that everything works by opening the Node.js shell and importing our package:

$ node
> require("rpn")
{}

Writing the Lexer

A lexer (also known as a scanner or tokenizer) breaks the inputted raw source code into a stream of semantic tokens. For example in our case, we would want to break the raw input string "5 3 +;" into something like ["5", "3", "+", ";"].

Because we are using Jison, rather than writing the lexer and parser by hand, our job is much easier. All that is required is to supply a list of rules that describe the types of tokens we are expecting. The left hand side of the rules are regular expressions to match individual tokens, the right hand side are the snippets of code to execute when an instance of the corresponding token type is found. These tokens will be passed on to the parser in the next phase of the compiler.

Create the rules for lexical analysis in lib/rpn/lex.js:

exports.lex = {
  rules: [
    ["\\s+",                   "/* Skip whitespace! */"],
    ["#.*\\n",                 "/* Skip comments! */"],
    [";",                      "return 'SEMICOLON'"],
    ["\\-?[0-9]+(\\.[0-9]+)?", "return 'NUMBER';"],
    ["print",                  "return 'PRINT';"],
    ["[a-zA-Z][a-zA-Z0-9_]*",  "return 'VARIABLE';"],
    ["=",                      "return '=';"],
    ["\\+",                    "return '+';"],
    ["\\-",                    "return '-';"],
    ["\\*",                    "return '*';"],
    ["\\/",                    "return '/';"],
    ["$",                      "return 'EOF';"]
  ]
};

Writing the Parser

The parser takes the tokens from the lexer one at a time and confirms that the input is a valid program in our source language.

Once again, the task of writing the parser is much easier than it would otherwise be thanks to Jison. Rather than writing the parser ourselves, Jison will programmatically create one for us if we provide a grammar for the language.

If all we cared about was whether the input was a valid program, we would stop here. However, we are also going to compile the input to JavaScript, and to do that we need to create an abstract syntax tree. We build the AST in the code snippets next to each rule.

A typical grammar contains productions with the form:

LeftHandSide ? RightHandSide1
             | RightHandSide2
             ...

However, in Jison we are a) writing in JavaScript, and b) also providing code to execute for each rule so that we can create the AST. Therefore, we use the following format:

LeftHandSide: [
  [RightHandSide1, CodeToExecute1],
  [RightHandSide2, CodeToExecute2],
  ...
]

Inside the code snippets, there are a handful of magic variables we have access to:

  • $$: The value of the left hand side of the production.
  • $1/$2/$3/etc: The value of the the nth form in the right hand side of the production.
  • @1/@2/@3/etc: An object containing the line and column coordinates where the nth form in the right hand side of the production was parsed.
  • yytext: The full text of currently matched rule.

Using this information, we can create the grammar in lib/rpn/bnf.js:

exports.bnf = {
  start: [
    ["input EOF", "return $$;"]
  ],
  input: [
    ["",           "$$ = [];"],
    ["line input", "$$ = [$1].concat($2);"]
  ],
  line: [
    ["exp SEMICOLON", "$$ = $1;"]
  ],
  exp: [
    ["NUMBER",           "$$ = new yy.Number(@1.first_line, @1.first_column, yytext);"],
    ["VARIABLE",         "$$ = new yy.Variable(@1.first_line, @1.first_column, yytext);"],
    ["exp exp operator", "$$ = new yy.Expression(@3.first_line, @3.first_column, $1, $2, $3);"]
  ],
  operator: [
    ["PRINT", "$$ = new yy.Operator(@1.first_line, @1.first_column, yytext);"],
    ["=",     "$$ = new yy.Operator(@1.first_line, @1.first_column, yytext);"],
    ["+",     "$$ = new yy.Operator(@1.first_line, @1.first_column, yytext);"],
    ["-",     "$$ = new yy.Operator(@1.first_line, @1.first_column, yytext);"],
    ["*",     "$$ = new yy.Operator(@1.first_line, @1.first_column, yytext);"],
    ["/",     "$$ = new yy.Operator(@1.first_line, @1.first_column, yytext);"]
  ]
};

Implementing the Abstract Syntax Tree

Create the definitions for the abstract syntax tree nodes in lib/rpn/ast.js.

Since we will be maintaining line and column information in all of the AST nodes, we can reuse some code by making a base prototype:

var AstNode = function (line, column) {
  this._line = line;
  this._column = column;
};

The definitions for the rest of the AST nodes are pretty straight forward. Link up the prototype chain, assign relevant attributes, and don’t forget to call AstNode‘s constructor:

exports.Number = function (line, column, numberText) {
  AstNode.call(this, line, column);
  this._value = Number(numberText);
};
exports.Number.prototype = Object.create(AstNode.prototype);
 
exports.Variable = function (line, column, variableText) {
  AstNode.call(this, line, column);
  this._name = variableText;
};
exports.Variable.prototype = Object.create(AstNode.prototype);
 
exports.Expression = function (line, column, operand1, operand2, operator) {
  AstNode.call(this, line, column);
  this._left = operand1;
  this._right = operand2;
  this._operator = operator;
};
exports.Expression.prototype = Object.create(AstNode.prototype);
 
exports.Operator = function (line, column, operatorText) {
  AstNode.call(this, line, column);
  this.symbol = operatorText;
};
exports.Operator.prototype = Object.create(AstNode.prototype);

Compilation

Generated JavaScript

Before we generate JavaScript, we need a plan. There are a couple ways we can structure the outputted JavaScript.

One strategy is to translate the RPN expressions to the equivalent human readable JavaScript expression we would create if we had been writing JavaScript all along. For example, if we were to port this RPN example:

a 8 =;
b 2 =;
c a b 1 - / =;

We might write the following JavaScript:

var a = 8;
var b = 3;
var c = a / (b - 1);

However, this means that we are completely adopting the nuances of JavaScript’s arithmetic. In an earlier example, we saw that a helpful runtime error was thrown when any number was divided by zero. Most languages throw an error when this occurs, however JavaScript does not; instead, the result is Infinity. Therefore, we can’t completely embrace JavaScript’s arithmetic system, and we must generate some code to check for divide-by-zero errors ourselves. Adding this code gets a little tricky if we want to maintain the strategy of generating human readable code.

Another option is treating the JavaScript interpreter as a stack machine of sorts and generating code that pushes and pops values to and from a stack. Furthermore, stack machines are a natural fit for evaluating RPN. In fact, it is such a good fit that RPN “was independently reinvented by F. L. Bauer and E. W. Dijkstra in the early 1960s to reduce computer memory access and utilize the stack to evaluate expressions.”

Generating JavaScript code for the same example above, but utilizing the JavaScript interpreter as a stack machine, might look something like this:

push(8);
push('a');
env[pop()] = pop();
push(2);
push('b');
env[pop()] = pop();
push('a');
push('b');
push(1);
temp = pop();
push(pop() - temp);
temp = pop();
if (temp === 0) throw new Error("Divide by zero");
push(pop() / temp);
push('c');
env[pop()] = pop();

This is the strategy we will follow. The generated code is a bit larger, and we will require a preamble to define push, pop, etc, but compilation becomes much easier. Furthermore, the fact that the generated code isn’t as human readable only highlights the benefits of using source maps!

Creating Source Maps

If we weren’t generating source maps along with our generated JavaScript, we could build the generated code via concatenating strings of code:

code += "push(" + operand1.compile() + " "
  + operator.compile() + " "
  + operand2.compile() + ");\n";

However, this doesn’t work when we are creating source maps because we need to maintain line and column information. When we concatenate strings of code, we lose that information.

The source-map library contains SourceNode for exactly this reason. If we add a new method on our base AstNode prototype, we can rewrite our example like this:

var SourceNode = require("source-map").SourceNode;
AstNode.prototype._sn = function (originalFilename, chunk) {
  return new SourceNode(this._line, this._column, originalFilename, chunk);
};
 
...
 
code = this._sn("foo.rpn", [code,
                            "push(",
                            operand1.compile(), " ",
                            operator.compile(), " ",
                            operand2.compile(), ");\n"]);

Once we have completed building the SourceNode structure for the whole input program, we can generate the compiled source and the source map by calling the SourceNode.prototype.toStringWithSourceMap method. This method returns an object with two properties: code, which is a string containing the generated JavaScript source code; and map, which is the source map.

Implementing Compilation

Now that we have a strategy for generating code, and understand how to maintain line and column information so that we can generate source maps easily, we can add the methods to compile our AST nodes to lib/rpn/ast.js.

To play nice with the global JavaScript environment, we will namespace push, pop, etc, under __rpn.

function push(val) {
  return ["__rpn.push(", val, ");\n"];
}
 
AstNode.prototype.compile = function (data) {
  throw new Error("Not Yet Implemented");
};
AstNode.prototype.compileReference = function (data) {
  return this.compile(data);
};
AstNode.prototype._sn = function (originalFilename, chunk) {
  return new SourceNode(this._line, this._column, originalFilename, chunk);
};
 
exports.Number.prototype.compile = function (data) {
  return this._sn(data.originalFilename,
                  push(this._value.toString()));
};
 
exports.Variable.prototype.compileReference = function (data) {
  return this._sn(data.originalFilename,
                  push(["'", this._name, "'"]));
};
exports.Variable.prototype.compile = function (data) {
  return this._sn(data.originalFilename,
                  push(["window.", this._name]));
};
 
exports.Expression.prototype.compile = function (data) {
  var temp = "__rpn.temp";
  var output = this._sn(data.originalFilename, "");
 
  switch (this._operator.symbol) {
  case 'print':
    return output
      .add(this._left.compile(data))
      .add(this._right.compile(data))
      .add([temp, " = __rpn.pop();\n"])
      .add(["if (", temp, " <= 0) throw new Error('argument must be greater than 0');\n"])
      .add(["if (Math.floor(", temp, ") != ", temp,
            ") throw new Error('argument must be an integer');\n"])
      .add([this._operator.compile(data), "(__rpn.pop(), ", temp, ");\n"]);
  case '=':
    return output
      .add(this._right.compile(data))
      .add(this._left.compileReference(data))
      .add(["window[__rpn.pop()] ", this._operator.compile(data), " __rpn.pop();\n"]);
  case '/':
    return output
      .add(this._left.compile(data))
      .add(this._right.compile(data))
      .add([temp, " = __rpn.pop();\n"])
      .add(["if (", temp, " === 0) throw new Error('divide by zero error');\n"])
      .add(push(["__rpn.pop() ", this._operator.compile(data), " ", temp]));
  default:
    return output
      .add(this._left.compile(data))
      .add(this._right.compile(data))
      .add([temp, " = __rpn.pop();\n"])
      .add(push(["__rpn.pop() ", this._operator.compile(data), " ", temp]));
  }
};
 
exports.Operator.prototype.compile = function (data) {
  if (this.symbol === "print") {
    return this._sn(data.originalFilename,
                    "__rpn.print");
  }
  else {
    return this._sn(data.originalFilename,
                    this.symbol);
  }
};

Gluing it Together

From here we have done all the difficult work, and we can run a victory lap by connecting the modules together with a public API, and by creating a command line script to call the compiler.

The public API resides in lib/rpn.js. It also contains the preamble, to initialize __rpn:

var jison = require("jison");
var sourceMap = require("source-map");
var lex = require("./rpn/lex").lex;
var bnf = require("./rpn/bnf").bnf;
 
var parser = new jison.Parser({
  lex: lex,
  bnf: bnf
});
 
parser.yy = require("./rpn/ast");
 
function getPreamble () {
  return new sourceMap.SourceNode(null, null, null, "")
    .add("var __rpn = {};\n")
    .add("__rpn._stack = [];\n")
    .add("__rpn.temp = 0;\n")
 
    .add("__rpn.push = function (val) {\n")
    .add("  __rpn._stack.push(val);\n")
    .add("};\n")
 
    .add("__rpn.pop = function () {\n")
    .add("  if (__rpn._stack.length > 0) {\n")
    .add("    return __rpn._stack.pop();\n")
    .add("  }\n")
    .add("  else {\n")
    .add("    throw new Error('can\\\'t pop from empty stack');\n")
    .add("  }\n")
    .add("};\n")
 
    .add("__rpn.print = function (val, repeat) {\n")
    .add("  while (repeat-- > 0) {\n")
    .add("    var el = document.createElement('div');\n")
    .add("    var txt = document.createTextNode(val);\n")
    .add("    el.appendChild(txt);\n")
    .add("    document.body.appendChild(el);\n")
    .add("  }\n")
    .add("};\n");
}
 
exports.compile = function (input, data) {
  var expressions = parser.parse(input.toString());
  var preamble = getPreamble();
 
  var result = new sourceMap.SourceNode(null, null, null, preamble);
  result.add(expressions.map(function (exp) {
    return exp.compile(data);
  }));
 
  return result;
};

Create the command line script in bin/rpn.js:

#!/usr/bin/env node
var fs = require("fs");
var rpn = require("rpn");
 
process.argv.slice(2).forEach(function (file) {
  var input = fs.readFileSync(file);
  var output = rpn.compile(input, {
    originalFilename: file
  }).toStringWithSourceMap({
    file: file.replace(/\.[\w]+$/, ".js.map")
  });
  var sourceMapFile = file.replace(/\.[\w]+$/, ".js.map");
  fs.writeFileSync(file.replace(/\.[\w]+$/, ".js"),
                   output.code + "\n//@ sourceMappingURL=" + sourceMapFile);
  fs.writeFileSync(sourceMapFile, output.map);
});

Note that our script will automatically add the //@ sourceMappingURL comment directive so that the browser’s debugger knows where to find the source map.

After you create the script, update your package.json:

...
"bin": {
  "rpn.js": "./bin/rpn.js"
},
...

And link the package again so that the script is installed on your system:

$ npm link .

Seeing Results

Here is an RPN program that we can use to test our compiler. I have saved it in examples/simple-example.rpn:

a 8 =;
b 3 =;
c a b 1 - / =;
c 1 print;

Next, compile the script:

$ cd examples/
$ rpn.js simple-example.rpn

This generates simple-example.js and simple-example.js.map. When we include the JavaScript file in a web page we should see the result of the computation printed on the page:

Screenshot of simple-example.rpn's result

Great success!

However, we aren’t always so lucky, and our arithmetic might have some errors. Consider the following example, examples/with-error.rpn:

a 9 =;
b 3 =;
c a b / =;
c a b c - / =;
c 1 print;

We can compile this script and include the resulting JavaScript in a web page, but this time we won’t see any output on the page.

By opening the debugger, setting the pause on exceptions option, and reloading, we can see how daunting debugging without source maps can be:

Screenshot of enabling pause on exceptions.

Screenshot of debugging with-error.rpn without source maps.

The generated JavaScript is difficult to read, and unfamiliar to anyone who authored the original RPN script. By enabling source maps in the debugger, we can refresh and the exact line where the error ocurred in our original source will be highlighted:

Screenshot of enabling source maps.


Screenshot of debugging with-error.rpn with source maps.

The debugging experience with source maps is orders of magnitude improved, and makes compiling languages to JavaScript a serious possibility.

At the end of the day though, the debugging experience is only as good as the information encoded in the source maps by your compiler. It can be hard to judge the quality of your source maps simply by looking at the set of source location coordinates that they are mapping between, so Tobias Koppers created a tool to let you easily visualize your source maps.

Here is the visualization of one of our source maps:


Screenshot of the source map visualization tool.

Good luck writing your own compiler that targets JavaScript!

References

View full post on Mozilla Hacks – the Web developer blog

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Remote Debugging Firefox OS with Weinre

If you’ve wanted to contribute to Gaia, or have been writing a webapp for Firefox OS, one of the pain points you probably ran into, when using either B2G desktop or testing on a device, is the lack of developer tools to inspect and debug your HTML, CSS and JavaScript.

Currently we have two tracking bugs, go ahead an vote on them to bump their priority, to track the work going into developing a native remote inspector and style editor for Firefox OS but, I have some pretty exciting news. You can have access to a remote debugger today.

And how is this possible I hear you ask? Well, one word: Weinre. Weinre is a project of the Apache Foundation and stands for WEb INspector REmote and is exactly what it’s name suggests, a tool in the same vein as Firebug or Webinspector but, able to run and debug web pages remotely. So, if you have used tools such as the Firefox Developer Tools or Chrome Dev Tools, using Weinre will be second nature. But enough talk, let’s get this up and running.

Setting Up Weinre

As Weinre runs on top of Node.js your first port of call would be to install Node.js. Node.js comes with NPM (Node Package Manager) bundled nowadays and this is then then what we are going to use to install Weinre. From a terminal run the following:

npm -g install weinre

NOTE: The -g flag is used to install Weinre as a global Node.js module for command line goodness but, on Linux and Mac, this means you most likely are going to need to run the above by prepending sudo to the above command.

Once the installation process is complete, we are ready to use Weinre to debug. But first, let’s make absolutely sure that Weinre was indeed installed successfully. In your terminal, run the following:

$ weinre --boundHost 127.0.0.1 --httpPort 9090
2013-01-28T10:42:40.498Z weinre: starting server at http://127.0.0.1:9090

If you see a line similar to the last line above, your installation was a success and the Weinre server us up and running. With that, fire up a browser (NOTE: The UI for Weinre is built specifically for Webkit based browsers so, while it might work to some degree in other browsers, I would suggest you use Chrome) and point it to http://127.0.0.1:9090

Above then is the landing page for the Weinre server giving you access to the documentation, some other trinkets, as well as the Weinre client, the page we really want to head to so, go ahead and click on the debug client link.

From the above you can see that we have one connected client, this is the current instance of the web inspector, some general properties of our server but, no targets. Let’s get our target set-up.

NOTE: If the UI of winery looks very familiar that’s because Winery uses the same UI code as the web inspector in Chrome and Safari.

Setting Up A Weinre Target

In Weinre targets are the web pages or apps that you want to debug, and in order for the target to be able to connect, we need to add a one liner to the relevant file of our app. For this post, let’s inspect the Calendar app. Go ahead and open up gaia -> apps -> calendar -> index.html and scroll right to the bottom. Just before the closing body tag, insert the following line:

<script src="http://127.0.0.1:9090/target/target-script-min.js#anonymous"></script>

Before we can launch B2G Desktop and try this out however, there is one more step. Gaia uses a Content Security Policy and as part of that scripts are said to only be allowed to load, if from the same origin as the application. So, if we were to try and load the Calendar now, the script from above would be blocked as it is not being loaded from the specified origin.

To overcome this, we need to temporarily disable CSP. To do this, open up gaia -> build -> preferences.js and add the following line, around line 24:

prefs.push(["security.csp.enable", false]);

Debugging Using Weinre and B2G Desktop

Now we are ready to test Weinre. If you are not already inside the Gaia root directory, change into it now and execute:

DEBUG=1 make

Once the profile is built, launch B2G desktop:

/Applications/B2G.app/Contents/MacOS/b2g-bin -profile /Users/username/mozilla/projects/gaia/profile

Once B2G launches, unlock the screen, swipe two screens to the right and click on the Calendar icon to launch the Calendar app. Once the app launches, you will see a new item appear on Weinre’s client dashboard:

As you can see, the Calendar has successfully connected and is now listed as one of our targets. Go ahead and click on the ‘Elements’ tab.

Listed here is the HTML of our app on the left and our CSS on the right! You can go right ahead and edit either the HTML or the CSS as you normally would and see the changes reflect live. Note that even though the CSS looks grayed out and disabled, it if fully editable. You can also add completely new styles to the current element using the empty element.style block or amending existing rules. You will also notice you have access to the computed styles as well as metrics of the current element.

Working With The Console

The next tab of interest to us is the Console tab. Here you can code away and run any JavaScript you want directly against the current app or execute code exposed by the app. To see how this works, let’s interact with the call log portion of the Dialer.

First step then is to move our script import from Calendar to Dialer. Grab the code from Calendar and then open up gaia -> apps – > communication -> dialer -> index.html and paste the code. Next rebuild your profile using ‘make’ and finally relaunch B2G desktop.

Once it is launched again, click on the Dialer icon at the bottom left of the home screen. Once loaded, confirm that the communication channels are open to Weinre by opening http://127.0.0.1:9090/client/#anonymous and confirming that the target now looks as follows:

127.0.0.1 [channel: t-7 id: anonymous] - app://communications.gaiamobile.org/dialer/index.html#keyboard-view

With the dialer open, click on the call log icon, bottom left. Currently the call log is already populated with some dummy data but, let’s create our own. Click over to the Console tab in Weinre, type the following and press enter.

RecentsDBManager.deleteAll();

If you look at the view on desktop now, it would seem that nothing has happened but wait, there is more. Type in the following and press enter:

Recents.refresh();

Aha! As you can see, our call log is empty. Next step then, is to add an entry back. To do this, we will create a dummy call entry Object and then pass this to the add function of the RecentsDBManager to store it:

// Dummy entry
var recentCall = {
    type: 'incoming-refused',
    number: '555-6677',
    date: new Date()
};
RecentsDBManager.add(recentCall);
Recents.refresh();

And as you can see now, the entry we just created has been added to storage, IndexedDB to be exact, and is visible in the call log view. As you might very well have noticed, another of the great features that comes with the console is auto-complete which will further speed up development.

The combination of features that this exposes already opens new doors and will make working on Firefox OS, or writing apps for the OS, much easier with less time spent between building profiles, tearing down and relaunching B2G. Which all makes for happier developers.

But hey wait a second, what about debugging on the device? This will work exactly the same as the above with one small difference, the IP. When you want to debug on the device you first need to know the IP address of your host computer. Then you need to start up Weinre using this IP as the buondHost and also as the IP when including the script into you target documents.

On Mac and Linux you can get this address using ifconfig and on Windows it is ipconfig. Once you have the new IP, just stop the current instance of Weinre and then do the following:

weinre --boundHost 192.168.1.1 --httpPort 9090

Then inside you target document add:

<script src="http://192.168.1.1:9090/target/target-script-min.js#anonymous"></script>

Make and push your Gaia profile to the device using:

make install-gaia

Launch your target app and you are in business!

Conclusion

While this solution is not perfect, you need to remember to undo your changes before committing anything to source control, having to manually add the script every time is not ideal and then there are also some things that do not work 100%, such as, highlighting DOM elements as you hover over the HTML source and debugging JavaScript with breakpoints and such, this does go a long way towards improving the lives of developers both working directly on Gaia as well as those writing exciting new apps for Firefox OS.

But there is already some light at the end of the tunnel with regards to managing the injection of the script manually, disabling CSP and ensuring things are cleaned up before pushing to source control. Jan Jongboom has opened a pull request against the Gaia repo that looks extremely promising and will alleviate a lot of this so, go on and give him a hand and let’s get this merged into Gaia. Happy Hacking!

An important note: None of the above would have happened if it was not for Kevin Grandon who remembered using Weinre and sent out the email that set the ball rolling. Thanks Kevin!

View full post on Mozilla Hacks – the Web developer blog

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Responsive Patterns, automatic jQuery app tester, RESTClient for debugging – Hacks Weekly

Responsive Patterns, automatic jQuery app tester, RESTCLient for debugging RESTful web services are just some of the things Mozilla’s Developer Engagement team want to recommend to you this week!

Weekly links

If there is anything you think we should read or know about, don’t hesitate to post a comment, contact us on Twitter or through any other means.
The picks this week are:

The Developer Engagement team

Mozilla’s Developer Engagement team work with writing articles, documentation – such as MDN (Mozilla Developer Network) – public speaking and generally helping and informing about open technologies and Mozilla products. If you are interested in following our work, here are the team members:

View full post on Mozilla Hacks – the Web developer blog

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Remote Debugging on Firefox for Android

A month ago we introduced the Debugger and other new developer tools introduced in Firefox 15. Our new Debugger lets you connect Firefox Desktop to Firefox on Android so that you can use your nice, roomy desktop monitor to troubleshoot JavaScript that is running on your mobile device. 95% of Android devices are running a Firefox-compatible version of the operating system (Android 2.2 and higher). Firefox won’t run on all of these devices, but there are certainly many millions of devices that you can pick up and begin remote debugging on.

We will now explain how to remote debug Firefox on your Android device. The following screencast also shows you how to setup and run the Remote Debugger:

Set up your desktop

First of all, make sure you’re running Firefox 15 at least. At the time of this article, Firefox 15 is in the beta channel.

Next, type about:config into the Firefox location bar so that you can change a setting in the browser. The browser will display a warning, but don’t worry… we’ll be careful.

In the search box at the top of about:config, type “remote-en“. The only setting that should match is “devtools.debugger.remote-enabled“. Set that to true by double clicking it.

Finally, restart your browser. After you do, you’ll spot “Remote Debugger” in the Web Developer menu (either in the main Firefox menu or in the Tools menu on Macs).

Set up your Firefox for Android

For this to work, you need to use Firefox 15 on your Android device as well, and you can get that by downloading Firefox Beta from the Google Play store.

Now, fire up Firefox Beta on your phone. As on Desktop, you’ll need to go to about:config to change settings. Search for “debugger” and

  1. toggle devtools.debugger.force-local to false
  2. toggle devtools.debugger.remote-enabled to true

To get the remote debugging server started, you’ll need to restart Firefox. Tap the home button to exit Firefox.

On a Galaxy Nexus running Jelly Bean you can force Firefox to restart by opening the task switcher and sliding the Firefox Beta task off to the right.

Extra note: if you’re uncomfortable with the idea of having your Firefox listening for connections on the network, you can leave force-local set to true and follow Mark Finkle’s instructions for getting remote debugging running over USB.

Find your IP address

Before we’ll be able to connect to the device, we need to know its IP address. To find the address of your phone:

  1. open the Settings app
  2. tap Wifi
  3. tap the network that you are currently connected to

The display you’re presented with will include your IP address.

Your IP address on an Android phone

Make the connection

You’re now all set for remote debugging!

On your mobile device, browse to a page that you want to debug. Fire up the Remote Debugger on your desktop Firefox. The Remote Debugger appears as a new window and it will prompt you for the address to connect to. Replace “localhost” with the IP address of your phone. By default, the remote debugging server on your mobile device will be running on port 6000, so leave the “:6000″ there in the connection address.

You should see a dialog on your phone warning you about a new incoming connection. You should only accept connections that you start as otherwise someone might be trying to hijack your browser. In this case, we’ve started the connection, so go ahead and allow it.

Once you’ve connected successfully, you’ll be able to inspect the scripts loaded in the page, set breakpoints and so forth.

Let us know what you think!

Give it a try and let us know what you think of this feature and what more you’d like to see. You can respond in the comments below or email us on the dev-apps-firefox mailing list.

View full post on Mozilla Hacks – the Web developer blog

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)