Sunday, December 6, 2009

mjst - Micro JavaScript Template Engine

This is the content I've written so far in the official mjst repository. I'd love to know your thoughts about this little experiment, cheers!

What

mjst is a lightweight, about 1Kb minified and gzipped, and performances focused template engine with standards support. There are dozens JavaScript? Template Engines solutions out there but as far as I know this is the only one based on validation for both client and server side programming languages.

How

There is one single function to call:

// via text (e.g. Ajax or innerHTML from a template node)
var transformed = mjst('<?js var hello = "mjst!"; ?><js-hello/>');

// classic template node example
<script id="mytpl" type="text/html">
 
<?js
   
var hello = "mjst!";
 
?>
 
<js-hello/>
</script>
myEl.innerHTML = mjst(document.getElementById("mytpl"));

/
/ via XML (e.g. Ajax or a created document)
// this is a configuration object as second argument example as well
myEl
.innerHTML = mjst(
 
new DOMParser().parseFromString(
   
'<root><js-hello/><br class="${myBrClass}" /></root>',
   
"text/xml"
 
), {
    hello
:"Hi there!",
    myBrClass
:"clear-both"
 
}
);

// Ajax example with an XML template as response
/*
<root>
  <ul>
    <?js
      for(var i = 0; i < collection.length; ++i) {
    ?>
      <li class="li-${i}">Item No: <js-i/> Name: <?js print(collection[i]); ?></li>
    <?js
      }
    ?>
  </ul>
</root>
*/


// transformation ...
myEl
.innerHTML = mjst(xhr.responseXML, {collection:["a", "b", "c"]});

It is possible to write whatever we want inside an mjst template block via print(1, 2, 3, N, func(), whatever) function, the only one injected in the execution scope.

JavaScript? block VS Attributes

To make mjst templates XML compatible I decided to remove JavaScript? block inside attributes. It is still possible to access JavaScript? variables via ${myVar.propName} as showed, as example, in the br node class.

PHP And JavaScript? Template Example

<?php // mjst example
function mjst($name, $value){
    echo
'<?js var ', $name, '=', json_encode($value), ' ?>';
};
header('Content-Type: text/xml');
$a = array(
    'some database result' => array(1, 2, 3),
    'some other value' => 'Hi There'
);
?>
<?xml version="1.0"?>
<root>
   
<?php mjst('result', $a); ?>
   
<?js
   
for(var k in result)
       
print(result[k], "<br />")
   
;
   
?>
</root>

<script type="text/javascript" src="mjst.js"></script>
<script type="text/
javascript">
onload
= function(){
   
with(this.XMLHttpRequest ?
     
new XMLHttpRequest :
     
new ActiveXObject("Microsoft.MSXML")
   
){
      open
("get", "mjst.php", true);
      onreadystatechange
= function(){
       
if(readyState == 4)
          document
.body.innerHTML = mjst(responseXML);
       
;
     
};
      send
(null);
   
};
};
</script>

Why Standards

Apparently every other JavaScript? template engine is implementing a manual, char by char, or RegExp? based, parser, over a syntax that in PHP world has basically defined deprecated ages ago: the classic <%=stuff%> (<?=stuff?> in PHP). The first reason to avoid this kind of syntax is ambiguity. Which program language should consider that block of code? PHP, ASP, JavaScript? ... who else? Moreover, to make things as clear and simple as possible, avoiding ambiguity, PHP community decided to solve the classic <?xml?> problem, caused by short code block, via an explicit <?php ?> which is the suggested, default one. This is also standards safe, in the meaning that a nodeType 7 is defined as a PROCESSING_INSTRUCTION_NODE In few words mjst is compatible with both valid XML and XHTML, which means that its templates can be created via standard DOM API being sure the template will be valid as well. At the same time, using official engines to validate and parse templates blocks mjst could be considered more reliable than other solutions, more robust, and at the same time fast, thanks to core functionality. The good part, at least for me, is that delegating templates parsing operations to the browser core, is more probable that the error is inside a non valid template block, rather than inside this tiny library source code: a welcome side-effect about standards and core validation.

Why Fast

Every transformation could create one or two pre-compiled functions reused every time we would like to transform an already parsed template block. The reason mjst creates up to two functions is that we can arbitrary send a second argument as configuration object. Since this procedure requires the usage of the with statement, avoiding outer scope name conflicts during the execution, and since we do not necessary need to send this second configuration object, the runtime compiled function will consider both cases, only when encountered, making the with free execution 3 to 10 times faster. In other words, if we transform the same template block always without the second argument, the function that will contain the with statement will never be created and vice versa.

Why Robust

The XML core parser and transformer is also another reason to consider mjst both fast and reliable. As example, probably the most famous JavaScript micro templating, from John Resig, is surely lightweight, but it could suffer some problem if the template block is not perfect. For instance, try to use a string like s = "<%" for whatever reason, and you'll be able to break the John micro suggestion, but this is not only about John code. Another well known product is the EJS library, adopted in many different environments and definitively a good piece of code. EJS offers somethng more than mjst but it is still based on manual parsing over a syntax that XML does not like that much. This means that we could have hard life to generate EJS templates via server or directly in our page while mjst delegates template validity to the markup itself.

<?xml version="1.0" ?> 
<script id="mytpl" type="text/html">
 
<!-- msjt accepts valid XML (as XHTML as well) or HTML5 -->
 
<?js
   
var hello = "mjst is here!";
 
?>
 
<js-hello />
</script>

Why ... Generally Speaking

I am a PHP developer, before being a JavaScript? one, and I have always complained about the open close everythere PHP style. This technique does not split presentation and business logic layers and it is generally "slower than a single echo". I have started to appreciate the couple XML and XSL-T years ago but XSL-T sometimes is just a pain in the ass and via server is difficult to cache XSL-T instances. As example, the New York Time had to create their own core library to speed up transformations over common XSL files allowing fast serialization. Brilliant, but still this ad hoc library does not scale for the whole World Wide Web scenario. Different hosts, different PHP versions, but at the end of the day, what could happen if we delegate the transformation to the client side via core JavaScript? features? Nothing bad, actually it's almost a year I am transforming complex layout into html in the fastest way ever even in IE6 and only thanks to XSL-T but, as I have said, the XSL syntax could not sound familiar at all, specially for programmers that would like to have much more power while they generate a layout. This is basically the reason PHP had such great success over these years, so why don't try to emulate XSL bringing there directly JavaScript? rather than XSL syntax, and in a PHP familiar style? This is all about this experiment. It delegates layout render into the client side, the best place imho to do this. It avoids useless server side stress, it could be embed in the page thanks to the fake script with an unknown type, and as side effect, it could speed up client server interactions re-using templates blocks whenever we need and requiring only data, which nowadays, it is just all we need to make Web users happy. So, as pure final evil plan, we could create a view able to produce mjst templates directly via CouchDB resolving some greedy map reduce operation in the db moving partial logic in the client via simple row data sets or generating clear layout directly via Server Side JavaScript? so that all we need is a language, rather than 3 different technologies ... does it sound insane? ;-)

Thursday, December 3, 2009

with: Some Good Example

OK, I have said my last post would have been the last one about the with statement ... well, before I close the argument, I would like to show a couple of common good examples.

Solve References Problem

This is what somebody defined
a tiny masterpiece


with({o:myreference}){
o.doStuff();
o.var1 = "whatever";
// etc etc ...
};

Above example annihilate every blame about the not sure if that is the var I meant to use since there is an explicit reference as is, as example, in Python programming language (until version 3.11)

// Python
with open("x.txt") as f:
data = f.read()

// JavaScript equivalent
with({f:/* as */open("x.txt")})
data = f.read()


Memory Safe Operations

Another recycled example before I show more interesting stuff ... apparently the whole problem is about write silly code inside with, as if everybody has to define variables or assign stuff inside this statement. Hilariously, I have basically never assigned anything inside a with in 8 years of ECMAScript programming ... does it tell you anything?
A clever usage could be the one to beat whatever compressor and make code less redundant:

with(document)
with(documentElement)
insertBefore(
createElement("script"),
firstChild
)
.text = "alert(1)"
;

Try to do more in less characters considering I am not creating a single reference at all, as I am not creating a useless closure just to justify references creation ... done? Now use whatever compiler/compresor/minifier you want and try to obtain less than 99 bytes, respecting the same cross browser, clean, memory safe and leaks free nature ...

Discover Named Arguments

A too much common approach in JavaScript is the one to create an inline executed closure to perform some task being sure local variables won't disturb outer or global scope.

(function(obj, collection, callback){
// a common argument normalizer
if(!collection)
collection = []
;
// a loop, 'cause we don't want to perform it
// in the global scope
for(var i = 0, length = collection.length, tmp; i < length; ++i){
// callback returns an object only under certain conditions
tmp = callback.call(obj, collection[i]);
if(tmp){
tmp.doStuff();
collection[i] = tmp;
};
};
})();

Guys, above example is daily basis JavaScript programming
  1. create a closure to avoid outer scope properties declaration

  2. normalize, if necessary, unexpected/undefined/empty arguments

  3. define internal closure variables with or without a value

  4. perform some operation

  5. exit from the executed closure (implicit)
Now, consider we all love to call methods or functions via configuration objects

ajax({
url:somestring,
params:someObject,
success:callback,
failure:shenanigans
});

A configuration object is semantic, it allows us to perform some task inline, like ternary operator assignment (and I bet since somebody does not know ternary operator ES5 team will decide it's evil ...) and make callback calls more friendly ... but what we have always been envious about Python is the ability to use named arguments defining defaults, if necessary, for each value.
In JavaScript we can send named arguments, via a configuration object, but we cannot define arguments defaults ... but in one shot, we could emulate everything defining both defaults and, since we need to define them in any case, local variables ... don't you follow me?

// zero closure named arguments example
with({
obj:myObject,
// inline defaults, if necessary
collection:collection || [],
callback:myCaseAnalyzer,
// local variables
i:0,
length:collection ? collection.length : 0,
// local undefined variable
tmp:null
}){
for(;i < length; ++i){
tmp = callback.call(obj, collection[i]);
if(tmp){
tmp.doStuff();
collection[i] = tmp;
};
};
};

Same behavior, zero ambiguity, if we call another variable name is because we need that variable from the outer scope. If we create a variable we are doing wrong because inside a closure we would have used var, while with this approach we need simply to define, rather than var, a property with null value.
Moreover, if tomorrow we need an extra argument via with statement all we need to do is to define it in whatever position we like, without being worried about arguments order (let's say bad design) since there is no order, except the one we prefer, in object properties.

Global Clear Ajax Call

This is the last example for this post, just ajax when we need it, whenever we are, without the classic reference and the missed "this" feature in the onreadystatechange function:

with(this.XMLHttpRequest ?
new XMLHttpRequest :
new ActiveXObject("Microsoft.XMLHTTP")
){
open("get", "?ajax=true", true);
onreadystatechange = function(){
if(readyState === 4)
// do stuff with the responseText/XML
alert(responseText)
;
};
send(null);
};


Global Paradox

The funniest part ever about this with statement is that basically every JavaScript has an implicit global object scope.
In few words every time a script is executed is basically the same of a massive with

with(window){
// same as window.alert(123);
alert(123);

// same as window.onload = function(){};
onload = function(){
};

(function(){
// same as new window.XMLHttpRequest
var xhr = new XMLHttpRequest;
})();

// same as window.String.fromCharCode
String.fromCharCode(1);
}

Of course being the global object the last possible outerscope, if we define a variable directly or via var in that scope we will attach this property to the window one.
In few words the with is the ABC of JavaScript but again, some clever guy decided that since somebody cannot understand closures, then extended closures, the basic principle of the programming language itself has to be removed ... right?

Think about it, if ES5 will remove even the with statement making this unavailable in the "use strict" future proof declaration, JavaScript won't be the one that created all its success around its closure and prototype based nature.

Think!

Wednesday, December 2, 2009

with: The World's Most Misunderstood Statement

Update If after this post and its comments you are not convinced yet, have a look into with: Some Good Example one.


Every reference to this post is purely casual ... no it's NOT!
It's time to talk about the fuss around the innocent with statement, and why somebody decided it's bad or it should not be part of next JavaScript.

Extends the scope chain for a statement

Extended chains are probably what we love more about JavaScript libraries ...

$("stuff")
.more()
.again()
.somethingElse()
;

but for some sadistic reason we decided that we don't like native extended chains in our code ...

with(stuff){
more()
again()
somethingElse()
};


Bad Examples And Bad Usages

Every time I say that with statement has nothing bad, somebody perpetually points out this bloody post ... OK, from Yahoo! ... so what?
Honestly, I could post thousands of bad examples and point my finger into this or that library, function, method, evaluation, etc etc ... it's like the myth eval is evil ... so how come one of the first one to talk about evil is using eval to speed up code interpretation?
Easy: there are cases and cases!

The Power Of with(){}

I give you the most basic example ever, something happened in twitter few minutes ago, OK?
I wanna use the smallest amount of characters to obtain a script injection ... right?

with(document.documentElement)
insertBefore(
document.createElement("script"),
firstChild
)
.text = "alert(1)"
;

Elegant, memory aware (no variables declarations, no need to create a closure), compact, efficient, cross-browser ... where are "JavaScripters" that used with to assign variables here?

with has been used since first ECMAScript 3rd Edition implementation and nobody has never had a single problem. Moreover, if we contest that we could cause a wrong variable assignment, what about a required closure plus a required var assigned to a possible node causing possible leaks plus name conflicts?

We remove a statement but we don't remove those developers that constantly forget to define local variables so they can destroy the global namespace?
for(i = 0; ... ) // FIRED FFS!!!

I would go for the second specie before the first one!

In any case, I'd love to see who is able to produce a code smaller than the precedent example taking care about removing any trace of that operation in the current scope.

Pointless Points

Somebody could say: "doode, you are not sure that insertBefore is the document.documentElement " ... The classic piece of antagonist code would be 99.9% of cases this one:

// avoid global scope pollution
(function(e){
e.insertBefore(
document.createElement("script"),
e.firstChild
).text = "alert(1)"
;
// feeling cool and clever
// just to avoid "var"
})(document.documentElement);

Wait a second ... in the compress everything to improve performances Web era we need to create a closure and a scoped variable over greater number of characters? ... REALLY? And what about the "non being sure point", does anybody check if document.documentElement, always present, has an insertBefore method? NO because sometimes we simply don't have to!

Assignments

If we willing to remove a historical valid statement as with is because people could assign stuff there and they don't know where the stuff is assigned, we should remove every function able to evaluate code from a scripting language as JavaScript is.
It's like blaming features because some silly junior developer could cause disasters:

<textarea
onchange='eval(this.value)'
></textarea>

... we cannot be serious here, can we? So, remove textarea HTML element because it could be used from somebody to write malicious code directly evaluated when the content changes ... is this the ES5 philosophy? I do hope no, but apparently, and at least for the with statement, it is.
Maybe we should consider the fact that if we have not control over our objects and scopes we can remove every evil thingy but the bad code will be still there ... isn't it? ... but it's a language fault, right?

Real Reasons To Remove with()

Compilers, munger, minifier, however we call them, these analyzers have hard life because of with statement.
The reason is that these parsers do not consider a with statement as an in-line scope and cannot predict in that moment the accessible outer scope, unless these parsers won't be able to virtually replicate every possible scenario in that scope.
Furthermore, since some compressor could optimize properties access, bytewise speaking, a with statement will make that optimization redundant:

myO.doStuff(1);
myO.doStuff(2);
myO.doStuff(3);
// sometimes enough to decide
// doStuff could be optimized via
myO[s](4);
// and here it cannot
// or it requires extra effort ...
with(myO)
doStuff(5)
;

If we use the string shortcut in the statement we'll assign the variable or raise an error (if the string is for some reason hard coded).
Another example is property access optimizations typical via ordered namespaces as YUI! dojo Closure or other libraries have.

MyLib.namespace.stuff.doSomething(1);
MyLib.namespace.stuff.doSomething(2);
MyLib.namespace.stuff.doSomething(3);
// enough in this scope to optimize access via
var s = MyLib.namespace.stuff,
d = "doSomething"
;
s[d](4);
s[d](5);
s[d](6);

Can you spot the difference? And what if in the middle of the closure there is a with?
These are just few examples where a statement like with could become uncomfortable ... but why nobody else has ever had this problem?

Implicit this

If we would like to compare a closure with a classic Object Oriented Programming world, we could consider that most of the languages uses implicit reference inside methods.

function plusPlus(void):MyClass {
++i;
return this;
}

That "i" is referencing an instance variable and it does not requires the "this" prefix. So how come implicit scope is permitted in many other languages, Java and C# included, but we don't want an in-line implicit scope in JavaScript, as with statement basically is?

Python way

There is a basic rule in the Python local/global variable choice: explicit is better than implicit ... and I cannot agree more:

class MyOne():
value = 0
def add(self, i):
# requires to be explicit
self.value = self.value + i
# otherwise it will generates an error
# since value, as local, has not been defined
# value = value + i

value = 5

o = MyOne()
o.add(1)
o.value

To avoid the error we need to define the value as global one:

global value
value = value + 1

In JavaScript we have the opposite scenario, what has been explicitly declared as local, arguments included, is local, everything else is considered from outer scopes or, if nothing there, global. It could sounds messy but this is all about JavaScript and closures so if we don't like it, why are we using JavaScript? Just use GWT or whatever hybrid way to end up with translated closures ... no? I see ... we think and we feel we are truly programming in Java, as example, rather than JavaScript, don't we? ... so, how come this unicorn became the most used programing language?

Anyway, we are not here to decide who made the best choice, we are here to discuss why with statement is still present in other programing languages, right?
The answer is simple: to make things simpler!
This Fredrik Lundh post is the first one I found searching an example ... so, does with make sense?

If We Need It, Create A Reference

Imagine we have a global open function able to return a File instance ... now imagine we justify python because there is an implicit "as" to reference that pointer ...

// Python with style simulation
with({f:open("x.txt")}){
var data = f.read(1);
}

// if we need a temporary variable
// but we would like to be sure about
// variables in the scope
with({f:open("x.txt"), data:null}){
data = 123;
// do other stuff
}

But what is wrong if we are sure that we are dealing with a variable and its method?

// I don't want to care
// about variables declaration
// neither about leaks
// scope, names, etc ...
with(open("x.txt")){
if(read(1) === "+")
write("-")
;
close();
}

In few words, there are really thousands of bad examples online and rarely good one but the blamed one is always the with statement rather than people unable to understand it.
Another example, still twitter, few minutes ago ...

var obj = {};
obj.var1 = "objvar1";
var var1 = "globalvar1";
var var2 = "globalvar2";

// so we chose the object to use ...
with(obj) {
// .. and we have no idea what kind of object is it?
print(var1); // the object one, EASY!

// if we think obj should have a var2
// property but it does not
// we are doing wrong in any case
// and we don't know our instances
print(var2); // scoped var, EASY

// if we know what we are dealing with
// we have already resolved this problem
// knowing var1 and var2 values
var1 = "bar1";
var2 = "bar2";

// this is quite obvious ... isn't it?
// we have assigned these references one line ago
print(var1); //'bar1'
print(var2); //'bar2'
};

// is there anything unexpected here?
print(var1); // 'globalvar1'
print(var2); //'bar2'
print(obj.var1);//'bar1'
print(obj.var2);//undefined


As Summary


At the end of the day these developers scared by with do not simply use it, while developers able to use it in a proper way should say good-bye because somebody else decided that with statement should not be part of ES5: WTF ... I mean, am I the only one able to spot the beauty of the with statement here?
We could find solutions for arguments.callee but with statement ... guys, please be serious about that, and ask every other programing language to remove it, while somebody could find even the good old goto useful.
Remove features, for what? Performances? Who cares if it is rarely used, I do care if it is not possible to use anymore.

Even More Evil

with(document)
with(documentElement)
insertBefore(
createElement("script"),
firstChild
)
.text = "alert(1)"
;