Saturday 30 January 2010

jQuery, Google Closure and packing or minifying for performance?

One thing I love about writing jQuery plug-ins is that every line of code counts towards performance. Its like real programming again. There are lots of difference ways in which you can improve performance but one must have is reducing the size of the JavaScript that comes down the wire from the server.

There are two options to use when reducing JavaScript file size – packing and minifying. Packing uses a compression algorithm such as gzip to physical compress the file to it’s smallest possible size, whilst a decent minifyer removes whitespace, renames variables and internal functions and removes unreachable code.

A while ago I came across this post by John Resig pointing out that although packing reduces the file size, it takes longer to ultimately execute, due to the repeated unpacking process every time the script is used.

After reading this I started using the Yahoo YUI Compressor to minify all of my plug-ins, and it worked well, typically reducing file sizes by 70% or more.

Looking through the jQuery 1.4 release notes this week I noticed that the jQuery team have switched to Google Closure Compiler. This really is a great tool, for example it reduced my Exclusive Checkbox plug-in to only 191 211 bytes and the MapReduce plug-in to under 1k in size! You need to be careful when using the compiler with the advanced option (due to the possible renaming of public methods and variables), but this doesn't apply to plug-ins because of the use of a closure – perhaps this is the reason for the name of the compiler.

 Update   I had to abandon the use of the Advanced option for the Exclusive Checkbox plug-in, increasing the size from 191 to 211 bytes ...

Links:

Online YUI compressor
Online Google Closure compiler service

Thursday 28 January 2010

Simple OOXML Updated

Simple OOXML is a helper library that I wrote for the Open XML Format SDK 2.0 to make the creation of Open Office XML documents easier. You can modify or create any .docx or .xlsx document without Microsoft Word or Microsoft Excel or any other software (ideal for server environments).

Version 2.1.3678 has been released over at Codeplex which works with the December CTP of the SDK.

Batch conversion of docx files to pdf

I recently worked on a project where we had over 50 .docx files to convert to .pdf. Now Office 2007 can do this for you, but only one file at a time. Luckily I was able to piece together some code (.net 3.5 csharp) which automated this process. Select a folder and all .doc and .docx files in the folder and any subfolder will be converted to pdf and saved with a .pdf extension.

pdfbatch

You still need Office 2007 or later to do this – download the code here:

http://www.opencomponents.com/downloads/PDF Batch Converter.zip

Disclaimer: Use at own risk. No kittens were harmed during the making of this software.

Monday 25 January 2010

MapReduce plug-in Release Candidate now available

The MapReduce plug-in for jQuery has been uploaded to the following url:

http://plugins.jquery.com/project/MapReduce

The plug-in implements a a type of map/reduce algorithm that executes synchronously or asynchronously in the browse whilst minimising loss in performance. Supports long running operations without script timeout messages.

Add map and reduce functions using the $.map and $.reduce methods and execute the functions using the $.execute method. Nested map and reduce functions are supported.

The download contains two samples of finding the prime numbers in a sequence of numbers, one using a plain javascript function and one using map reduce to calculate this asynchronously without causing script timeout errors in the browser.

Options

The following options can be used when calling the $.execute method:

async: true or false - determines whether the $.execute method runs asynchronously (default false)
race: no of functions to call per iteration when async is true (default 100)
interval: interval in ms between iterations when async is true (default 1ms)

Saturday 23 January 2010

Introduction to Map Reduce using jQuery

(Solved: How to avoid the "A script on this page is causing Internet Explorer to run slowly." message)

When writing the spellayt spell checking plug-in, I came across an interesting problem with long running JavaScript code – the browser would timeout and display a message box to the user with the text "A script on this page is causing Internet Explorer to run slowly." Even worse, the dialog gives the user the option to cancel the script, potentially breaking your page.

The solution was to use a timer to simulate a background thread, but the solution wasn’t generic and felt far from elegant. This week, with the approval of Google’s Distributed MapReduce patent, I realised that a simpler non-distributed map reduce algorithm would enable the running of code both asynchronously and over long periods of time in JavaScript without complex state management and stack issues.

Map reduce works by taking key/value pairs and running them through a function to create intermediate key/value pairs until all the data has been processed (map). The data is then filtered (reduced) to obtain a result. Joel Spolsky (Joel on Software) has a nice article that explains the fundamentals behind map reduce.

Sieve of Eratosthenes

The Sieve of Eratosthenes is an algorithm for finding prime numbers, this Wikipedia article has a  visual explanation of how it works. Its a great example of something that is processor intensive but can be broken up into smaller steps.

A standard JavaScript implementation (using Euler’s optimisation) would look something like this:

function eratosthenesSieve(upperBound) {

   var upperBoundSquareRoot = Math.floor(Math.sqrt(upperBound), 0);
   var isComposite = [];

   for (var m = 2; m <= upperBoundSquareRoot; m++) {

      if (!isComposite[m]) {

         _primes.push(m);

         for(var k = m * m; k <= upperBound; k += m) {
            isComposite[k] = true;
         }
      }
   }

   for(m = upperBoundSquareRoot; m <= upperBound; m++) {
      if (!isComposite[m]) _primes.push(m);
   }
}

The isComposite variable contains an array of boolean indicating whether is element is prime (false) or not (true). The _primes array also contains the prime numbers.

Redefining the problem as a set of Map / Reduce functions

The great thing about map/reduce is that once the problem is defined as a set of functions, it doesn’t matter how the map/reduce is implemented, you should always get the same results. Usually, each for loop will result in a matching $.map or $.reduce function. Reorganising the eratosthenesSieve function to use map reduce requires the following steps:

  • Map an array containing the initial values – in this case the index of the array is the number we are testing, and a boolean will indicate whether it is a prime or composite number.
  • Map each prime up to the square root of the upper bound. For each Map that is a prime, strike out it’s square multiplied by the remaining numbers until the upper bound is reached.
  • Reduce the array by totalling the number of primes (false values) left.
  • Execute the functions to return the result.

The code now looks like this:

function eratosthenesSieve2(upperBound) {

   //Create an array of booleans representing prime = false
   var composites = new Array(upperBound);

   //Set all to prime (false) to start
   $.map(composites, function(index, value) {
      return false;
   });

   //Calculate the sqr root of the upper bound
   var upperBoundSquareRoot = Math.floor(Math.sqrt(upperBound), 0);

   //For each item, calculate the non primes
   $.map(composites, function(index, value) {

      if (index < 2 || index > upperBoundSquareRoot) return value;

      //If the array item is a prime then map all non primes
      if (value == false) {

         var k = index * index;

         $.map(composites, function(index2, value2) {

            if (index2 == k) {

               k += index;
               return true;
            }
            return value2;
         });
      }
      return value;
   });
                
   //Count only the primes
   $.reduce(composites, 0, function(index, value, result) {

      if (index >= 2 && value == false) result++;
      return result;
   });

   return $.execute();
}

The first $.map function initialises the composites array. The second and third map $.map strikes out multiples of primes. The $.reduce function counts the remaining primes by accumulating the result variable. Note the extra work we have to do to simulate the functionality of the for loop and when we haven’t performed any processing we return the same value that was passed into the function.

Finally, we $.execute the functions. If you attach a debugger to the script, you’ll notice the functions only start executing once this method is called.

Download the plug-in code from the Map Reduce plug-in page.

Next time, I’ll dive into the details of implementing the Map Reduce plug-in and the pitfalls encountered along the way.

Object Data Blocks 2.0 Release Candidate

 

Object Data Blocks is object persistence style ORM for new database implementations. Define you data layer using classes to define tables and fields and decorate them with attributes to define keys, indexes, constraints etc. A provider model for a specific relational database (currently sql server 2005 or later) takes care of the rest. Any changes to the assembly can be deployed at runtime seamlessly to the database without any loss of data.

Read more information and view code samples here.

Object Data Blocks is released under the GPL 2.0 licence and is hosted at http://objectdatablocks.codeplex.com