Ajax Patterns And Best Practices (2006)

The typical network of the late eighties had unique addresses for all devices, so that all devices were uniquely identifiable and information could be routed to the device without any resolution problems. But then something bad happened, and that was the Web. The Internet existed before the Web and was used by a few people,1 and those who used it generally respected the unwritten rules. I am not trying to knock the Web—the Web created the Internet economy, which is no small feat. Along the way of transitioning from the traditional Internet to the Internet economy, the structure of the Internet changed radically. The transition has made writing applications that use the Internet more complicated. In the transition, the server side of the Internet has remained the same, but everything with respect to the client has changed. The changed structure of the Internet is illustrated in Figure 8-4.

Figure 8-4. Structure of the Internet after the Web

Looking at Figures 8-3 and 8-4, your first reaction might be, “So, yeah, some radical change—two new unique addresses.” The changed unique addresses make an entire world of difference with respect to Internet structure. The IP addresses 192.168.1.10 and 192.168.1.11 are so-called reserved addresses that cannot be used to uniquely identify computers on the Internet because multiple local area networks will use the same IP addresses. This means that the server 66.35.250.151 does not see the computers 192.168.1.10 or 192.168.1.11, and sees only the router 212.254.35.68. Adding on to the complexity, the router address 212.254.35.68 changes often and cannot be used to uniquely identify a calling network of computers.

1.If you know (without searching Google) about Archie, Whois, WAIS, Veronica, Jughead, or Gopher+, then you are showing your age with respect to the Internet!

The result is that the server 66.35.250.151 cannot send information updates to the clients (192.168.1.10 or 192.168.1.11) because the server has no idea how to address the client. The transition did not cause a collapse of the Internet because the router has become more intelligent and created something called Network Address Translation (NAT). NAT is like an early 1900s telephone operator who received a call and then transferred the call to another operator or to the destination. The client making the call and server receiving the call do not contact each other. The router is in contact with both. When the client makes a call to the server, NAT works extremely well and transparently. However, in the other direction, the router cannot decide which client to contact. The solution has been to assign ports on the router that redirect the call from the router to the internal computer. Even with that solution, the router can dedicate only a single computer to a specific port. The NAT solution is not a general networking solution as in the 1980s.

This change in the Internet architecture occurred for multiple reasons: money, IPv4 addressing problems, security, and maintenance. Why these are the reasons does not matter and changes nothing. Focusing on how to deal with the change is more important. From the perspective of an Ajax developer, it is not possible for the server to arbitrarily send messages to a particular client. However, looking at peer-to-peer solutions such as BitTorrent, it would seem the problem is solved. No, the problem is not solved,2 but delegated as an implementation issue that the user needs to address before running BitTorrent. With respect to the Ajax developer, that is not a solution. The only solution that is both robust and viable is to poll for data.3

Implementing a Polling Solution

Having identified that a poll is required for making a reliable and robust server-to-client communication, the challenge is implementing a poll that is effective and that wastes as little resources as possible. For the example that spans the scope of this chapter, the client is a web browser, and the server is the HTTP server. As per the HTTP protocol, the client initiates the request, and the server responds to the request.

Polling involves the querying of a server at specific intervals, like the polling of a Post Office Protocol (POP3) e-mail server. Typically, e-mail is retrieved by polling a POP server for available messages every x minutes. The problem with polling is that it can be inefficient and can miss important events. Imagine polling a server for available messages. The server could say, “No messages,” and the client would then wait x minutes. If a message arrives right after a poll is made, the client will know about the message only after waiting x minutes. If the polling frequency period were two hours, a message that arrived after a poll and that was valid for only one hour would be stale by the next poll. If the polling frequency period were 10 seconds, the message would be retrieved fairly quickly and would not be stale.

The downside to a higher polling frequency is that the network and server suffer. The network suffers from excessive network bandwidth usage, and the server suffers from having to constantly respond to a request and respond with a “No message available” answer. In a nutshell, polling when done too slowly will cause messages to not arrive in time, and polling when done

2.http://www.bittorrent.com/FAQ.html#firewall. The solution involves opening the router or firewall and redirecting the Internet traffic to the appropriate computer.

3.Some companies will sell components promising to send data asynchronously so that you don’t have to poll. The fact is that the poll is hidden within the code of the component. There is no easy technical solution around the NAT addressing problem when writing server-to-client communications.

too quickly will waste network and server resources.4 This is a sort of damned-if-you-do and damned-if-you-don’t situation.

The solution is to create two streams used by the client and server to communicate with one another. The first stream is used to receive messages, and the second stream is used to send messages. To receive messages, the client polls the server with a request for messages. If there are no messages, the server does not respond with an answer immediately. The server puts the poll request on hold for a specific amount of time or until a message is generated by

the server. Putting the poll on hold puts the client on hold while waiting for a message. A traditional poll will query the server, get an answer, and then wait until the next poll. The waiting period until the next poll is dead time during which neither the client nor the server can communicate with each other. By converting the dead time into a wait created by the server, the client is waiting for the potential of a message being generated.

Two streams are necessary because while the client is waiting for a response from the server, the client might want to send a message to the server. If one stream is waiting for a response, it is not possible to send content by using the waiting stream. The solution is to create another stream for sending content to the server or for writing purposes. From the perspective of the server, the other stream is a request that sends data.

In terms of HTTP, the reading stream that is put on hold waiting for content is an HTTP GET, and the writing stream that sends content is an HTTP POST or PUT. In technical implementation terms, for the reading stream the HTTP server puts the socket and thread that are processing the request on hold. Putting the thread on hold is not a problem for HTTP servers. What is a problem is that thousands of threads could be waiting for messages that may or may not be generated. One solution is to get a big enough computer with enough RAM. Another solution is to specifically find an HTTP server that can deal with this problem elegantly.5

Another potential problem on the HTTP server is that the two streams might conflict. The reading stream executes on one thread, and the writing stream executes on another thread. Both threads might be accessing the same piece of data, and hence synchronization is required.

From an architectural perspective, the two-stream communication mechanism appears similar to Figure 8-5.

In Figure 8-5, the browser interacts with a type called ClientCommunicator. The purpose of ClientCommunicator is to create the two-stream communication mechanism and process messages that are sent and received from the server. In the implementation of ClientCommunicator, two separate instances of XMLHttpRequest are used. One instance of XMLHttpRequest represents the reading stream and calls the resource /resource/receive. The second instance of XMLHttpRequest represents the writing stream and calls the resource /resource/send. On the server side is something called ServerCommunicator, which is responsible for combining the two streams.

The resources /resource/receive and /resource/send were used to illustrate the nature of the data direction; they do not refer to actual URLs. As mentioned earlier, writing involves using HTTP POST or PUT, and reading involves using HTTP GET. Because these two HTTP verbs are distinct from each other, the same URLs can be used for both streams.

4.It is possible to use HTTP persistent connections, and doing so reduces network bandwidth and is a good thing in general. However, HTTP persistent connections do not solve the server-to-client communication problem.

5.http://jetty.mortbay.org/jetty/ is a URL to the Jetty HTTP server. For version 6.0, Jetty has solved the waiting thread and resource problem and is a recommended solution for Java programmers implementing the Persistent Communications pattern.

Figure 8-5. Architecture of Persistent Communications pattern

Implementation

For the rest of this chapter, the two-stream communication mechanism is going to be implemented in the context of the three scenarios: status updates, presence detection, and server push. The explanations start with the simplest scenario and finish with the most complex. The client code will be developed first because it is identical for all scenarios. The server code is what changes from simple to complex.

Example: A Global Status Resource

Status updates are by their nature global. Global status updates does not mean global data. What it means is that the data referenced by the status update is accessible by all, and in terms of the Permutations pattern is a single resource that can have multiple representations. In a nutshell, what you have is shared data that has a single state and does not care about the client. The client considers the data as read-mostly. Read-mostly data is updated very little by the client, but may be updated constantly by some external influence. From the perspective of the two-stream communication mechanism, the reading stream will be used most of the time to retrieve the latest status updates. The writing stream (which would be used to update the state) is not used, or at least is used very rarely. However, this is not to say that the writing stream is never used; whether it is used depends entirely on the context.

Implementing the HTML Page

An HTML page that contains the ClientCommunicator and implements the reading and writing streams is presented to the user. The responsibilities of the HTML page are to define a URL, provide a callback function, and begin querying the server using the reading stream. For the example, the HTML page that realizes the responsibilities will have buttons to start and stop the reading stream. In your production application, the triggers to start and stop the reading stream might be an event or a default action started by a script. Figure 8-6 shows the HTML page.

Figure 8-6. HTML page that interacts with a global status

The Start Communications button is used to start receiving updates of the global status from the reading stream. The End Communications button is used to stop receiving updates. And the Send Data is used to send text by using the writing stream and to update the global status. The received updates of the global status are stored in the table below the row of buttons. As the HTML page illustrates, any client that accesses the global status resource will receive similar representations of the resource.

The HTML page’s code is as follows:

<html>

<head>

<title>Global Status Page</title> </head>

<script language="JavaScript" src="../lib/factory.js"></script> <script language="JavaScript" src="../lib/asynchronous.js"></script>

<script language="JavaScript" src="../lib/clientcommunicator.js"></script> <script language="JavaScript" type="text/javascript">

var client = new ClientCommunicator(); client.baseURL = "/ajax/chap06/status";

client.listen = function(status, statusText, responseText, responseXML) { document.getElementById('httpcode').innerHTML = status; document.getElementById('httpstatus').innerHTML = statusText; document.getElementById('result').innerHTML = responseText; document.getElementById('xmlresult').innerHTML = responseXML;

}

function StartCommunications() { client.start();

}

function EndCommunications() { client.end();

}

function SendData() {

var buffer = "hello world"; client.send("application/text", buffer.length, buffer);

}

</script>

</head>

<body>

<button onclick="StartCommunications()">Start Communications</button> <button onclick="EndCommunications()">End Communications</button> <button onclick="SendData()">Send Data</button>

<p><table border="1"> <tr><td>Document</td>

<td><span id="httpcode">No Http Code</span></td> <td><span id="httpstatus">No Http Status</span></td> <td><span id="result">No Result</span></td>

<td><span id="xmlresult">No XML Result</span></td></tr> </table></p>

</body>

</html>

The HTML page has multiple script tags where the src attribute is defined. Each of the src attribute values represents a JavaScript file that contains reusable generic Ajax code used by the client. The reusable code is from other patterns presented in this book. The code related to the client side of the Persistent Communications pattern is located in the file clientcommunicator.js and will be explained shortly.

At the point in the HTML where the script tag does not have a src attribute, some JavaScript code is defined. The JavaScript code is the code used by the HTML page to make the Persistent Communications pattern code do something useful. After the closing script tag, the remaining HTML code is responsible for creating the HTML page illustrated in Figure 8-6.

Getting back to the JavaScript code that makes the Persistent Communications pattern code do something, the first line of code instantiates the type ClientCommunicator and assigns the instance to the variable client. The type ClientCommunicator is an implementation of the ClientCommunicator, as defined in Figure 8-5. The common URL used by the reading and writing streams is represented by the property client.baseURL. The property client.listen is assigned a function and is called by ClientCommunicator whenever the server sends an update on the reading stream.

Quickly going through the rest of the JavaScript code, you can see that the method client.start() starts the update process of reading content from the reading stream. The method client.end() ends the updates. The method client.send(...) sends updates to the server by using the writing stream and has three parameters. The first parameter ("application/ text") represents the MIME type of the sent content, the second parameter (buffer.length) is the content length, and the last parameter (buffer) is the data that is sent to the server.

Without explaining the details of the ClientCommunicator and ServerCommunicator, Figure 8-7 illustrates running the HTML code and how the Persistent Communications pattern functions so that you get an idea of what plumbing code needs to be implemented.

Figure 8-7. Status updates illustrated in the HTML page

In Figure 8-7, the table contents have been replaced with some information sent by the server. The data replaced in the table is the result of a round-trip that starts with clicking the Send Data button. Clicking this button generates some content that is written to the writing stream. The server receiving the content on the writing stream stores the data as a global status resource. Then, when the Start Communications button is clicked, the reading stream queries the global status resource. The server will respond by sending the updated global status resource. The client receives the updated information on the reading stream and updates the table. What has occurred in this example is an illustration of how a global status is written and read.

Implementing the ClientCommunicator

With the HTML page illustrated and the behavior of the HTML page explained, it is necessary to begin the implementation of the pattern’s plumbing. In the HTML code example, the property client.baseURL defines the base URL, and the example base URL is /ajax/chap06/status. The URL definition is provided by some human and applied in the JavaScript.

The ClientCommunicator code is a larger piece of code; therefore, instead of presenting all of the code at once, I will present and explain smaller pieces. The following code starts the explanation by showing the constructor-related code of ClientCommunicator:

function CounterHack() { this.counter = 0;

}

function ClientCommunicator() { this.server2Client = new Asynchronous(); this.baseURL = null;

this.username = null; this.password = null;

this.listen = null; this.doLoop = false; this.callDelay = 500;

this.preferredTypes = "text/xml"; this.index = this.instanceCount.counter; this.instances[ this.index] = this; this.instanceCount.counter ++;

}

ClientCommunicator.prototype.start = ClientCommunicator_start; ClientCommunicator.prototype.end = ClientCommunicator_end; ClientCommunicator.prototype.send = ClientCommunicator_send; ClientCommunicator.prototype.instances = new Array(); ClientCommunicator.prototype.instanceCount = new CounterHack();

The function ClientCommunicator is defined to be a constructor because the script of the HTML page in Figure 8-6 instantiates the type ClientCommunicator. When ClientCommunicator is instantiated, many properties are defined and explained as follows. The property server2client is an instance of Asynchronous that encapsulates XMLHttpRequest. The property server2client is responsible for implementing the reading stream. The property baseURL has already been discussed. The properties username and password are used to identify the user, which in the case of the global status resource is not important. Of course, just because the authorization information is not relevant for this specific example, it does not mean that authorization to the global status resource is not needed. The property listen has already been discussed. The property doLoop is a flag that indicates whether the periodic reading stream checks should be continued. The property preferredTypes indicates the preferred MIME types that result in the preferred representation to receive as per the Permutations pattern.

The properties callDelay, index, instances, and instanceCount are all related and warrant a more detailed explanation to the problem they solve. The problem relates to how a repeating loop is implemented in JavaScript. In the “Architecture” section, you saw that to implement the reading stream, the client polls the data—meaning a repeating loop is created. Using a JavaScript-defined loop cannot solve the repetitious nature of a poll. A JavaScript loop takes control of the user interface and would lock the client from accepting further input or processing data. Additionally, because asynchronous requests are made, the loop would result in an infinite number of requests to be issued. The real problem here is that JavaScript does not implement true multithreading. Had JavaScript implemented true multithreading, creating a never-ending JavaScript loop would not be a problem.

Even though JavaScript does not have true multithreading capabilities, it is possible to make it appear that it does. An example of pseudo-multithreading was illustrated in Chapter 2 and required the use of a timer. The solution to the problem of a locked browser when using loops is to create the impression of a loop by using a timer that goes off right away and calls a function. The sequence of events that makes a timer look like a loop is defined as follows:

1.When the function is executed, an asynchronous XMLHttpRequest request is made, resulting in a calling of the reading stream.

2.The function exits immediately as an asynchronous request is made, and no timer is called. The web browser can continue accepting input and processing data.

3.In the background, the HTTP server has put the asynchronous request that is waiting for a message on the reading stream on hold.

4.When the asynchronous request returns with a response, the response is processed and then a timer is started to call the function that executes another asynchronous request.

What is important about the sequence of events is to call the timer only when the asynchronous request has returned. Calling the timer earlier would result in an infinite number of requests being made immediately, causing the web browser to lock and the server to suffer a denial of service attack.

Using the timer, we are confronted with another problem: the window.setTimeout method requires a reference to a text-based script. The reference cannot be an object reference, because JavaScript when converting an object reference will reference either an undefined reference or a value that does not exist. The problem is clearly illustrated in the following source code:

function runIt(value) { window.setTimeout("Loop(value)", 1000);

}

The function runIt has a parameter, value, which is an object reference and is used in the script expression of the method setTimeout. The problem is that the script expression is a piece of text, and the timer can execute only a piece of text, not a reference to the variable value.

A solution is not to reference the variable, but to copy the value of the variable to the text script, as the following source code illustrates:

function runIt(value) {

window.setTimeout("Loop(" + value + ")", 1000);

}

In the modified source code, the function Loop will be called properly with the value of the variable value.

Knowing that a variable has to be converted to a value is a step closer to the solution, but is not the entire solution. The main problem is that the variable value will reference an object instance, and serializing an object instance is a bad practice because it results in multiple object instances with similar states. Having multiple versions of the same object results in object state consistency problems. The solution is to pass a value that represents an index of an array. The resulting implementation uses the array property this.instances and associated properties (callDelay, index, instances, and instanceCount) to store and manage references of the reading streams shown in abbreviated detail as follows:

this.index = this.instanceCount.counter; this.instances[ this.index] = this; this.instanceCount.counter ++;

ClientCommunicator.prototype.instances = new Array();

ClientCommunicator.prototype.instanceCount = new CounterHack();