
Dr.Dobb's journal.2006.02
.pdf
that instrumentation can cause serious performance problems. But if you use a package or class-based filter to limit the instrumentation, you might end up not profiling an important part of the application. Further, if changing the instrumentation requires restarting the application, the profiling process slows down considerably.
An alternative to bytecode instrumentation is to monitor events generated by the JVMPI or JVMTI interface. Both APIs generate events for method entry/exit, object allocation, monitor entry, and so on. But there are some drawbacks: Using these features disables VM optimizations and not much flexibility is provided. As a result, most profilers avoid using these features, although there are some VM events that must be tracked this way (garbage collection, for instance).A third approach is to use sampling instead of instrumentation or
Figure 1: CPU performance profiling options.
event monitoring. The profiler periodically examines the stack of each thread in the VM. The interval is typically between 10 and 100 milliseconds (ms). The stack traces are recorded and the method at the top of the stack is assumed to have been running since the last examination. So if the interval is 10 ms and method foo( ) is at the top of the stack four times, then the profiler will report that method foo( ) ran for 40 ms. This technique imposes little overhead, but suffers from some drawbacks because the results are generally less accurate. Most importantly, there is no information about the number of times foo( ) was called. Another drawback is that the profiling overhead grows as the number of threads grows. And finally, sampling provides no solution for memory profiling, so if your application has memory-al- location problems, a profiler that does only sampling will be of no use.
Dynamic Bytecode Instrumentation
A new approach, profiling with dynamic bytecode instrumentation (DBI), offers some important benefits. With DBI, the insertion of instrumentation bytecodes is done at runtime. After you have selected a class or method for profiling, the next time it is invoked by the application, the profiler will insert the necessary instrumentation. This is done using
Figure 2: Information overload. Every application thread is listed.
Figure 3: Incomplete performance results for the run() method.
Figure 4: Complete performance results for the run() method.
the JVMTI’s redefineClass(Class [ ] classes, byte[ ][ ] newBC) method.
This means you can change the profiling overhead as the application is running. You can even remove all instrumentation so that your application is no longer being profiled. These changes can be made adaptively as you learn more about the behavior of the application. Perhaps best of all, the changes can be made without restarting your application.
CPU Performance Profiling
To get the maximum benefit from DBI, the profiler has to provide additional functionality. For CPU performance profiling, this means intelligent usage of DBI so that only the methods that need profiling have bytecodes added for instrumentation. Performance problems are typically associated with certain operations or functionality in an application. As an example, in a reporting system, a specific report might take longer than others to display. Once you identify the specific method that starts the execution of the feature with the problem, the profiler should be able to instrument just that method, allowing the rest of your application to run at full speed. Limiting the profiling also reduces the output from the profiler so that it only reports on the interesting parts of your application.
The top-level methods you select for profiling are root methods. To provide a complete picture of the performance problem, the profiler must instrument not only the selected root method(s), but also all methods that it invokes. This analysis must be done iteratively so that the profiler calculates the transitive closure of the call chain. When an instrumented method is invoked by the application, the instrumentation code must efficiently determine if a root method is in the call chain. If no root method is in the call chain, then the instrumentation code exits immediately to minimize overhead. If a root method is in the call chain, then profiling information is recorded.
A profiler that lets you select root methods and then applies DBI intelligently allows you to define the problem in your terms. Filters based on package and class names are still useful, but selecting root methods is a much more precise way of controlling profiling overhead while maintaining useful results in the profiler’s output.
Memory Profiling
As with CPU performance profiling, to get maximum benefit from DBI when doing memory profiling, a profiler must use additional logic. This means smarter techniques for keeping track of objects because that is the largest source of overhead when doing memory profiling. It turns out the
46 |
Dr. Dobb’s Journal, February 2006 |
http://www.ddj.com |

smartest technique is one of the simplest: Track only a subset of the allocated objects. Tracking as few as 10 percent of the allocated objects usually yields the same information, but with much less overhead. This is particularly true on larger applications.
A profiler should not only track object allocations, it should also help you determine if your application is leaking memory. A memory leak occurs when your application is no longer using an object but is inadvertently still holding a reference to that object, thereby preventing the VM from freeing up the memory used by that object. The sooner you can identify the objects that are potential leaks, the sooner you can reduce profiling overhead by having the profiler use DBI to remove instrumentation from all other classes.
Identifying potential leaks can be difficult, especially in large applications. The memory heap in a large application can be so big that attempting to compare heap snapshots frequently offers little help because the leaking objects are not obvious. This is especially true for leaks that start out small but continue to grow over time; sometimes weeks of application uptime are required before the problem is large enough to be noticeable.
A statistical approach for identifying memory leaks works well and imposes very little overhead. There are two key metrics: the age of each object and the generation count for each class. The age of an object is the number of garbage collections it has survived. The generation count for a class is the number of different ages of all objects of that class. A low generation count indicates that all the objects of a class have been in memory about the same amount of time. A high generation count (or a generation count that always increases) indicates that your application is continuing to allocate new objects of that class without letting go of references to older objects of that class. These are the best candidates for being memory leaks because applications do not typically intend to periodically allocate long-lived objects. Instead, they typically have longlived objects that are allocated at roughly the same time and short-lived objects that are periodically allocated and then removed by the VM once they are no longer needed.
CPU Performance Profiling Example
We will use the NetBeans Profiler (http:// profiler.netbeans.org/) to demonstrate DBI. For CPU performance profiling, we chose the Java2D program, which is one of the demos included with Sun’s Java Development Kit (JDK). When we started the profiling session, we configured the NetBeans Profiler to perform instrumentation of the entire application, as in Figure 1.
This caused a dramatic slowdown in application startup time: 10 seconds instead of 2 seconds.
Application performance was also sluggish. When run without profiling, the application was able to keep up with a rapid series of mouse clicks on its top-level display tabs. When run with full instrumentation, it was not able to keep up. An additional problem is that, with full instrumentation, it took longer to find the information we needed. We were interested in the performance of the java2d.AnimatingSurface.run( ) method. With full instrumentation, we had to wade through 13 threads to find the three that invoke the run( ) method; see Figure 2.
We then restarted the profiling session and applied a filter that removed the instrumentation from all java.* classes. Application performance improved dramatically, but the comprehensiveness of the reported results suffered quite a bit. Compare Figure 3 with Figure 4. In Figure 3, the java.* classes were excluded and as a result important details about how much time was spent executing the methods called by the run( ) method are not available. Figure 4 shows the output when all classes were profiled.
To get acceptable application performance and comprehensive results, we used a DBI feature of the NetBeans Profiler to change the instrumentation. We removed
Figure 5: Only the interesting threads are listed.
Figure 6: Memory profiling options.
Figure 7: Default heap view sorted by size.
http://www.ddj.com |
Dr. Dobb’s Journal, February 2006 |
47 |

the filter that had eliminated the java.* classes and we chose a single root method: java2d.AnimatingSurface.run( ). As a result, application performance was very responsive and it was much easier to find the information we needed; only the three threads that had invoked the run( ) method were displayed, as in Figure 5. And because no java.* classes were filtered out, full details of the run( ) method were available; see Figure 4.
This application is relatively small, but the lessons learned apply to larger applications as well. When full profiling was done, 19,579 methods were instrumented. Adding a filter to remove the java.* classes dropped the number of methods with instrumentation to 712, but also reduced the amount of information available. With a single root method selected and the java.* classes no longer filtered out, the
number of methods with instrumentation dropped to 344. Additionally, we still had comprehensive profiling results for the method that was of interest.
Memory Profiling Example
The sample application we created to demonstrate memory profiling contains a common Java antipattern: Objects are periodically placed into a Map and then forgotten. This prevents the VM from removing those objects from memory. Given enough time, the application eventually runs out of memory and crashes. The objects are small, however, so the amount of time before a crash occurs could be days or weeks. Instead of waiting for it to crash, we used the NetBeans Profiler to watch the sample application’s memory usage. The NetBeans Profiler can track just allocations, or allocations and garbage
Figure 8: Heap view sorted by generation count.
collection. We chose the second option and also to track 10 percent of allocations, as in Figure 6.
The NetBeans Profiler has a dynamic display of the heap contents, as in Figure 7. By default, it lists classes by the amount of space that the objects of that class are using. When we switched the display order to be by generation count and then let the application run for a while, the memory leak was easy to spot. Three classes had generation count values that were noticeably larger and were continuing to increase as the application ran (Figure 8). The three classes are java.util.HashMap$Entry, double[ ], and float[ ]. After those were identified, we used a DBI feature within the NetBeans Profiler to turn off the instrumentation of all other classes. This reduced profiling overhead dramatically.
Knowing which classes are leaking is usually only part of the challenge because frequently there are leaking and nonleaking instances of a class. In other words, some of the allocations are okay, some are not. The NetBeans Profiler helped us find the allocations that are leaking by showing us stack traces for each allocation. The top-level list of stack traces in Figure 9 lists two methods that are allocating HashMap$Entry objects: addEntry( ) and createEntry( ). The generation count for addEntry( ) is dramatically larger than for createEntry( ), so it is the more likely source of the leak.
Expanding the stack trace of addEntry( ) shows that it is called by put( ), which in turn is called by several different methods. Of all the different execution paths that result in the creation of HashMap$Entry objects, one has a generation count that is dramatically larger than the rest. That code path begins with the run( ) method in the aptly named demo.memoryleak.LeakThread class, as in Figure 10. The NetBeans Java editor was used to display the source code, which allowed us to spot the memory leak bug:
map.put (new float[1], new double [1]);
Figure 9: The two top-level stack traces for HashMap$Entry allocations.
Figure 10: Expanded stack trace for HashMap$Entry allocations.
Conclusion
You do not have to endure untenable amounts of overhead and an overwhelming amount of output to profile your Java applications. Dynamic Bytecode Instrumentation, when properly applied, can make profiling fast and easy.
References
Dmitriev, Mikhail. “Design of JFluid: A Profiling Technology and Tool Based on Dynamic Bytecode Instrumentation,” http://research.sun.com/techrep/2003/ abstract-125.html.
DDJ
48 |
Dr. Dobb’s Journal, February 2006 |
http://www.ddj.com |

Range Tracking &
Comparison Algorithms
Managing/comparing ranges denoted by base/limit pairs
KIRK J. KRAUSS
Some information is best viewed as a list of ranges. Most software developers are familiar with the concept of virtual address ranges occupied by
memory heaps, stacks, executable code modules, and the like. Application programs that need awareness of such ranges might store a range list that gets updated over time; for example, when a heap shrinks or grows. Similar range lists could also be used in other applications to represent sections of an image in a motion picture, or age groups of participants in a long-term study, or layers in a collection of sediment samples, or candidate choices at exit polls across the country, and so on. Range-based data is commonly used by accounting, vision, storage, operatingsystem software, and elsewhere.
In any of these examples, a range might represent a contiguous set of items, such as the depth of certain strata in a sediment sample, or opinions of voters between 25 and 35 years old. Each range in the list might be represented as a pair of numbers — one number that represents the base of the range and another number that represents that range’s size or limit. To keep a list of ranges up to date and useful, you need algorithms for the creation, modification, comparison, and deletion of the numeral pairs that represent the ranges in the list.
Range-list comparison comes in handy when the ranges can change in size. For example, suppose a range list is used to
Kirk is a software developer working in the Rational Software division of IBM. He can be contacted at kjkrauss@us.ibm.com.
represent heap memory in a virtual address space, where ranges of heap memory are listed as a series of base/limit pairs. Each base represents the bottom address of a range of contiguous virtual address space that the operating system has committed for use by that heap. Each limit represents the top of such a range. As the heap grows and/or shrinks, a new list of base/limit pairs is generated for comparison with the existing list. A diff comparison of an existing list and a new, updated one then selects the areas of heap growth and shrinkage, possibly for use in memory-state tracking or similar purposes.
Building and Managing Lists
Range-list creation and destruction are implemented via a pair of algorithms (Listings One and Two) that assume that the ranges to be tracked reside in the virtual address space for a process. You might need to make some minor implementation changes to apply these algorithms to other specific types of range data. To use these algorithms, you should provide code to pinpoint the appropriate beginning and ending points of an item’s range, at lines in the pseudocode that are identified via relevant text. You should also provide code that allocates, initializes, and deallocates the range-tracking elements themselves, also at points shown as text in the listings.
A notable feature of the range-list creation algorithm is the coalescing of ranges for contiguous items. Multiple objects that occupy contiguous virtual address space are represented by a single range-tracking element. This is done to achieve optimal performance when these range lists are accessed in situations such as heap-range tracking where the boundaries between contiguous items aren’t important. If you need to keep track of boundaries between separate ranges even when they are contiguous, you can simplify the algorithm so that it doesn’t coalesce contiguous ranges.
A part of a range list can be replaced by splicing in a new set of ranges using the Splice Range List algorithm (Listing Three, available electronically; see “Re-
source Center, page 6). This saves you from having to replace an entire list when updates are needed for just part of the list. This pseudocode assumes that the ranges to be tracked reside in the virtual address space for a process. With some minor implementation changes, the algorithm could be applied to other types of range data. You should provide code that accesses the appropriate list elements wherever this pseudocode contains statements such as
“Range-based data is commonly used by accounting, vision, storage, and operating-system software”
“look up the next range on the new list.” You should also provide code that allocates, initializes, and deallocates the rangetracking elements themselves, at the element insertion and deletion points specified in this pseudocode. The startAddr and stopAddr input parameters denote virtual addresses in the context of this pseudocode. More generically, these values correspond to beginning and ending delimiters that apply to both the old and new range lists. Old list elements corresponding to ranges that do not reside between these delimiters are left in place. New list elements effectively replace the old ones between these delimiters. In some conditions, where an old range spans one of these delimiters, an old list element may be modified to reflect the changes represented via the new list. The pseudocode provides special case logic to address these conditions.
Comparing lists of base/limit pairs requires a more complex algorithm than one might first suspect. Before I developed
50 |
Dr. Dobb’s Journal, February 2006 |
http://www.ddj.com |

such an algorithm, a coworker and I first searched for any comparable base/limit, list-wise diff routine. We searched through textbooks and web pages showing algorithms used in computer vision and DNA sequencing. Because we could find no such algorithm, I wrote my own. This algorithm is suitable for any situation where lists of ranges represented by base/limit pairs need to be compared. In the pseudocode for the range-list comparison algorithm (Listing Four; available electronically), I’ve chosen to represent the two lists as “old” and “new” because diff algorithms are probably most often used to determine which elements of a dataset have been updated between two points in time.
Figure 1 shows the kind of information supplied by the range-list comparison algorithm. Given an old and new list, this algorithm indicates which subsets of the list have changed, including whether the changes represent data specific to the old list or the new list. A notable aspect of the algorithm is the number of Boolean variables needed to control the passage through the old and new lists. It is coded this way largely for “debugability” because the state information at any point in the algorithm is rather complex. There is also a performance consideration: This algorithm frequently needs to know whether there is a current and next element on each list, and to repeatedly dig into the
list elements, as that information could be costly. Another performance optimization is that the number of comparisons of list elements themselves has been minimized. For each pair of list elements, the algorithm checks for range overlap, for growth or shrinkage at the bottom of the range, and for growth or shrinkage at the top of the range — and only then does it bump forward to do the same thing with the next element(s) of one or both lists.
As with the splicing algorithm, to implement range-list comparison, you should provide code that accesses the appropriate list elements wherever this pseudocode contains statements such as “look up the next new range.” The diff results are collected at points designated “tell the outside world <whatever>.” You could add function calls at these points to update appropriate tracking elements in other lists, and so on. Alternatively, you could collect elements of a third list at these points and return a reference to those results when the diff is complete.
A potentially time-saving feature of the range-list comparison algorithm is that, like the splicing algorithm, it lets you specify the boundaries applied to the beginning and ending points of the list. Suppose you’re interested in comparing only the ranges of objects that reside within a portion of a heap occupying a subset of the virtual address space. You can speci-
|
|
New range list |
|
Old range list |
|
||
|
|
. . . |
. . . |
|
removed |
||
Lower end of |
the overall |
space |
|
|
|
|
|
|
|
|
|
||||
|
|
|
|
|
|
|
added |
|
|
|
|
|
|
|
|
Upper end of |
the overall |
|
|
|
|
added removed |
|
space |
|
|
|
|
|||
|
|
|
|
||||
|
|
|
|
|
|
|
|
|
|
. . . |
. . . |
|
|
||
|
|
|
|
|
|
|
|
Figure 1: Information supplied by the range-list comparison algorithm.
fy starting and stopping addresses to obtain a list applicable to the interesting portion of virtual memory. The algorithm ignores any objects that lie outside of these specified boundaries, which you specify as input parameters. No matter what kind of ranges you want to compare, if you want to limit the comparison to part of a larger list, you can apply these parameters to limit the set of ranges that will be compared, to save time.
DDJ
Listing One
/* The Create Range List algorithm |
*/ |
|
/* Tracks a current set of address ranges for the objects that meet |
*/ |
|
/* specific criteria. Coalesces the ranges of a contiguous set of |
*/ |
|
/* these objects into a single range list element, minimizing the |
*/ |
|
/* list's size and optimizing subsequent range list diff performance |
*/ |
|
/* Returns the list of ranges that are identified and tracked here. |
*/ |
|
/* Input parameters */ |
|
|
ADDRESS startAddr |
|
|
ADDRESS stopAddr |
|
|
/* Local variables */ |
|
|
LIST |
a new list of ranges |
|
BOOL |
bListRange = FALSE |
|
/* All other variables are references to address range tracking |
*/ |
|
/* list |
elements. At minimum, there will be a references to the |
*/ |
/* "current" element in the new list being created. |
*/ |
|
Allocate and initialize the header for the new list of ranges |
|
|
IF (startAddr) DO |
|
|
Determine the first object that resides at or above startAddr |
|
|
Make that object "current" |
|
|
END |
|
|
ELSE DO |
|
|
Determine the first object that can be found |
|
|
Make that object "current" |
|
|
END |
|
|
/* Walk through the address space, building the list of address |
*/ |
|
/* that meet a user-defined set of selection criteria |
*/ |
|
WHILE ((the address of the "current" object) < stopAddr) DO |
|
|
IF (the "current" object meets range listability selection criteria) |
||
|
bListRange = TRUE |
|
IF (bListRange) DO |
|
|
|
IF (there is a "current" range tracking element) DO |
|
|
/* Add up the sizes of any contiguous objects that |
*/ |
|
/* meet the selection criteria for our list. |
*/ |
|
Add the "current" object's size to the size of the |
|
|
"current" range tracking element |
|
|
END |
|
|
ELSE DO |
|
|
/* This is the first listworthy object we've |
*/ |
|
/* encountered lately. |
*/ |
|
/*** A GOOD BREAKPOINT ***/ |
|
|
Allocate a range tracking element and make it "current" |
|
|
Set the range base to the base of the "current" object |
|
|
Set the range size to the size of the "current" object |
|
END
END
ELSE
bListRange = FALSE |
|
IF (there is a "current" range tracking element) DO |
|
/* Did we just walk past the "upper" end of a set of |
*/ |
/* contiguous objects to be listed? |
*/ |
IF (!bListRange) DO |
|
/*** A GOOD BREAKPOINT ***/ |
|
Add the "current" range element to the range list |
|
Update local state so that there is no "current" range |
|
element |
|
END |
|
END |
|
find the object at the next "higher" address |
|
make it "current" |
|
END /* WHILE loop */ |
|
IF (there is a "current" range tracking element) DO /* Clean up loose ends. */
Add the "current" range element to the range list
END
RETURN the new list of ranges
Listing Two
/* The Destroy Range List algorithm |
*/ |
|
/* Frees all the memory associated with an outdated list of |
*/ |
|
/* contiguous address ranges. |
*/ |
|
/* Input parameter */ |
|
|
LIST |
a list of ranges |
|
/* All other variables are references to address range tracking |
*/ |
|
/* list |
elements. These include references to the "current" and |
*/ |
/* "next" elements in the list being destroyed. |
*/ |
look up the first range on the old list and make it "current"
WHILE (there is a "current" list element) DO
look up the next range on the old list and make it "next" delete the "current" list element
make the "next" element "current" END
free the list header
DDJ
http://www.ddj.com |
Dr. Dobb’s Journal, February 2006 |
51 |

Displaying GIF Images on
J2ME Mobile Phones
With some phones, you have to roll your own GIF decoder
TOM THOMPSON
Today’s mobile phones are versatile devices. Besides placing calls, they offer features such as Bluetooth connectivity, Internet support, built-in cameras that take pictures or capture video, and 3D graphics rendering capability, to name a few. Surprisingly, they sometimes lack certain important features. For instance, despite their ability to connect to and surf the Web, many mobile phones, from within their own Java environment, are unable to display a Graph-
ics Interchange Format (GIF) file.
I discovered this odd omission in graphics capability while writing a program that made use of a mobile phone’s GPS APIs. The idea was to get a GPS fix of the phone’s location (presumably its owner would be nearby) and present a map of the area. I thought that the hard part of the project would be to generate the location map, while the easy part would be to get the latitude and longitude position fix from the GPS APIs. I didn’t anticipate any problems displaying the map, given the mobile phone’s rich and varied graphics support.
Tom is a technical writer providing support for J2ME wireless technologies at KPI Consulting Inc. He can be contacted at tom_thompson@lycos.com.
Typical of software-project scheduling, I got it all backwards: Obtaining the map turned out to be the easy part. I discovered the U.S. Census Department’s Tiger server that, if provided latitude and longitude and the desired image size in pixels, renders a map and returns it as a GIF file to you. As you can see in Figure 1, the map is exquisitely detailed, using standard map colors for parks, cities, state boundaries, and highways labeled with their route numbers.
The hard part turned out to be displaying that GIF file. It took only several crashes in the debugging emulator to trace the problem to the Java 2 Mobility Edition (J2ME) platform, which is the phone’s Java runtime environment. Simply put, J2ME couldn’t decode and display GIF files. As the census server provides its maps only as GIF files, completing the project required that I roll my own GIF decoder for J2ME. In this article, I show how this was done. The complete source code that implements this technique is available electronically; see “Resource Center,” page 6.
The Law of Unintended Consequences
J2ME’s inability to display GIFs is particularly baffling considering that some mobile phones can download JPEG, BMP, and animated GIF files as wallpaper. How could this gap in graphics capability occur? The answer lies in the unintended consequences of how J2ME implements security and graphics.
The J2ME platform uses a “sandbox” model to carry out its security mechanism. MIDlets — mobile applications that are similar in design to a web browser’s Java applet — execute within a separate memory space that’s walled off from the phone’s native operating system and resources. MIDlets are free to play in the
sandbox, but can’t access any hardware or resources outside of the box by themselves. Access to the phone’s screen, audio system, and other hardware features is mediated through J2ME API classes. Simply put, if an API isn’t available for a particular hardware feature (such as Bluetooth), a MIDlet can’t use it. This prevents a malicious MIDlet from crashing the phone just before a critical call is placed, or tampering with the phone’s address book, or doing funny things with Bluetooth.
“J2ME’s inability to display GIFs is particularly baffling”
Per the specifications of J2ME’s support APIs, the execution environment only has to implement the display of Portable Network Graphics (PNG) images. Ironically, support for this one particular format has to do with the legal problems surrounding GIF files.
CompuServe invented the GIF format in the 1980s to handle the transfer and display of online graphics. Given the limited capabilities of the graphics hardware of the time, GIF files contained a 24-bit color table, with pixel color information represented as 8-bit values. The 8-bit value
52 |
Dr. Dobb’s Journal, February 2006 |
http://www.ddj.com |

limits GIF files to displaying only 256 colors. Because of the low bandwidth connections, the file contents were reduced in size through the use of a Lempel-Ziv- Welch (LZW) compression scheme. For a while, the fact that the LZW algorithm was patented wasn’t an issue. Then came the Web, and with it, the demand for graphic images soared. At the start of 1995, Unisys — the owner of the LZW patent — and CompuServe began aggressively pursuing fees for the use of GIF images.
This event precipitated the development of the PNG format as an alternative to the GIF format, and by late 1996, the PNG file format was adopted as a standard by the W3C. PNG is a lossless image format that supports images with pixel depths ranging from 1 bit (black-and-white), up to 48 bits (what’s termed “truecolor”), an alpha channel, and data compression. Crucially, PNG was carefully designed so that its data structures and compression algorithms didn’t infringe on any patents. For more information on the PNG file format, see “PNG: The Portable Network Graphic Format,” by Lee Daniel Crocker (DDJ, July 1995).
In 1999, when Sun Microsystems developed the J2ME platform specification and its supporting APIs, PNG was chosen as the default image format because of its graphics capabilities, small file size, and that it was unencumbered with patent issues. GIF was, to a large extent, sidelined as a graphics format.
The situation has improved for GIF since 2004, when the world-wide Unisys patents expired. While GIF has a limited color palette, this range is adequate for most graphic images, and its small size is still valuable for the mobile phone’s limited bandwidth wireless connections. Its animation feature is widely supported by all web browsers.
Because of GIF’s checkered history, the J2ME platform often doesn’t support the format. For the same reason, the native operating systems of many mobile phones don’t handle GIF files, either. Even if the phone’s native OS happens to support GIFs, the sandbox security mechanism blocks its use unless the vendor goes through the trouble to expose the GIF decoder routine to J2ME.
Checking for Native GIF Support
Once I knew what the problem was, how could I go about fixing it? It might seem that when your MIDlet receives a GIF file, the best course of action is to simply invoke the GIF-decoder routine. Practically, to conserve the phone’s limited working memory, you want to check for the presence of any GIF-decoding capability. J2ME has a collection of classes known as the Mobile Information Device Profile
(MIDP) that implement the MIDlet’s runtime environment and thus constitute its operating system. The MIDP 2.0 implementation provides utility methods that can query the host phone as to its capabilities and features.
To determine if J2ME handles a specific media format, you invoke the getSupportedContentTypes( ) method. This method returns a string array of supported media types, and includes the audio and video formats along with the image formats. The strings present this information in MIMEtype format. To check for GIF support, you scan this array, looking for the GIF MIMEtype string. If there is a match, then the phone’s J2ME implementation supports native display of GIF files. Listing One shows how to use getSupportedContentTypes( ) to obtain the media types array, along with loop code that does the array scan. The code sets the Boolean flag, gifSupported, if a match is found. Because this check only has to execute once, a good place for this code is in the class’s constructor.
If the MIDlet executes on one of those rare phones with GIF support, then displaying it becomes a matter of writing a few lines of code. J2ME stores its graphic data in an Image object. A method in this class, createImage( ), takes as an argument an array whose data is organized into a self-identifying image format such as PNG, GIF, and others. All that’s required is to route the data stream returned from the server into createImage( ), and it generates the Image object for you.
For the more common case, the mobile phone lacks GIF decoder support, so you must invoke the homemade GIF decoder class (call it GifDecoder) to handle the chore. You’d therefore make an instance of the GIF decoder object, supply it with the data stream, and invoke one of GifDecode’s methods (call it read( )) to read and decode the data. Another method (getImage( )) then extracts the converted image and returns it in an Image object. Listing Two shows the code that tests the state of the gifSupported flag and calls the appropriate method.
Undertaking the Port
Now all that was left to do was to close that gap in Listing Two by writing the GIF decoder class. Seasoned programmers don’t want to reinvent the wheel if they can help it, particularly because the odds were good that a Java-based GIF decoder already existed. Therefore, I immediately Googled to see what code was available on the Internet. Unfortunately, Sturgeon’s Law applies equally to Internet content: 90 percent of what the search turned up was junk. Still, the remaining 10 percent appeared promising and could be sifted through quickly.
I soon happened upon a lightweight Java-based GIF decoder written by Kevin Weiner, from FM Software. His decoder class, GifDecoder, has a small resource footprint, provides LZW decompression,
Figure 1: GIF map output by the Tiger server.
Figure 2: Format of a GIF89a animated image file.
http://www.ddj.com |
Dr. Dobb’s Journal, February 2006 |
53 |

and offers methods that can read an image either from a data stream or from a file (http://www.fmsware.com/stuff/gif
.html). The decoder could even parse animated GIF files and methods were provided to help animate such images. In addition, Kevin offered the source code to the community at large, with no stringent copyright conditions. GifDecoder was written for the J2SE platform, but porting this class to J2ME appeared manageable.
Figure 3: How an animated GIF image is displayed.
The port of GifDecoder was relatively painless, thanks in large part to the fact that J2ME’s capabilities are identical to J2SE in many areas. The largest variations between the two Java platforms lie in their GUI classes. Because GifDecoder processes data behind the scenes for other objects, this averted dealing with the GUI differences.
I started by commenting out those GifDecoder methods I didn’t need for my project. I eliminated GifDecoder’s file read( ) method because my application receives its images over the air through an HTTP connection. I would have handled this differently if the target phones used either the Symbian OS or the JSR 75 FileConnection API, both of which provide file access. However, I did modify the file read( ) method to retrieve data from MIDlet resources. Resources are part of the MIDlet’s archive file, and often store the graphic images the MIDlet uses to draw its GUI.
Next, I changed all instances of
BufferedInputStream to DataInputStream. The BufferedInputStream class isn’t available on J2ME, and DataInputStream’s capabilities were adequate for the job. I did spend some time struggling with getting the code to read the GIF file’s color tables. I tracked this down to a bug where, when reading an HTTP stream, DataInputStream’s read( ) method can throw an exception if the amount of bytes for the requested read is larger than the stream buffer’s size (255 bytes). The workaround was to write a loop that used readByte( ) to fetch the color table, byte by byte.
Another modification was to replace Image’s getRaster( ) J2SE method with a pair of MIDP 2.0 methods, getRGB( ) and
Result |
Constructor/Method |
Description |
Constructor |
|
|
GifDecoder |
GifDecoder(void) |
Constructor that generates an instance of the GIF |
|
|
decoder object. |
Methods |
|
|
int |
getDelay(int frameNumber) |
Returns the delay time, in milliseconds, associated |
|
|
with the image specified by frameNumber. |
Image |
getFrame(int frameNumber) |
Returns the image specified by frameNumber. |
int |
getFrameCount(void) |
Returns the number of images (frames) in an |
|
|
animated GIF file. |
Image |
getImage(void) |
Returns the first image in an animated GIF file. |
int |
getLoopCount(void) |
Returns a value that represents the times the |
|
|
animation should loop. |
int |
read(DataInputStream is) |
Reads the referenced DataInputStream object. |
|
|
Returns a status code of zero if no errors occurred |
|
|
during the read. |
int |
read(InputStream is) |
Reads the referenced InputStream object. Returns |
|
|
a status code of zero if no errors occurred during |
|
|
the read. |
int |
read(String name) |
Reads the named resource in the MIDlet’s JAR file. |
|
|
Returns a status code of zero if no errors occurred |
|
|
during the read. |
|
|
|
Table 1: GifDecoder class and methods.
createRGBImage( ). J2SE’s getRaster( ), in one swoop, converts the decoded array of RGB pixels that comprise the picture into the Image object’s native format. Because J2ME lacks getRaster( ), the data translation became a two-step process. The getRGB( ) call converts the decoded GIF pixels into integers, and the subsequent call to createRGBImage( ) translates the integer array into the Image format.
GifDecoder is also able to read GIF files that contain multiple images for animation, plus any control blocks that contain a delay interval associated with each image. See Figure 2 for the structure of an animated GIF file. On J2SE, GifDecoder stores the images and delay values using the ArrayList class. ArrayList isn’t implemented on J2ME, but its superclass, Vector, is. So I modified all appearances of ArrayList to use Vector, and rewrote the code to use Vector’s methods.
The entire process didn’t take long, and I had an example GIF appearing on the phone’s screen within a few days. The bulk of that time was spent tracking down and identifying the read( ) bug.
Using the GIF Decoder
Adding the GIF decoder in a mobile application is easy: Just include the GifDecoder.java file along with the MIDlet’s source files. In your application code, make an instance of GifDecoder, then use the appropriate methods to read and display the GIF images. Table 1 documents the methods available. Listing Two shows how it’s done. Once you’ve made an instance of GifDecoder, you invoke the appropriate read( ) method, depending upon whether you obtained the GIF file from a data stream or are reading a MIDlet resource. If the read completes without errors, then you call getImage( ), which returns the GIF picture in an Image object. If getImage( ) is applied to an animated GIF file, this method returns the first image in the file.
Presenting an animated GIF is relatively straightforward, thanks to the utility methods GifDecoder provides. You first use getFrameCount( ) to fetch the value that specifies how many images (or frames) make up the animation. This becomes the termination value for a loop counter. Within the loop body, you pass the loop’s index as an argument to both getDelay( ) and getFrame( ) to obtain each frame’s delay time and its corresponding image, respectively.
The delay values represent time intervals in hundredths of a second. Because most computers and cellphones have timers with millisecond resolution, getDelay( ) multiplies the value it obtains by 10 to convert the interval value into milliseconds. The result from getDelay( )
54 |
Dr. Dobb’s Journal, February 2006 |
http://www.ddj.com |

can therefore be jammed straight into J2ME’s Thread.sleep( ) method to implement the delay. GifDecoder supports the drawing options specified in the GIF file’s graphic-control extension blocks. It also supports a transparent color, although the phone’s J2ME implementation can affect this feature’s behavior.
You’ll execute the GIF animation code in a separate thread, either as part of a Thread class or as part of another class’s run( ) method. This is done so that the MIDlet’s UI — which runs in another thread — is free to respond to any events that the user generates. Listing Three shows how to use a run( ) method to read and display the images in an animated GIF file. Figure 3 shows how the successive images in a GIF can present the animation of a UFO. The serial serviceRepaints( ) method acts as a simple synchronization mechanism in that it ensures that the image is
drawn completely before the next delay interval and image are fetched and displayed. An example MIDlet, GIF_Display, is available for download, and it displays several animated GIF images. Note that a phone that natively supports the GIF format may only display static images, not animated ones.
When compiled, GifDecoder is only 11 KB in size, suitable for use in any MIDlet that has special graphics needs. Note that this size is before an obfusticator is applied to GifDecoder to further reduce its code footprint. The downside to using GifDecoder is that large and lengthy image animations can consume a lot of memory. An animation that consists of 24 144×52-pixel images can easily consume around 60 KB of memory. On a phone with 256 KB of working memory, that consumes over 20 percent of available memory just to support a graphic. You can re-
duce the animation’s memory footprint by using smaller images and fewer of them.
Conclusion
While a GIF image has a limited color palette, it’s often adequate for most graphics purposes, including the display of some photos. In exchange for the limited color range, GIF images require less storage than JPEG images. In addition, there are plenty of GIF editing animation tools that allow you to easily add your own graphics and animations for addition to a MIDlet’s UI. The ability to read, display, and animate GIF images that appear on many Internet sites is an important asset to have for any MIDlet performing network tasks. GifDecoder is thus a valuable tool to have in your J2ME programming toolbox.
DDJ
Listing One
private String |
mediaTypes[]; |
private final |
String GIF_MIME_TYPE = "image/gif"; |
private boolean |
gifSupported; |
//Get the media types to check for support of GIF file display mediaTypes = Manager.getSupportedContentTypes(null); int count = mediaTypes.length;
//Check list for GIF MIME type; set support flag if present gifSupported = false;
for (int i = 0; i < count; i++) {
if (mediaTypes[i] == GIF_MIME_TYPE) gifSupported = true;
} // end for
Listing Two
private static final String READ_OK = 0
String |
url = "http://www.nosuchsite.net/StarTrek/enterprise.gif"; |
HttpConnection |
hC = null; |
DataInputStream dS = null; |
|
Image |
mapImage = null; |
//Open the connection as an HTTPConnection; send request try {
hC = (HttpConnection) Connector.open(url); hC.setRequestMethod(HttpConnection.GET); hC.setRequestProperty("IF-Modified-Since",
"10 Nov 2000 17:29:12 GMT"); hC.setRequestProperty("User-Agent","Profile/MIDP-2.0
Configuration/CLDC-1.1"); hC.setRequestProperty("Content-Language", "en-CA");
}catch (IOException e) { } // running without safety net!
//Read the data stream for the returned GIF image
int iByteCount;
iByteCount = (int)hC.getLength(); dS = hC.openDataInputStream();
//Does J2ME implementation support native GIF format decode? if (gifSupported) {
mapImage = Image.createImage(dS);
}else {
//No, do it ourselves: get instance of GIF decoder and decode stream GifDecoder d = new GifDecoder();
if (d != null) {
int err == d.read(dS); if (err == READ_OK) {
mapImage = d.getImage();
}//end if
}end if
}// end else
Listing Three
// The run method for the class. |
|
public void run() { |
|
int t; |
|
if (gifImage != null) { |
|
while (action) { |
|
int n = d.getFrameCount(); |
// Get # of frames |
for (int i = 0; i < n; i++) { |
// Loop through all |
gifImage = d.getFrame(i); |
// Get frame i |
// Delay duration for frame i in milliseconds |
|
t = d.getDelay(i); |
// Get frame's delay |
repaint(); |
|
serviceRepaints(); |
|
try { |
|
Thread.sleep(t); |
// Delay as directed |
}catch (Exception ex){}
}// end for
}// end while
}// end if
}// end run
DDJ
http://www.ddj.com |
Dr. Dobb’s Journal, February 2006 |
55 |

Sudoku &
Graph Theory
Algorithms for building solvers
EYTAN SUCHARD, RAVIV YATOM, AND EITAN SHAPIR
Sudoku is a logic puzzle in which there are 81 cells (vertices) filled with numbers between 1 and 9. In each row, the numbers 1,2,3,..,9 must appear
without repetition. Likewise, the numbers 1,2,3,..,9 must appear without repetition in the columns. In addition to the row and column constraints, the numbers 1,2,3,..,9 must appear in the nine nonoverlapping 3×3 subsquares without repetition. So in short, the puzzle board is separated into nine blocks, with nine cells in each block (see Figure 1).
There are several possible rules you can use to successfully fill in missing numbers. In this article, we examine two rules — Chain Exclusion and Pile Exclusion — for solving Sudoku puzzles. These rules are at the heart of a Windows-based Sudoku solver that we built using Visual C++. Executables and the complete source code for this solver are available electronically (see “Resource Center,” page 6). The goal of this logical Sudoku solver is to prove that only one possible number can be assigned to each vertex, and to find that number for each vertex in which the num-
Eytan, Raviv, and Eitan are software engineers in Israel. They can be contacted at esuchard@012.net.il, ravivyatom@bezeqint
.net, and eitans@ima.co.il, respectively.
ber is not defined. Illogical Sudoku puzzles can also be solved, but require guesses (Implementation, OK button).
We refer to possible numbers that should be assigned to a row, column, or one of the nine 3×3 subsquares as a “Permutation Bipartite Graph” or nodes. A node consists of a vector of n>1,n=2,3,4… vertices and all possible numbers that can be assigned to these vertices, such that there exists at least one possible match between the vertices of the vector and the numbers 1,2,…n.
For example, the following are nodes:
({1,2,3,5},{2,3},{2,3,4},{3,4},{4,5}, n=5 ({1,2,3,7},{3,6},{3,4},{1,4},{5,6,7},{4,6},{2,7},
{8,9},{8,9}, n=9
A possible match for the first vector is easy:
1 -> {1,2,3,5}
2 -> {2,3}
3 -> {2,3,4}
4 -> {3,4}
5 -> {4,5}
A possible match for the second vector is more tricky:
2 -> {1,2,3,7}
3 -> {3,6}
4 -> {3,4}
1 -> {1,4}
5 -> {5,6,7}
6 -> {4,6}
7 -> {2,7}
8 -> {8,9}
9 -> {8,9}
A number can be only assigned to a vertex that contains the possibility of assigning that number. For instance, only the following possibilities are accepted:
7 -> {2,7} or 2 -> {2,7}.
Pile Exclusion and Chain Exclusion provide the basis of logical elimination rules.
To understand Pile Exclusion, consider the following nodes:
({1,2,3,5},{3,6},{3,4},{5,6},{1,7,8,9},{4,6},{5,7,8,9}, {4,6},{6,7,8,9},{1,4}, n=9
“There are several possible rules you can use to successfully fill in missing numbers”
The numbers 7,8,9 appear only in three vertices:
{1,7,8,9},{5,7,8,9},{6,7,8,9}
Because there is at least one possible match in the Permutation Bipartite Graph, one vertex will be matched to 7, one to 8, and one to 9. Thus, you can erase the other numbers from these three vertices to get the following three augmented vertices:
{1,7,8,9} -> {7,8,9} {5,7,8,9} -> {7,8,9} {6,7,8,9} -> {7,8,9}
and the entire Permutation Bipartite Graph becomes:
({1,2,3,5},{3,6},{3,4},{5,6},{7,8,9},{7,8,9},{4,6}, {7,8,9},{1,4}), n=9
56 |
Dr. Dobb’s Journal, February 2006 |
http://www.ddj.com |