Launch JavaFX Media Browser Application Download NetBeans Project

Introduction

Earlier in Module 3 the number of search results was limited to 50 - the maximum number of results returned by the Yahoo API. In this module you get more results by taking advantage of a field in the API that specifies the starting number of the search. Having more search results raises the issue of loading all the thumbnails. This task shows how to create a mechanism to control the number of thumbnails loaded at one time.

Running the Project

1. Download the Module 3 Task 3 NetBeans project and open the NetBeans IDE.
   
   
2. Run the project.
   
   Because more images are being loaded the scroll bar behavior changes. As shown in Figure 1, the scroll overlay, which represents the visible thumbnails, is centered horizontally. As you scroll to the right the ridges representing columns of thumbnails move to the left and the overlay re-centers itself. If you have a large number of search results the scroll bar fades off both ends of the window.
   
   If you scroll quickly you will encounter thumbnails that are not "preloaded" in which case you see placeholders until the thumbnails are populated. Loading is managed by the ThumbnailController, as described in Throttling Thumbnail Loading.

Figure 1

Architecture

This task adds one new class, ThumbnailController.fx, and modifies existing classes as shown in blue in Figure 2.

Figure 2

Getting More Search Results

WebSearch.searchLocation is the URL used by the HttpRequest. In YahooAPI, it is defined according to the Yahoo image REST query specification. In module 2, task 1, searchLocation is initialized in YahooAPI as:

    override var searchLocation = bind
           "http://search.yahooapis.com/ImageSearchService/V1/imageSearch"
           "?appid=={yahooSearchAppID}&query={yahooSearchQuery}"
           "&results={numberOfYahooResults}";

Per the Yahoo image search API, this query will return a maxium of 50 results. The Yahoo API has a start parameter which can be set to the starting results position. Therefore, in order to get more search results, another HTTP GET request must be sent with start set to the next block of results.

The approach in the code is to have a sequence of HttpRequests. When the first request is done, the next request is sent, and so on until the end of the sequence is reached. In WebSearch.fx searchLocation is changed to a sequence:

    package var searchLocation : String[];

In YahooAPI.fx, which extends WebSearch, the searchLocation sequence is initialized as follows:

    def yahooSearchLocation = bind
           "http://search.yahooapis.com/ImageSearchService/V1/imageSearch"
           "?appid=={yahooSearchAppID}&query={yahooSearchQuery}"
           "&results={numberOfYahooResults}"
       on replace {
            searchLocation = for (i in [1..500 step numberOfYahooResults]) {
                "{yahooSearchLocation}&start={i}"
            }
        };

Here, whenever the yahooSearchQuery changes, a new searchLocation sequence is created. The chain of events that causes this to happen begins in SearchTextBox.fx where the WebSearch instance's searchQuery variable is set to the value of the text box. The trigger on searchQuery in YahooAPI replaces spaces with '+' for "include terms" and sets the local var yahooSearchQuery. Because yahooSearchQuery is changed, bind on yahooSearchLocation is evaluated and the on-replace block creates a new sequence of searchLocations.

In WebSearch, some changes had to be made to accommodate searchLocation as a sequence.

def httpRequest : HttpRequest[] = bind for (loc in [0..<sizeof searchLocation])
 {
    HttpRequest {

    method: HttpRequest.GET

        location: bind searchLocation[loc]

         onResponseMessage: function(msg:String) {
            if ( httpRequest[loc].responseCode != 200 ) {
                println("HTTP response 
                         {httpRequest[loc].responseCode}: {msg}");
            }
        }

        onException: function(ex: Exception) {
            ex.printStackTrace();
        }

        onDone: function() {
            (httpRequest[loc].context as HttpRequest).enqueue();
        }

        onInput: function(is: InputStream) {
            try {
                pullParser.input = is;
                pullParser.parse();
            } finally {
               is.close();
           }
        }
    }
 }

Notice that the code now creates a sequence of HttpRequests. The for loop here is very similar to that in Wall which creates the thumbnails sequence from the metaData sequence. Notice also the use of the sequence slice in the range expression of the for loop which gives the loop an upper bound of sizeof searchLocation - 1.

The main difference in this block of code, other than the sequences, is the implementation of the HttpRequest.onDone function. Notice the use of the HttpRequest.context variable. HttpRequest itself does nothing with context, so the variable can be assigned any arbitrary value. Here the code is using context to hold a reference to the next httpRequest to be sent. Thus, when one request is done, the next request in the chain is sent.

The request chain is set up in the following block, which is executed after the instance has been initialized.

    postinit {
        // Chain the Requests}
        for (i in [1..<sizeof httpRequest]) {
            httpRequest[i-1].context = httpRequest[i];
        }
    }

Throttling Thumbnail Loading

As the data returns from each request, it is appended to the results sequence and new Thumbnail instances are created in the Wall. All of the images for those thumbnails do not have to be loaded, however, and doing so incurs quite a bit of overhead in terms of processing and memory consumption. To overcome this, the code has a rate limiting feature wherein only thumbnails in the immediate "neighborhood" of the scene are loaded. The workings of this feature are described herein.

Three classes come into play: Thumbnail, a new class called ThumbnailController, and (to a minor extent) Wall. The basic architecture is that Thumbnail will not load its image unless told by the ThumbnailController to do so. Thumbnail decides that it should be loaded by triggering off its boundsInScene variable. If the Thumbnail is in the viewable neighborhood, it adds itself to the ThumbnailController's queue with a request to be loaded. Likewise, if the thumbnail is no longer in the viewable neighborhood, it queues itself to be unloaded. The ThumbnailController services the queue telling, in turn, the Thumbnail to load (or unload) its image.

The following code from Thumbnail.fx loads the thumbnail image and should look somewhat familiar. The difference from previous tasks is that the Image is only loaded if the variable loadStatus is set to Constants.THUMB_LOAD. Otherwise, the image is one of the placeholder images.

    function checkLoadStatus() : Void {
        if (loadStatus == Constants.THUMB_UNLOAD) {
            if (metaData.media_type == MetaData.type_image) {
                image = Constants.PHOTO_PLACEHOLDER;
            } else {
                image = Constants.VIDEO_PLACEHOLDER;
            }
        } else {
            if (metaData != null) {
                image = Image {
                    url: metaData.thumb.url
                    placeholder:
                        if (metaData.media_type == MetaData.type_image) {
                            Constants.PHOTO_PLACEHOLDER
                        } else {
                            Constants.VIDEO_PLACEHOLDER
                        }

                    width : Constants.THUMB_WIDTH * Constants.EXPANDED_THUMB_SCALE
                    height: Constants.THUMB_HEIGHT * 
                            Constants.EXPANDED_THUMB_SCALE
                    preserveRatio : true
                    backgroundLoading: true
                };
            }
        }
    };

When the loadStatus flag changes value, the checkLoadStatus() function is called. The loadStatus flag is set from ThumbnailController, as will be seen later, but the Thumbnail instance must be in the ThumbnailController's queue first. The Thumbnail itself decides whether or not to add itself to the queue by triggering on its boundsInScene value.

    var desiredLoadStatus = bind {
        if (boundsInScene.maxX >= -Constants.STAGE_WIDTH and
            boundsInScene.minX <= 2 * Constants.STAGE_WIDTH) {

            true;
        } else {
            false;
        }
    }

If the Thumbnail is within the viewable neighborhood, which is defined here to be three stage widths wide, then desiredLoadStatus will be true. It could be, however, that the Thumbnail is already enqueued, so a determination must be made to either add to the queue, remove from the queue, or do nothing. This is accomplished by this trigger:

    var loadStatusTrigger = bind (not controller.scrolling) and
                                 (desiredLoadStatus != requestedLoadStatus) on replace {
        if (loadStatusTrigger) {
            requestedLoadStatus = desiredLoadStatus;
            controller.queueAction(this,
                                   if (requestedLoadStatus) {
                                       Constants.THUMB_LOAD
                                   } else {
                                       Constants.THUMB_UNLOAD
                                   },
                                   priority);
        }
    }

The call to controller.queueAction places the Thumbnail into the ThumbnailController queue. The priority flag is set to true if the Thumbnail viewable.

ThumbnailController has a queue and a set of loaders that process the queue. LoadSpec is a container that holds a reference to the Thumbnail and the action (load or unload). ThumbnailLoader is more interesting and is discussed further below. Both LoadSpec and ThumbnailLoader are defined in ThumbnailController.fx.

    var queue: LoadSpec[];
    var loaders: ThumbnailLoader[];

The queueAction() function handles managing the queue. The code is heavily commented, but suffice it to say that the end result is that a LoadSpec is either removed from the queue or added to the queue. The main loop that processes the queue is called serveQueue().

    function serveQueue() {
        while (sizeof queue > 0 and sizeof loaders < 7) {
            var spec = queue[0];
            delete queue[0];

            var loader = ThumbnailLoader {
                thumbController: this
                thumbnail: spec.thumbnail
            }
            insert loader into loaders;
            loader.load(spec.action);
        }
    }
}

Notice that a maximum of seven loaders are allowed at any time, which means that there can only be a maximum of seven images loading at any time. This prevents a tight loop calling into Image which can make the UE unresponsive (since all the image loads are in the background). The serveQueue() function does the ordinary dequeue of the head of the queue by saving a reference to the head of the queue and then deleting the first element of the queue. A new ThumbnailLoader is created. The thumbController back-reference is needed since serveQueue() may be called from the load() function. Finally, the loader is inserted into the loaders sequence and then the loader's load function is invoked.

    function load(action: Integer) {
        if (thumbnail.loadStatus != action) {
            if (action == Constants.THUMB_LOAD) {
                loading = true;
            }
            thumbnail.loadStatus = action;
        }
        if (not loading) {
            delete this from loaders;
        }
    }

Thumbnail.loadStatus is finally set in ThumbnailLoader.load. This causes the thumbnail image to load, or to revert to the placeholder image, depending on the value of action. After the thumbnail's load status is triggered, the loader is removed from the set of loaders provided that the image is being unloaded. Keep in mind that image unloading is simply replacing the Thumbnail image with the static placeholder image and, therefore, incurs no overhead to speak of.

But if the Thumbnail image is being loaded, the ThumbnailLoader remains in the queue until the image is loaded, thus preventing another image load while this one is in progress (allowing for seven simultaneous image loads). ThumbnailLoader watches the Thumbnail's image progress. When the progress reaches 100%, the ThumbnailLoader is removed from the queue to make way for another image to be loaded.

    var progress = bind thumbnail.image.progress on replace {
        if (loading and progress == 100) {
            delete this from loaders;
            thumbController.serveQueue();
        }
    }

It is necessary to give the queue a kick with the call to serveQueue() since all of this code is running on the same thread.

Try It

• To see the request chain in action, modify WebSearch.fx to change the onDone function as follows:

    onDone: function() {
        var next = httpRequest[loc].context as HttpRequest;
        println("{next}");
        next.enqueue();
    }