Trouble with Random Long-Running request in ColdFusion 8

I've been experiencing some run-time weirdness with ColdFusion for the past year or so, and I've finally decided to post my observations and see if anyone else has been having similar troubles.

The trouble shows up on any number of scripts, but is most likely to appear with scripts that are called frequently. The screenshot I've included shows a Slow Request report from the ColdFusion Server Monitor. You can see that this request took 110 seconds to complete. The VERY strange part is that the runtime of the Application.cfc onRequest method (the outermost piece of code to run on any request) took only 468ms to complete. So what happened to the 109 seconds that were not spent executing my request? How can I prevent this from happening?

Also observe the Min/Max/Avg response times for the script. At the time of this screenshot, this script had been called 250 times. Multiplying the average response time (.505 seconds) by the request count (250) tells us that this script has occupied 126.25 seconds of server time. Removing this one long running request (250-1, 126.25-110.234) from the statistics, we find the script has an average running time of 0.064 seconds, which is a more reasonable run-time, considering the reported min response time of 0.015 seconds.

Eliminating these random long-running requests will do great things for the usability of the website, as well as free up server resources.

Misc Details

  • ColdFusion Enterprise 8,0,1,195765
  • Monitoring and Profiling Enabled
  • Windows 2003
  • Java 1.6.0_04

Summary

Scripts will randomly take MUCH longer then they usually do.

There seems to be a massive discrepancy between the Response Time and the Time Taken by the onRequest method.

Why is this happening, and what can be done about it?

UninvocableEntityException: Puzzling ColdFusion Exception

So I'm calling a function in the same CFC, and I get the following error:

Entity has incorrect type for being called as a function.
The symbol you provided getProductColors is not the name of a function.

Intersting. The top of the stack trace reads:

coldfusion.runtime.CfJspPage$UninvocableEntityException: Entity has incorrect type for being called as a function.
   at coldfusion.runtime.CfJspPage._invokeUDF(CfJspPage.java:2441)
   at coldfusion.tagext.lang.InvokeTag.doEndTag(InvokeTag.java:432)
   at cfceilume2ecfc2110149128$funcSAMPLES.runFunction(D:\site\insideDev\com\ew\feed\ceilume.cfc:71)

While I have no idea why the error occurred, I was able to make the problem go away by modifying the name of the method I was calling, changing it from getProductColors to getProductColorList.

Strange. A Google Search only returned one result, so I thought I'd contribute a little info in case someone else finds the same error.

jEditable TinyMCE Plugin

Over the past view weeks, I've been working with Mika Tuupola, author of the jEditable jQuery extension. jEditable makes edit in place forms easy, and is itself extensible, allowing the creation of custom edit types, such as a date picker.

I've been using TinyMCE as a richtext editor, and decided to create a custom edit type for jEditable that makes TinyMCE easy. Mike added a few additional customization hooks that enabled me to complete the plugin, and I've included example code below. This example is not meant to be a full and complete explanation, so make sure you dig into the jEditable and TinyMCE documentation.

A big thanks to Mike for his help. He has released the updated jEditable code that contains some additional hooks needed by my example code below.

First, I include the necessary javascript files for each library. Your specific versions or names may vary:

<script src="/resource/tiny_mce/tiny_mce_src.js" type="text/javascript"></script>
<script src="/resource/jquery/jquery-1.2.3.js" type="text/javascript"></script>
<script src="/resource/jquery/jquery.jeditable.js" type="text/javascript"></script>

Then, I setup TinyMCE, and the TinyMCE jEditable plugin as well:

<script type="text/javascript">

$.fn.tinymce = function(options){
   return this.each(function(){
      tinyMCE.execCommand("mceAddControl", true, this.id);
   });
}

function initMCE(){
   tinyMCE.init({mode : "none",
      theme : "advanced",
      theme_advanced_toolbar_location : "top",
      theme_advanced_toolbar_align : "left",
      theme_advanced_statusbar_location : "bottom",
      theme_advanced_buttons1 : "bold,italic,underline,strikethrough,separator,bullist,numlist,undo,redo,link,unlink",
      theme_advanced_buttons2 : "",
      theme_advanced_buttons3 : "",
      theme_advanced_resizing : true});
}

initMCE();


$.editable.addInputType('mce', {
   element : function(settings, original) {
      var textarea = $('<textarea id="'+$(original).attr("id")+'_mce"/>');
      if (settings.rows) {
         textarea.attr('rows', settings.rows);
      } else {
         textarea.height(settings.height);
      }
      if (settings.cols) {
         textarea.attr('cols', settings.cols);
      } else {
         textarea.width(settings.width);
      }
      $(this).append(textarea);
         return(textarea);
      },
   plugin : function(settings, original) {
      tinyMCE.execCommand("mceAddControl", true, $(original).attr("id")+'_mce');
      },
   submit : function(settings, original) {
      tinyMCE.triggerSave();
      tinyMCE.execCommand("mceRemoveControl", true, $(original).attr("id")+'_mce');
      },
   reset : function(settings, original) {
      tinyMCE.execCommand("mceRemoveControl", true, $(original).attr("id")+'_mce');
      original.reset();
   }
});
</script>

This next block of code initializes TinyMCE, first as a regular jQuery plugin (not in place editable), and then as the jEditable enabled in place editor:

<script type="text/javascript">
$(function(){
   $('#foo').tinymce();
   
   $("#bar").editable(function(value, settings){
      console.log(this, value, settings);
      return value;
   
   }, {
      type : 'mce',
      submit : 'OK',
      indicator : "Saving...",
      tooltip : 'Click to edit...',
      width : '500px',
      height : '100px'
   });
});

</script>

In this example, both of the selected elements are HTML textareas. Clearly, your settings for jEditable and TinyMCE will change with your own needs, but this should get you started.

Finally, a big thanks to Mike, for writing the jEditable plugin in the first place, and for being so supportive and patient in helping me write this TinyMCE plugin!

Edited 24Jan09: Applied IE6 Fix as recommended by Rick in comments.

Google IO: State of Ajax

My first session here at Google IO was on the State of Ajax. I wasn't there till the end, but the main message seemed to be that Ajax is good.

More specifically, If you are writing Javascript apps without a library, stop it! They do nice things to handle cross browser differences and nice features like animation.

Some of the libraries also provide nice UI widgets and things as well.

So seriously. Start using a library. jQuery is my personal favorite.

Using an arbitrary sort order in a SQL ORDER BY clause

Yesterday, I spent a minute helping a coworker with a SQL problem: He needed to order the results by a varchar field, but not in the easily available lexical order. The order was set by the marketing folks, and our task was to make it work.

There are three ways to do this, and I'll talk about each one.

First, you can perform the sorting after the query is done, in whatever dynamic language you are using. This is probably bad for several reasons.

Second, you could create a table that contains the values to be sorted and a field that contains the sort order in numerical fashion. This is a pain.

Third: Use a scalar function to get things to fall into place. Observe:

ORDER BY CHARINDEX(myField, 'Foo,Bar,Baz,Fizz') ASC

Quick and easy.

Blog.cfc posting with Windows Live Writer

I've become attached to Windows Live Writer as a blogging client. It is much nicer to use then the web based client, and it allows saving local and remote drafts of posts.

I've just had to setup my laptop again, and had to hunt down the settings to allow Live Writer to talk to Blog.cfc (gifted to us by the famous Ray Camden). I've decided to document this for my future needs, and hopefully help anyone else with the same problem.

Configure Live Writer to use the MetaWeblog API, with the following URL: http://example.com/xmlrpc/xmlrpc.cfm (with the correct domain and path to your blog.cfc installation, of course.)

Other blogging clients may use similar settings. If you haven't tried using a blogging client, give it a try.

Designing an Interface - SimpleDB CFC

I'm working on a ColdFusion library for Amazon's SimpleDB, and I'm mulling over the options for how I want the library to work. I've completed the authorization code and the methods that deal with creating, listing, and deleting domains. They exist well as function calls, with the listing method returning a query object for easy iterating.

The two pieces that remain are the methods that deal with items (records within the domain) and queries against a domain. Here are a few options that I have, and I'd love some feedback!

Item Methods

SimpleDB allows you to add attributes to an item and remove them, as well as retrieve all (or a subset) of the attributes of an item.

There are two main approaches here: Present an Item object with methods, or present the methods within the main CFC itself.

Presenting an Item as an object would work like this:

item = simpledb.getItem(domainname, itemname)
item.setAttribute(attributename, attributevalue, replace=false)
item.deleteAttribute(attributename)
foo = item.getAttribute(attributename)
item.persist()

Presenting methods from the main cfc would look like this:

simpledb.setAttribute(domain, item, attribute, value, queue=false)
simpledb.deleteAttribute(domain, item, attribute)
foo = simpledb.getItemAttribute(domain, item, attribute)
mystruct = simpledb.getItemAttributes(domain, item)
simpledb.persistChanges(domain, [item])

In either case, changes could be queued in the system until persist() or persistChanges() was called. This way, multiple attribute sets could be combined together to speed processing.

The examples I provide here are a bit rough, but I'm sure you get the idea.

Which of these options is going to be more simple / coldfusion like?

Queries

The SimpleDB API returns a list of itemnames to queries. This means that to access the data, you need to query the attributes for each item. The simplest option would be to return the list of itemnames, and allow use of the other methods to get the desired attributes.

A more full-featured interface would have the option to get the attributes for each item and form it into a query. An iterator object could be another option, with either attribute access methods or simply returning an item object similar to the one mentioned in the item section.

Plea For Thoughts

So, does this rambling question make any sense? Which style (thin and rough, fat and friendly) of API interface do you prefer?

If my thoughts need to be better formed before any worthwhile answer can be given, please let me know.

Amazon Web Services SQS Price Changes

Amazon Web Services announced a price change for SQS today. They changed prices, but also how the prices were charged. Some are claiming huge savings. I did a little math to see who it would affect and how much.

The price changed from $0.10 per 1,000 messages to $0.01 per 10,000 requests. Charging by request means that pushing a message and asking for messages counts as two requests. You can make 100 requests (new pricing) for the cost of 1 message (old pricing). This means that if you poll 99 times for every message you pass through the queue, you will pay exactly the same amount.

If you poll once every 5 minutes, 3 messages per day will have you break about even. Polling once a minute, it takes 15 messages per day to break even. Systems of this size will not see much change. The monthly fee with 15 messages/1 minute polling is only 5 cents (not counting bandwidth) anyway.

Larger systems will see a much bigger difference. If you have 10 servers polling a queue once per second, you will need 8640 messages per day to break even. But seriously: Your monthly fee for a system like this? $26.44 (not counting bandwidth).

In either case, clever polling schemes (fallback, based on time of day, etc) could save some money. If the other end of your queue is a scalable EC2 layer, and you only run servers when there is stuff in the queue, then your savings are likely to be pretty good.

So in any case, it's all still pretty cheap.

Sample HTML Apps with Adobe AIR

I've been playing with Adobe AIR recently, with the intent to create an application. After some struggles making the switch from Beta 2 to Beta 3, I feel like I'm finally getting things worked out.

Before diving headlong into the project, I spent some time with the sample applications provided on Adobe Labs. There are several HTML based apps (AIR can be Flash/Flex based as well), so I had plenty of examples to observe. They even have a little view source screen that each includes, making it easy to learn from the examples.

The first thing I noticed was that with a few minor exceptions, all the HTML applications were created with a single file. Inside the file, there is generally a div for each screen, and some JavaScript magic to move between the screens of the application. This struck me, as I was expecting to develop the app in a way similar to a website, where you typically have multiple files, and move between them for different sections of your application.

Having it all in one file does leave it quite a bit more cluttered, although the apps seem to work well and be well designed. The JavaScript in particular was very well organized.

My second observation has to do with the fact that not a single app contained a non-application sandbox. AIR uses sandboxes to deal with certain security problems. The sandbox can be a little difficult to deal with, and it looks like the sample app developers left it well enough alone.

This leads me to my third observation: AIR needs some jQuery like beauty. Dealing with all the APIs and things can get a little messy, and some jQuery like syntax would help quite a bit. Unfortunately, jQuery itself doesn't seem to fare well within the sandbox confines of AIR, as it seems to use eval() in some difficult ways. Perhaps a jQuery plugin can make the necessary modifications so that jQuery is usable for app development. (I've been spoiled by jQuery, can you tell?)

Beyond these observations, I can't say much about app development yet, although I hope to be able to soon.

ColdFusion Powered Label Printing

My current project at work requires labels printed on demand, each label custom to the moment they are printed. Previously, labels were printed in batch mode on an inkjet printer, and then applied to each box during the manufacturing process. We are changing the process so that each label contains information about the manufacturing process, including the machine, operator, identification of the plastic material used, and the quantity in each box.

The quantity one is tricky, as the number of items in each box is sometimes adjusted by the machine operator. A computer at the machine is used to show the planned production schedule to the machine operator. We are adding some functionality so that the operator can input the desired information (including quantity) during the manufacturing process. We then needed a simple way to print a label with that information.

We chose a Zebra label printer that ships with Windows drivers. We needed a way to prepare a label in a way that looks professional, and is easy to print. ColdFusion has several ways to generate printable files (both PDF and FlashPaper). The CFDocument tag is very easy to use, but we have struggled in the past with precise positioning of elements on the page. This time, I used the ColdFusion Report Builder to lay out the label with the visual tools.

At runtime, I pass in the information I need and generate the label with the CFReport tag. This tag has the ability to either save the generated file to disk, or return the file in a binary variable. (It will also return it to a browser, which I used in testing.) CF8 Introduced the CFPrint tag, which conveniently accepts a binary variable and prints it to a configured network printer. Sweet.

(Not saving the file to disk prevents the creative coding necessary to avoid threading issues, and also does not require us to clean up the temporary files.)

So it works like this: The operator enters machine information at the start of each job. As he finishes the number of items to be put in a box, he taps the 'Box Done' button, which pops up a dialog box that allows changes to the default quantity of items in a box. He hits 'done', and a label spits out of the attached printer.

The best part is that there is no print dialog box that the operator has to deal with. It just prints.

High 5 to the CF Team for making this possible.

More Entries