Gary's Gambit

Buying online from Slow-dobe

2011-05-05T20:08:00.001+01:00

Here are some observations and tips for buying software direct from Adobe.com and a warning.

Don’t expect to download your newly purchased software on the same day you bought it. Adobe are very slow to release your product’s serial number. Imagine someone working in a tiny room somewhere in India printing out orders, using a ladder to stack them, as his grandmother slowly keys in serial numbers into another computer. The pair have been doing it for years and like it that way; they think their customers must be patient and wait their turn.
For UK consumers it’s cheaper to order a boxed copy than to go for the download-only option. Go figure that one out!
The US price is much cheaper than the UK price, but that’s no surprise. It won’t let you buy from the US store unless you register a valid US postal address. (I’m not sure if it would reject a credit card registered in the UK, but you could try using PayPal.)
If you are a business you can sometimes get a better price by going to Adobe’s Business Store – even taking VAT into account. This store also has the download and boxed options at the same price.
Adobe’s website is slooooow. The product pages are slow, the store is slow and the licensing site is slow. I hope the public don’t think this is a reflection of ColdFusion. They may get loads of traffic but that’s no excuse. They may be the market leaders and pretty much have a captive audience for many types of product but that’s no excuse either to be complacent.
If you think you can side step Adobe and buy from a reseller then think again. Downloads and the release of serial numbers for certain products (especially upgrades) must go through Adobe.
It’s been this bad for as long as I can remember. You should plan to make your purchase as early as possible, don’t leave it to the last minute.

I’ve just bought Dreamweaver in case you’re wondering. Previously I bought ColdFusion several times.

Genius Tech That Blew My Mind This Week

2011-02-18T22:16:00.001Z

I stumbled upon two things this week that blew my mind. The first is a Xbox Kinect “hacker” who writes his own software to fully exploit Microsoft’s amazing hardware add-on. (Yes, I know it’s a few months old but you may not have seen it.) First check out this video. Keep in mind the Kinect is sitting stationary on his desk.

Genius video manipulation in 3D

A couple of months later he’s discovered how to link 2 Kinects together. What you will see is video from two stationary Kinect cameras. They do not move.

Super genius manipulation in 3D space with 2 Kinects

Taking this to the next level but with practical gaming in mind, look at what he did with a Kinect set up in two remote locations, a network connection between him and his colleague, a Doom 3 map and lots of clever code.

Augmented virtual reality taken to a new level across a network

Lastly, I was really impressed to see that a radio controlled hummingbird has been invented that uses nothing but wing-flapping to propel itself and behave remarkably like the real thing. It’s only 19 grams in weight (0.7oz) but can carry a camera to transmit live video back to its base.

The video of it flying in action is at NewScientist.com

Critical CF hotfix must be applied pronto!

2010-08-16T23:24:00.001+01:00

Adobe released a security hotfix on 10th August 2010 and classed it as “important”. However, if security is even moderately important to your ColdFusion server, the file system, database and network then you must think of this hotfix as CRITICAL. Just to clarify, this is CRITICAL. An HP security expert has blogged about it and It also caught the eye of The Register.

Hackers have proven how easy it is to use a vulnerability in CF 7, 8 and 9 to gain access to the CF Administrator. Code to perform the hack has been made freely available (which I won’t directly link to, but others have). Mike Bailey tweeted “It works and it’s scary.” Someone else chimed in and showed how you don’t even need to hack the Administrator’s password by using a cheeky bit of JavaScript.

Why is this really bad? Well, once you have access to CF Admin you can run scheduled tasks to access the OS. Someone has kindly(?) written an FAQ explaining how it works and why it’s so bad.

Now that the world knows how to hack it, everyone running CF must now patch their server. Adobe need to hammer home the seriousness of this problem and how critical the hotfix is. “Important” doesn’t stress it enough.

If you have already made the “administrator” directory inaccessible to the Internet or IP protected it then you should be safe, but it’s a good idea to still apply this critical hotfix.

SP1 Joy for Windows 7 and Server 2008 R2

2010-08-11T22:39:00.001+01:00

My main issue with Windows 7 and Server 2008 R2 was the removal of the otherwise simple feature of restoring open folders after a reboot. Currently Windows restores the folders in a random cascaded position forcing you to tidy up your desktop every time you boot up. It appears that through pressure from many users Microsoft have listened. Yay!

SP1, to be released in early 2011, will restore this much missed feature that was present in XP and Server 2003. From the documentation:

“SP1 changes the behavior of the ‘Restore previous folders at logon’ function available in the Folder Options Explorer dialog. Prior to SP1, previous folders would be restored in a cascaded position based on the location of the most recently active folder. That behavior changes in SP1 so that all folders are restored to their previous positions.”

Forget dynamic memory and RemoteFX, this is what I’m missing! :-)

ColdFusion Bugtracker Bug

2010-05-29T21:28:00.002+01:00

How does one submit a bug about a bug submitting tool? Below, the ColdFusion Bugtracker after completing a 2 page form.

I tried this twice, logging in again the second time just to make sure I was logged in. Booo. Now I have 2 bugs to report to Adobe!

Sessions never expire bug in ColdFusion 8.01

2010-05-27T00:35:00.001+01:00

A couple of times a year I’ve encountered a strange problem in our ColdFusion servers where sessions mount up and aren’t removed after they should have expired. Just today we had 100’s of sessions left in memory with all of the session scope variables still there at the end of the day, hours after they should have been deleted. Automated session housekeeping ceased to be.

Two other symptoms drew my attention to the above problem which must surely be related. Emails stopped being sent, the spool directory filled up without any cfmail files leaving. And from the website CF sporadically threw the error “The session is invalid” which was temporarily resolved by closing the browser and logging into the app again.

Restarting the CF services doesn’t resolve the situation because the service refuses to stop if asked politely. A full reboot is the only way to restore normality with confidence.

You’re probably thinking what good is it complaining now because 8.01 isn’t the current release and we should upgrade to 9.x. Well how do we know that 9.x has fixed this problem? Are Adobe aware of the issue? We could run a 30 day trial of 9.x on a test server but we’d have to run it for at least 6 months with a constant load to mimic our production server keeping in mind the rare appearance of the bug.

If you have encountered this problem before or know how to fix it then please let me know.

Windows 2008 Hyper-V / VSS / Backup Bug Part III

2010-05-25T17:38:00.001+01:00

Good news! We’ve been issued by Microsoft with a public release of the hotfix KB982210, as it will be known. The fix will only work on 2008 R2 and not with any previous releases of the OS.

So how does the fix work? First let me explain the problem more clearly than in previous blog entries. Whenever a device is attached to Windows the Plug & Play Manager creates an entry for that device in the registry. If it’s a USB device, for example, and you unplug it then its entry will remain in the registry so when it’s plugged back in the computer will recognise it and any settings that have previously been set up for it. The same is true for snapshots created by VSS, the Volume Snapshot Service. A snapshot is treated like a device so one a new snapshot is created so too is a registry entry for the device.

Now here’s the problem. The registry entries are not removed – ever. While many users will never have a problem with that there are a number of power users who generate 1000’s of snapshots over a short period of time. For example, in our case where we use Windows Server Backup (WSB) with a backup schedule set to every 30 minutes which includes backing up 14 VHDs used by several VMs on Hyper-V. VSS will create 14 snapshots (one for each VHD) every time a backup is run. That’s 14 snapshots every 30 minutes. That’s 672 a day and over 20,000 per month. See how quickly they mount up and none of the device entries in the registry are being removed.

Severe problems manifest when the host server is rebooted and the registry is processed, analysing tens of thousands of devices, causing the server to look as if it has hung. It freezes for 2 or 3 hours, possibly more if I had let the server carry on taking more backups.

The registry key you need to check in Windows 2008 R2 is:
HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Enum\STORAGE\Volume

There should be 10 to 50 entries in there for a normal healthy machine, depending on how many devices you have attached. On our server we had 28,000 entries!

Now, how does the fix work?

The new hotfix from Microsoft makes a change to the Plug & Play Manager that adds a timestamp (or a tombstone date as they prefer to call it) for each new snapshot device that’s created. This means Windows is now aware of exactly when a snapshot was created and can make a decision to fully remove the device’s entry from the registry after a certain period of time. I’m not sure how long it waits but from our experience it can be counted in minutes rather than days or weeks.

So well done to Microsoft for creating a smart solution to a critical problem. It took a massive amount of effort to get our case brought to the attention of the right person. Before that I had spent weeks working with a Microsoft support team in India by phone and remote desktop trying to explain what the problem was (and no, the problem will not be resolved by re-installing Windows, thank you!). It wasn’t until a Premiere Support case was opened and a Microsoft account manager from the UK got involved who contacted the right technical person that we started to make rapid progress. Microsoft had been aware of the technical issue for a while but our case seemed to have given them the incentive to fully investigate it. And we are grateful that we were finally listened to and some very expensive new servers can finally be put into service.

Cleaning up the registry

The only problem that remains is for other people experiencing this. You need to clean up your registry before applying the hotfix otherwise the freezing symptom will persist. To do this you need a tool from Microsoft called devnodeclean. Sadly this is not available anywhere on the Internet to download based on my Google and Bing searches. Microsoft should be able to email you a copy if you open a support case with them and refer them to KB982210 and this blog entry for good measure. Run devnodeclean without any switches at first to see what it makes of your registry, then use the /r switch to force it to remove the unwanted devices from the registry.

Next you may need to compact your registry if it has become huge, anything over 50MB I would say. Ours peaked at over 450MB. The “system” hive (found in C:\Windows\System32\config\) can be compacted using regchk using the switches /l /c /r /v. Again, chkreg is only available by request from Microsoft and was in fact developed to repair the registry in Windows 2000 but amazingly still works for 2008 R2. Please note that regchk cannot compact a live “system” file. You need to back up your system settings first and run chkreg on a restored copy of the “system” file (restore it to a new folder somewhere else). Then boot up from the Windows setup DVD and enter the recovery console. Rename “system” to “system.old” and copy the compacted system file into the config directory. Then reboot into Windows.

Nasty Bite From the Apple

2010-04-10T01:46:00.001+01:00

Apple have gone bonkers. They have changed their developers license to make sure that Flash can’t run on iphone, itouch and ipad devices. Do they really have it in for Adobe Flash or are they just trying to hide the possibility that they’re going to launch their own software in competition to Flash, or maybe their devices will run really slowly if they were to support Flash?

Considering that Flash is on 98% of all desktop computers and most of the best or most visited websites use Flash this is very strange behaviour exhibited by Apple.

But you know what, I don’t care! Why? Because I don’t own any Apple kit. I never have done and I’m not sure if I ever will. You see, Apple produce fashionable portable devices and I’m not one to follow fashion, especially if it looks far better than it actually performs as far as features are concerned. (I admit the UI is good but the rest of the world is catching up – seen Windows 7 Mobile?) And denying i(pod|pad|touch) users the ability to tap into Flash is denying them a fair bit of functionality. But do the users really care? Probably not since millions of people have already spoken with their wallets.

Windows 2008 Hyper-V or Volume Shadow Copy Bug Part II

2010-04-02T11:10:00.001+01:00

Following on from the problem I blogged about a couple of weeks ago, this is definitely a bug in Windows. From what we’ve been told it’s an inheritary, built-in limitation with VSS that prevents it from cleaning up after itself once 9,999 snapshots have been created. Therefore it only manifests if your server has created that many snapshots, so the explanation goes.

Now, not many people run snapshots frequently enough to encounter the problem but since our servers are running a backup every 30 minutes and there are 14 VHDs connected to Hyper-V we would run into the problem in just 15 days. That’s the explanation we were given but that doesn’t explain why after cleaning up the registry using devnodeclean and chkreg the registry will immediately start to bloat during the next backup. Hmmm. And we encountered symptoms (freezing) only after the first week – that’s less than 9999 snapshots. Hmm.

I’m hoping the engineers investigating this can come up with a permanent fix very soon.

2008 Server Freeze, Hyper-V or Volume Shadow Copy Bug?

2010-03-15T12:13:00.001Z

We have been scratching our heads over a very strange problem for the last 4 weeks which causes two new servers to lock up for up to 2 hours after logging on after a reboot. They’re running Windows 2008 R2 with Hyper-V and Windows Server Backup roles installed.

After trying plenty of ideas to eliminate the problem it was pointed out to us by a Microsoft support guy that our System hive file was 343MB in size. It’s only supposed to be 15 to 20MB. I exported it as an ASCII file from regedit and opened it in Notepad. I counted 24,000 entries for VSS Snapshot devices! When Windows boots it tries to process 24,000 devices which causes it to choke killing the server for two hours – although the VMs limp on underneath and the host responds to pings but both the remote and local console is completely frozen.

Example registry entry:

[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Class\{533C5B84-EC70-11D2-9505-00C04F79DE2F}\0349]
"InfPath"="volsnap.inf"
"InfSection"="volume_snapshot_install"
"InfSectionExt"=".NTAMD64"
"ProviderName"="Microsoft"
"DriverDateData"=hex:00,80,8c,a3,c5,94,c6,01
"DriverDate"="6-21-2006"
"DriverVersion"="6.1.7600.16385"
"MatchingDeviceId"="storage\\volumesnapshot"
"DriverDesc"="Generic volume shadow copy"

Trying to delete the snapshots using vssadmin from the command prompt threw this error: “Error: Snapshots were found, but they were outside of your allowed context. Try removing them with the backup application which created them.”

So the question is what is causing 1000’s of VSS (volume shadow copy) snapshots to be created? A clue was found in the system event log when Windows Server Backups runs: “Failed to delete the shadow copy (VSS snapshot) set with id '1A1938A0-1590-4BF4-8173-20DF5FD69E36' in the running virtual machine 'MGT01': Unspecified error (0x80004005). (Virtual machine ID A3F941F1-ED7F-48E9-9CD7-CB7C28A6604A)”

We’re using Windows Server Backup (WSB) to take incremental backups every 30 minutes for a bare metal restore of the host and its Virtual Machines. That’s 48 backups a day of 14 VHDs for 42 days that the servers have been running for. Do the maths and that comes to 28,000 VSS snapshots. Taking into account that some backups failed to run and we stopped backups for a few hours here and there, this tallies with the 24,000 devices I counted in the registry. Bingo!

So the bottom line is that the VSS writer creates a snapshot for each VHD at backup time but for some reason isn’t deleting the entries from the registry, although it is deleting the actual snapshots otherwise we’d have run out of disk space by now. Everything points to a bug in either the VSS writer or perhaps WSB or Hyper-V. They’re so tightly integrated during the backup process it’s hard to say which of the 3 is the culprit.

Since this problem is reoccurring on two new servers from Dell we are sure this isn’t a one-off freak incident. There is only 1 other similar incident reported on the web and that was a year ago on a HP server using BackupExec with the Hyper-V aware option. I’m waiting for Microsoft to get back to me, although I’ve been warned that even if they admit it’s a bug it could take a long time to produce a fix. We’d love to know why 1000’s of people who use Hyper-V and take frequent backups aren’t experiencing the same problem. There is no other software installed on the host apart from standard Dell drivers. Weird!

CF Admin SQL injection flaw

2009-11-06T02:18:00.001Z

Nathan Mische has blogged about a surprising ‘flaw’ in the ColdFusion Administrator that’s supposed to allow you to disable SQL commands coming from CF such as delete, drop, alter, update, etc. Apparently this is ineffective against multi-line SQL injection attacks – the most common sort of injection.

Perhaps ‘flaw’ is the wrong word, but these settings do lead you to believe that any sql with delete, drop, etc in it would be prevented from being sent to the database.

The bottom line is to always use cfqueryparam or stored procedures and sanitise user input from forms and urls. Don’t trust users – assume they’re all up to no good! ;-)

ColdFusion 9 is out. Upgrade? Not this time.

2009-10-06T12:30:00.001+01:00

After nearly a year of alpha and beta testing the latest release has hit the streets. It’s available to download as a 30 day trial or as a free developer edition (limited to 2 IP addresses as per usual). The new features are listed on the Adobe site.

The license has changed for the better which allows you to run the full version of CF on your development and test environments for free providing you’ve bought a new CF9 license for your production environment. Therefore if you were previously limiting your dev and test systems to 2 pesky IPs you can lift that restriction, but the new license only applies to CF9, it cannot be retrospectively applied if you own CF8 and earlier editions. Well done Adobe!

The question everyone is asking is shall I upgrade our existing servers to CF9? I made the decision a while ago to skip this edition. You have to ask yourself what business benefits it will bring to the table and if that will enhance your apps sufficiently to pay for itself. Here’s my take on the new features. Obviously it’s different for every app and every business, but needless to say our new apps being launched in the months to come will be using CF9, but our existing ones will not.

cfspreadsheet: Let’s you read from Excel files and update them too. Good if you work with xls files. I can see benefits for intranet apps. Previously to create xls files you had to create data as an HTML table but CF8 was unable to read or update existing Excel created files.
Word to PDF: I really like this conversion facility but it does not officially support Office 2007 or 2010 docx files which is somewhat of an oversight. However, it will make a reasonable attempt at converting them but chokes on the more elaborate docx files as explained in this blog. OpenOffice needs to be installed for this feature to work. (While OpenOffice is free I’m not comfortable installing a bloaty desktop app onto production servers)
ColdFusion as a service: Access CF features such as cfhart, cfdocument, cfimages, etc, as a web service. Perfect for offloading tasks to other servers or opening up CF features to .NET or PHP apps.
Adobe AIR database synchronisation: If you’re into AIR this sounds like a useful addition.
Virtual file system: Save files to RAM as if it were an ordinary hard drive. Good if you have a load of files that need to be regularly written to or read from or if your network drives are already a bottleneck for performance. Don’t forget all files in the virtual file system are wiped when the server is rebooted or crashes.
Integration with CF Builder: If you’ve tried the beta of Builder (the new IDE from Adobe) then it will work better if you have CF9. I don’t like Builder. Sorry. I wish they had enhanced Dreamweaver instead because I also need access to design tools as well as coding tools, plus DW has a very good GUI.
ORM: It sounds clever – CF will interface with your database without having to write a single line of SQL. Admittedly I haven’t got into this too deeply but I fail to see how it can write SQL as intelligently as a human or work with the many complexities of our stored procedures and data intricacies. I like to know exactly what’s happening at the CF<>SQL layer so ORM would worry me. Maybe ORM is for non-enterprise apps and RAD? I’m sure someone will enlighten me.
New AJAX controls: I have a real problem with this. Why are Adobe wasting their time adding a CF layer for JavaScript when the JavaScript library becomes outdated within a matter of months? If developers use a highly flexible JavaScript framework such as jQuery (which is really easy to learn and oh so sexy) then they can always keep their apps up to date with the latest features instead of having to wait a year or two for the next CF update (which also updates the built-in Ext JS library). jQuery is less bloaty and give developers much finer control over ajaxy things, ready-to-go menus, slick UI panels and interactive data tables etc.
SharePoint integration: This is fantastic if your business or client uses SharePoint. It makes it so much easier for CF apps to take part or become the hub of new SharePoint apps. My biggest client has made a massive move to SharePoint which would have got me excited if they hadn’t banished all non-Microsoft technology from their organisation. Doh.
Server Manager: Administrate multiple CF servers from a central console. Very useful if you have a large server farm or regularly tinker with the Administrator on a couple of servers. Definitely good for rolling out new CF servers. Otherwise check out Merlin, an AIR based administrator for managing multiple servers running CF 7, 8 or 9.
Enhanced Flash Remoting: Never used Flash Remoting or Flex, I am not a Flash developer as I find using DHTML/jQuery fast and effective for enhancing the user experience, but obviously this is welcomed by those who use Flash Remoting. (I wonder what percentage of CF developers do use it?)
Speed enhancements: Other blogs have run tests to show that CF9 is faster than CF8. So if your current server is under strain maybe you can buy it some breathing room by upgrading? But realistically, new server hardware may be cheaper than a CF upgrade and if your server is 3+ years old a new Intel 55xx based server could quadruple your server’s speed and/or capacity. (We’re going down the hardware upgrade route)
64 bit Edition for CF Standard: Yay!
Cache enhancements: The popular Ehcache technology is now integrated into CF. Rob Brooks-Bilson talks about this in detail over 4 blog entries so I’ll leave him to explain since he’s done such a fantastic job. Part 1. Part 2. Part 3. Part 4.

Finally, Adobe have produced a useful product matrix showing the differences between CF7, 8 and 9 for both Standard and Enterprise editions of each version.

Validity – flexible jQuery form validation

2009-09-27T12:36:00.001+01:00

I was looking for a form validation plugin for jQuery that offers enough flexibility to hook in to my own rather complex forms and existing JavaScript. After auditioning several plugins I came across Validity which has seriously impressed me.

Validity, by Wyatt Allen, offers the key following features:

Built-in validators for email, number, url, date, range, length, etc.
Equal(), distinct(), sum(), etc, methods for advance validation.
Use jQuery selectors to create rules for which fields to validate and compare.
Easily extendable using your own regular expressions or js functions.
Fully chainable validation. e.g. $("#lastname").require().minLength(2).maxLength(40).nonHtml();
Optionally provide your own validation error messages.
Multiple validation error display modes: Creates neat arrowed validation messages by each field with an error, or Modal mode, or Summary mode, or create your own! e.g. javascript text based alert, or outline error fields in red, or display errors at top of page, etc.
Automatically picks up the name of each form field or use a friendly name with the title attribute. e.g. <input type=”text” id=”firstname” title=”First Name”>
Validity can intercept the submit button or be called when you want it to (if you use ajax for example).
Only 9KB in size

Some example uses:

$("#title").require().minLength(25).maxLength(250);
The title field is required, min char length is 25 and max length is 250.

$("#duration").require().match("integer").range(1,400);
The duration field is required, must be an integer, and numbers from 1 to 400.

$("#email,#email_confirm").require().match('email').maxLength(70).equal("Email addresses do not match");
Applies to both the email and email confirmation fields, both required, must be a valid email address up to 70 chars, and must be equal to each other. My own error message will be shown if they are not equal.

$("#postcode").require().match(/^([a-z][a-z]?\d\d? ?\d[a-z][a-z])$/i,"Postcode is invalid");
The postcode field is required and must match my own regex for UK postcode validation and will display my own message if it’s invalid.

Here’s the documentation with a couple of little demos. The home page is here and you can download the plugin from Google Code.

Well done Wyatt, this is a brilliantly thought out plugin leaving ample room for developers to customise their own validators, messages, and display rendering. Perfect! :-)

Windows 7 Annoyances & Backups

2009-09-06T14:35:00.002+01:00

Up until now Windows 7 was just running on my laptop, a non-critical machine I use to run presentations on. Now that the final code is available to developers and my shiny new Intel SSD “G2” drive has arrived I thought it was a perfect opportunity to upgrade my main desktop PC to Windows 7.

It was a fresh install using the new SSD drive. It took 4 attempts before I had a stable install; the installer hung once, after one successful install the drive refused to boot up, the “repair Windows” option failed, yadda yadaa. 24 hours after it all appears to be working my fingers are still crossed.

Now, onto the issue of this post. When I shut down my PC at night I want to carry on working where I left off the following day – at least I want all the folders to be open up at the same place. Therefore, on XP I had both these options checked in Tools/Folder Options: “Restore previous folder windows at login” and “remember each folder’s view settings”. Windows XP obeyed this command like a faithful dog, but Windows 7 doesn’t play ball. On rebooting it opens all my previous windows on top of each other, in a pile. What a mess! I then spend a minute rearranging them all, dragging them to their correct position and resizing them to how they should be. What a pain!

Furthermore, Windows 7 forgets they layout options for each window. The option “remember each folder’s view settings” that was in XP is missing in 7! A couple of my windows list files that I frequently access so I don’t want to see the navigation pane, just the file list. I use another window for managing files so I always want the navigation pane displayed. XP obliges by 7 is so stupid it can only remember 1 folder setting and applies it to all of them, so on reboot the windows are either all with navigation panes or without, not the customised mixture that XP used to respect.

For those reasons I find Windows 7 very frustrating. Someone suggested using hibernate or suspend instead of shutting down. That may be okay if Windows doesn’t refuse to recover from hibernation, but it shows an error on resume and reboots.

Remembering folder views and locations is a very, very simple function for an OS but Windows 7 has taken a huge leap backwards here. It does not compute. Windows 7 has been given the memory of a goldfish.

Backups – Aargh!

I want to back up my C (boot) drive which is 80GB onto an external 200GB drive. No, it can’t do that because Windows has decided that the backup has to include my D drive which is a massive 1TB internal drive. What? I can’t back up my boot drive because Windows insists that the D drive is also included? How stupid is that. I do not want to back up my entire 1TB “D” drive, just the critical C drive.

Admittedly I have installed non-essential apps onto the D drive such as Firefox, Visio and Dreamweaver, plus I changed the location of My Documents from the C drive to the D drives to save disk space on the precious 80GB SSD. But those files aren’t critical to restoring Windows should the need arrise.

I’m now testing out other backup options. My key requirements are:

Volume Shadow Copy – so open files can be backed up like Outlook’s .pst files
Back up specific files from any of my drives without having to backup the entire drive
Option to back up files into zip files so the backups are completely independent and not proprietary files formats.
Additionally create an image backup of a drive (to quickly restore my boot drive onto another drive should it die)
Incremental backups so only changes are backed up on a daily basis
Windows 7 bit-bit compatible

Cobian Backup has always been perfect for file backups (it’s a brilliant free app) but sadly it’s only a 32 bit app which means it can’t do volume shadow copies on a 64 bit OS – which is what I’m now running to utilise the full 4 GB of ram in my PC.

So I will now evaluate the following free or cheap alternatives:

Acronis True Image Home 2009 (commercial, £40/$60, or half price here)
GFI Backup Home Edition (free)
Macrium Reflect Free Edition (free)
Macrium Reflect (commercial, £20/$30)

Acronis is not yet fully Windows 7 compatible which is a shame as it has some very powerful features. GFI Backup sounds great as a file-only backup solution (it doesn’t do drive imaging, same as Cobian Backup). Macrium Reflect (commercial) does both file and disk imaging but backs up to its own proprietary file format.

I think I may use two apps for a separate disk imaging and file backup strategy.

8x ColdFusion Hot Fixes in just 2 weeks!

2009-09-01T11:45:00.001+01:00

First there were 7 hot fixes rolled out on the same day on 17th August – all to do with security vulnerabilities. Adobe’s documentation was sparse causing problems and lots of questions to be raised by early adaptors of the hot fixes.

Today, 1st Sep, Cumulative Hot Fix 3 has been released for CF 8.01 that fixes 21 new bugs as well as the fixes contained in the previous 2 cumulative hot fixes. This does not include fixes for the aforementioned 7 security vulnerabilities.

I can’t say I’ve encountered any of the bugs listed for Hot Fix 3 and they’re not security related so we’ll put some thought into whether to deploy it or not as I’m a great believer in “if it ain’t broke, don’t fix it”. Do read through the list of fixes to see if any issues are relevant to you. Maybe some of the descriptions will explain some strange errors you’ve encountered with your apps?

I notice there’s no fix for the mail spool bug I reported over a year ago.

Now, back to the 7 security hot fixes. The original documentation was, let’s face it, terrible. But on 28th Aug Adobe updated the text to make it clear that hot fix 1876 must only be applied if you’re running Apache. Do not apply it if you’re running IIS which is what I did on a test box. Luckily I held back on applying it to the prd servers before receiving confirmation that it’s not for IIS. The test box happily accepted the hot fix anyway and doesn’t seem any the worse for it.

Hot fix 1875 and 1878 are byte for byte absolutely identical which is really weird. Why didn’t Adobe roll them into the same hot fix instead of listing them separately and making people install two hot fix files which are the same in all but file name?

Universal Phone Chargers for Europe

2009-06-30T02:15:00.001+01:00

It’s not often that something the EU (European Union) does is so clearly a step in the right direction. So what did they do? They got the major phone manufacturers to agree to a single type of phone charger and this will be implemented in every phone by 1st January 2010. Whoo hoo!

They went with the micro USB connector which is already a fairly well used by many phones. Being USB it can carry data as well as power, so connecting your new phone to your computer won’t require a new cable either.

The EU wants to cut down on the number of mains chargers that are made and then chucked away. But since every new phone comes with a mains charger how will that change anything? It’s good for the consumer as we’d just need one mains charger plugged in to recharge any phone in the house. About frigging time.

Now how about standardising electrical plugs and sockets next? I propose the British system is adopted by Europe because it’s the safest with a built-in fuse, built-in earth, and the socket’s pin holes are closed so little fingers or children poking things into the holes can’t reach the live connection. The holes are automatically opened when a plug is pushed in; the earth pin is slightly longer, it mechanically opens the live and neutral pin holes. Sometimes when I buy new electrical items it comes with 2 different cables (for Europe/UK or US/UK) and sadly one has to be thrown away. If cables were made from liquorish I wouldn’t complain.

Windows Live Messenger makes me cry

2009-05-22T02:05:00.001+01:00

For years I’ve had MSN Messenger installed so I can chat with friends and family. Recently Microsoft release a new version, Windows Live Messenger 2009. (Does it hurt them too much to think of a short name I can spit out in just a couple of syllables?)

I installed it and it crashed every single time I tried to log in. There was no facility to report the bug yet alone take out my frustration with some anonymous support guy.

6 or 8 weeks had passed and I thought surely they have fixed the bug by now. I can’t be the only user out of a million who fell foul of the 2009 release. So I installed it again from their “Essentials” online installer and BAM! It crashed again. Aaargh!

My attempt to defy Microsoft by uninstalling Messenger was blocked by the small fact that I couldn’t find an uninstall option anywhere on my computer. Nothing in the Start menu and nothing in Add/Remove Programs. Oh how I despise thee. For art thee friend or foe? I really do wonder sometimes.

Windows 7 RC downgraded my graphics

2009-05-07T23:41:00.001+01:00

After battling through an incompatibility problem with AVG after upgrading from Vista (spit!), I was very pleased with Windows 7. It’s performance, looks and usability have all been improved.

Windows 7 rated my graphics card (ATI X1700 built into the laptop) as 4.4 for “Graphics” which is pretty good. Then I was alerted to some updates including one for ATI graphics which was unexpected. After the update and a reboot I re-ran the performance test and was astonished that it downgraded my respectable 4.4 rating to a measly 2.1. I ran it again to check and it was the same.

The “Gaming Graphics” rating remained at 3.3. So Windows 7 after the update reckons my card is more capable of 3D gaming that it is of moving 2D windows around the screen.

The most annoying thing is, Microsoft removed the reporting tool from the release candidate so I can’t inform Microsoft about this problem. Like everyone else I just have to blog about RC problems instead.

Twitter drives me cuckoo

2009-05-03T11:32:00.001+01:00

I had deliberately avoided using Twitter because I couldn’t understand why anyone would want to micro-blog every mundane thought or action, and even more so why would anyone want to be bombarded by other people’s mundane thoughts?

You’ve got to get your hands dirty to fully understand some things in life so I rolled up my sleeves and set up a Twitter account and installed what looks like one of the best Twitter desktop clients, TweetDeck, which uses Adobe AIR.

I became a follower of about 15 people, quite a lot I thought at the time, until I saw that others were following 100’s of people. Only 3 of my 12 regularly tweeted, sometimes several tweets each per hour. Where do these people get the time from to stop what they’re doing and compose a 140 character thought to share with others? Perhaps they were unemployed or had a very dull office job. Not at all, they’re very busy people working in the IT world, yet they have developed a mental condition that I couldn’t fathom which had become a compulsion to tweet about what they’re doing right now, what they think about something, or where they’re about to go. (Guys, if you’re reading this then I apologise, but at least your flames will be restricted to 140 characters!!! Ha-ha-ha! ;-)

TweetDeck interrupted my train of thought throughout the day alerting me to new tweets. About 1 a day turned out to be interesting, pointing to a URL containing something useful to my job or interesting in my capacity as a concerned citizen.

Celebrities clearly enjoy Twitter as it gives them a platform to perform and to be adored for every waking hour in their life. And fans who live and breath their idol can relish in their activities and musings all day and every day. Stephen Fry is one of the most followed twitterers with nearly half a million people clinging to his occasionally amusing tweets. There was that incident a few months ago where he was stuck in a lift (elevator) and twittered about it from his mobile phone and drew some optimism from his followers who replied to give him support during that very long 30 minute experience, stuck in a confined space with a load of strangers who no doubt started to smell after a short while.

My conclusion is pretty much what I expected before trying Twitter. If you didn’t have a compulsive disorder before using it then you’ll probably develop one after a week. It’s like drilling holes in your life, you’ll leak valuable time during the course of the day achieving very little by twittering or reading every tweet flashed up at you.

I reckon Twitter will be a phase, a fashion lasting a couple of years, or it will mature into something more sensible that can be better controlled or filtered or used in applications for a genuine practical purpose. Speaking of which, there’s a ColdFusion Twitter library that looks very good which gave me an idea of using Twitter in a constructive way for an existing application.

3 CFML Engines, a hard choice to choose

2009-04-16T23:19:00.001+01:00

With 3 great CFML engines to choose from, the choice for developers can be a painful one. Torn between Adobe ColdFusion, Railo 3.1 and Open BlueDragon? Yeah, me too.

The core CFML tags and functions are well supported by all 3, with mostly the bleeding edge features that separate them along with the support options if you’re concerned with needing expert help if you find a bug or quirk that becomes a showstopper.

The other consideration is the platform you have to host the CFML engine. Adobe ColdFusion is well supported by hosting companies if you’re not hosting it yourself, but the others have limited options at the moment.

UPDATE: I just read that OpenBD can run on the Google Apps Engine! Live demo! Look at the appserver value. (It’s not available on there yet… work in progress…)

Most of my projects are self-hosted so I don’t care about the platform, I just need stability, easy-to-get support (free or paid-for), and a promising development roadmap.

I’m not quite finished… each CFML engine have their own unique features. Take Railo with its CFVIDEO tag for example, for someone that could be a deciding factor. Creating YouTube sites could be as easy as pie. Open BlueDragon works natively with Amazon’s SimpleDB and can pull & push files to/from Amazon S3. ColdFusion 8 has built-in MS Exchange support (which is cool) and AJAX stuff which I don’t really care for. jQuery is the way to go. :-) But ColdFusion 9 is scheduled to be released by the end of the year and will arguably leapfrog the other two with a ton of new features.

I haven’t been too happy with the support from Adobe when I discover bugs and their 2-year product cycle means some issues don’t get addressed for a long time, although their intermediate patches are warmly welcomed.

A price to pay…

Then there’s the pricing issue. Sure, £6000/$7500 isn’t a massive price for big companies wanting to run with CF Enterprise, but small companies or personally financed start-ups can’t chuck that sort of money into application software when it costs the same amount to purchase new server hardware and host it for 4 years. Even on the enterprise level if you need to expand a cluster and whack in an extra web server it’s £1000/$1500 for the hardware and then £6000/$7500 on top. A large organisation would still ask why it’s costing that much to pop in an extra server.

The current economic climate has put pressure on many businesses to cancel or scale back on projects and the awful £/$ exchange rate has made CF a lot more expensive in the UK. So maybe now is the time to look closely the open source CFML engines.

Optimal SQL to page through large record set

2009-04-15T23:50:00.006+01:00

This is a tip for web application developers who want to page through results coming back from a MS SQL database. If there are 100 or 500 records coming back and you just want to show 25 at a time to the user then you can achieve that through your application code (CFML, PHP, etc). But what if there are 1000’s of records, it’s deeply inefficient to request that many records from your database if you only want to display 25 every time the user requests the next page, or jumps forwards several pages. If you want the results to be ordered by a user-defined column and not by the Row ID then the solution may not be obvious.

MySQL can use limit(start,count) but MS SQL doesn’t support that. Someone suggested using a memory table, someone else suggested a cursor. A bit of Googling revealed MS SQL 2005 onwards has a new function called row_number(). Here’s how it works in this example:


SELECT * FROM (
 SELECT row_number() over (
  order by users.surname, users.firstname
 ) AS rowNumber, users.id, users.firstname, users.surname
 FROM users
) table1
WHERE table1.rowNumber BETWEEN 150 AND 175

It will only return records 150 to 175 and importantly it will order it by surname then firstname. And it’s very, very fast.

AJAXbouncer – limiting Ajax tampering and leeching

2009-04-13T18:07:00.001+01:00

Yesterday I mentioned a proof of concept to try to stop script kiddies and data leechers from abusing server-side scripts that are intended to serve XMLHttpRequests (XHR or AJAX). Playing with URL or form parameters can get the server to return all sorts of data, sometimes even data that the developers didn’t intend you to have access to. The problem is that servers can’t tell the difference between a normal web page request and XHR.

Ray Camden blogged about how jQuery adds an extra HTTP header to help the server tell the difference, but headers are very easy to spoof.

My idea is for the server to issue the web page with an encrypted token. The token is the current date/time and must be sent back to the server for each and every XHR triggered by the current page. If the server doesn’t receive the token, or the token is invalid (i.e. it’s be tampered with) or the decrypted token reveals it’s older than, say, 5 minutes then the server returns a 404 error – page not found.

So, anyone who tries to submit data back to the server through dishonest means will find they get a 404 after 5 minutes. If they try to alter the token they get a 404 too. This will baffle script kiddies or hackers and hopefully they will move on to mess with someone else’s website. If they persist they will realise that they can’t generate their own encrypted token but will have to refresh the main web page every 5 minutes to obtain a new token and insert that into their script. That’s the only weakness in this concept, but taking it a step further you could log the IP from the first failed XHR and block serving that IP for the next 30 minutes. Or refuse to issue a new token within the same session or to the same IP.

As for genuine users you can set the web page to auto-fresh every 5 minutes. It will work best on sites where you don’t expect users to linger on the same page for too long, but of course you may prefer a longer token life (like 15 minutes).

Here’s a live demo – many thanks to Ray Camden for hosting it. The demo’s token will expire after just 90 seconds. The POST data is exposed in a grey area at the bottom of the page so you can tamper with the parameters to see what happens. The demo uses jQuery for XHR, of course.

I’ve commented the code so developers using PHP, .NET, RoR, etc can easily adapt the ColdFusion code. Download the demo code here.

If you improve upon it please let me know.

Oh, in case you’re wondering why I called it AJAXbouncer it’s because it offers a deterrent to potential trouble makers but doesn’t provide 100% safety – like bouncers standing outside pubs and clubs.

CFMAIL spool bug in ColdFusion 8.01

2009-04-12T22:55:00.001+01:00

After applying a patch to fix a cfmail problem in 8.01 I noticed a new problem. When creating more than 3000 or so emails in one go about 100 ended up in the Undelivr directory. There was nothing wrong with those emails, dropping them into the spool directory by hand sent them quickly on their way.

The exception.log was showing “IOException while sending message”. I reported this to an Adobe engineer in July 2008. In October the engineer found that the issue can be “fixed” by increasing the max session size in the Microsoft SMTP service. Microsoft don’t recommend increasing it too much but the more I increased it the less emails were sent to the Undelivr directory.

The bug is with ColdFusion and it appears to not behave correctly if the SMTP server closes a session because it has reached the maximum size. ColdFusion should react by simply opening up a new session and continue to send emails from where it last left off. But it doesn’t do that. This bug has been affecting a major application for nearly a year and Adobe haven’t been able to fix it.

MS SMTP logs “552 4.3.1 Session size exceeds fixed maximum session size” which is fine, it expects the app at the other end (CF) to continue by opening a new session. I’ve now increased the session limit to a very large amount but as the application is used more the limit is reached and emails stop being sent again. CF is misbehaving and is causing operational headaches.

Has anyone else had similar problems? I’m looking for others who can help add some pressure to get this bug fixed. Please do get in touch. Needless to say I’m very disappointed that CF is unable to send a large amount of (legitimate, non-marketing) emails problems.

AJAX tampering and leeching

2009-04-12T21:35:00.001+01:00

Ray Camden blogged about server side security when using AJAX. He discussed a way to detect the difference between normal HTTP requests for normal web pages and AJAX triggered HTTP requests. To the server they look virtually the same.

The problem with using AJAX is that is adds a vulnerability to web applications. People can play around with URL and form parameters to see what else they can extract from your application or database. It can also be used to leech data from your database on a regular basis because, if your data is of any value to someone else, you’re providing it freely in a machine readable format, typically JSON or XML. How can you do something to stop people do that? I mentioned a possible technique when commenting to Ray’s blog and was asked to come up with some code.

I brewed up a demo app and am seeing if I can get the live code hosted (it’s for ColdFusion but the technique applies to any server side language). Stay tuned…

If you want to play with the code on your own CF server you can download it here. No configuration is necessary, just drop the folder into your web root. The code is highly commented so PHP, RoR and .NET coders can easily modify it.

First post is always the toughest

2009-04-12T20:33:00.001+01:00

Finally I have a permanent blog after being on the Internet since 1991 and making my first web page in 1994. Finding the best blogging site and tools was a bit of an adventure since there are so many options and not everything does what you want or expect it to.

I’ve settled for Windows Live Writer for composing (I prefer a WYSIWYG desktop app) and Blogger.com for hosting. All I need now is a reliable way to post code snippets in a nicely formatted way – it needs to support CFML as well as JavaScript and HTML. Any ideas?