Overhead of error trapping in LotusScript

Having a little fun with the performance monitoring code from a previous post. Okay I have a warped idea of fun.

I’m always interested in performance, so I decided to analyze the overhead of using an error trap (On Error statement) to handle edge cases as opposed to an “if” statement or other branch. The error trap is often easier to code because it can be a catch-all for any unanticipated problem, but handling an error condition involves the LotusScript runtime coming up with an error message, determining whether the script has defined special handling for that error, etcetera. So it can be expected to take longer than a simple test and branch. My question was, how much longer?

Testing method

To test this, I created two subroutines to try the two different approaches:

Sub ifassign(valu As Variant)
	Dim tmp
	If IsObject(valu) Then Set tmp = valu Else tmp = valu
End Sub

Sub oopsassign(valu As Variant)
	Dim tmp
	On Error GoTo oops
	Set tmp = valu
	Exit Sub
oops:
	tmp = valu
	Exit sub
End Sub

These do the same thing — they each copy the supplied argument to a temporary variable. But to do an assignment in LotusScript, you have to know whether the value is an object. Assignment of an object requires a Set statement (Set a = b). Assignment of a non-object requires a Let statement (the Let keyword is generally omitted so it’s just a = b).

Use of the wrong assignment statement causes an error condition. So if we want to write a flexible assignment that doesn’t care whether the argument is an object or not, we can either (option 1) test first whether it is, or (option 2) try it one way and see whether it throws an error, and if so do it the other way. These are the options represented by the above two subroutines.

To test the performance of these, I wrote some code to use PerfTimer class to time the results of 1000 calls. The code:

Dim i%, pt As New perfTimer(10)
	
pt.start "ifassign scalar"
Do
	For i = 0 To 1000
		ifassign i
	Next
Loop Until pt.isdone
pt.start "ifassign object"
Do
	For i = 0 To 1000
		ifassign Nothing
	Next
Loop Until pt.isdone
pt.start "oopsassign scalar"
Do
	For i = 0 To 1000
		oopsassign i
	Next
Loop Until pt.isdone
pt.start "oopsassign object"
Do
	For i = 0 To 1000
		oopsassign Nothing
	Next
Loop Until pt.isdone
MsgBox pt.results

Test output

0.000282 - ifassign scalar
0.000289 - ifassign object
0.005724 - oopsassign scalar
0.000261 - oopsassign object

The times are in seconds, but since I do 1000 iterations between each call to pt.isDone, these times represent milliseconds per iteration. Each testing scenario ran for 10 seconds, which for in-memory operations is a long time, but I wanted to make sure it ran long enough for the signal to overwhelm the noise of variable CPU resources assigned to the process.

List access versus error handling

Testing isObject is pretty fast — what about something that takes longer? Searching a List variable for a particular key is a lot slower than testing a scalar or accessing an array.

In this case I’m comparing two different approaches to reading a value from a list:

Class ColorIndex
	docsByColor List As NotesDocument
	Function getDocOfColor(ByVal color$) As NotesDocument
		If IsElement(docsByColor(color)) Then
			Set getDocOfColor = docsByColor(color)
		End If
	End Function
	
	Function getDocOfColor2(ByVal color$) As NotesDocument
		On Error Resume Next
		Set getDocOfColor2 = docsByColor(color)
	End Function
End Class

The two methods in this class return identical results. The difference is in how they determine whether the requested key value was found — one tests with isElement, the other just tries it and traps the resulting error in case of failure.

The code that calls these functions to test their performance created a list of 1200 elements, then made 1200 calls for keys that did exist using each function, and 1200 calls for keys that did not exist, ditto. The results were:

0.00187899 - iselement succeed
0.00127097 - errortrap succeed
0.00169005 - iselement fail
0.00854701 - errortrap fail

The list takes longer to search if it contains more values — isElement on a 10,000 element list seems to take about .002ms, whereas from the above results we can see the isElement call added .0006s/1200 = .0005ms (comparing lines 1 and 2).

Analysis of results

It takes very little time for ifassign to test whether the argument is an object — comparing “ifassign object” to “oopsassign object” which does the same assignment without testing first, the difference is 0.000028ms.

On the other hand, oopsassign’s error trap takes an additional 0.0054ms to process the error condition caused by its use of Set to assign a non-object, compared to any of the other scenarios. It takes about 200 times longer to process the error condition, than to test for the situation that would cause the error. You would need 99.5% of calls to have an Object as argument, to make error trapping a better option performance-wise.

Other situations may take longer to test to prevent an error, so the tradeoff will vary. As we saw above with lists, the cost of an error even when we just tell it to ignore all errors is several times slower than any other scenario — on the other hand, it doesn’t take longer to trap an error with an immense list than with a smaller list. So if you expect a high rate of invalid keys or you have huge lists, the error trap could still make sense.

Or you may be less certain you’ve covered all the cases that might cause an error. If your main concern is reliability rather than performance, it may be better to use On Error just to be sure you don’t allow an unexpected error to slip through.

But if you do need to squeak out every ounce of performance, bear this result in mind.