Having a little fun with the performance monitoring code from a previous post. Okay I have a warped idea of fun.
I’m always interested in performance, so I decided to analyze the overhead of using an error trap (On Error statement) to handle edge cases as opposed to an “if” statement or other branch. The error trap is often easier to code because it can be a catch-all for any unanticipated problem, but handling an error condition involves the LotusScript runtime coming up with an error message, determining whether the script has defined special handling for that error, etcetera. So it can be expected to take longer than a simple test and branch. My question was, how much longer?
To test this, I created two subroutines to try the two different approaches:
Sub ifassign(valu As Variant) Dim tmp If IsObject(valu) Then Set tmp = valu Else tmp = valu End Sub Sub oopsassign(valu As Variant) Dim tmp On Error GoTo oops Set tmp = valu Exit Sub oops: tmp = valu Exit sub End Sub
These do the same thing — they each copy the supplied argument to a temporary variable. But to do an assignment in LotusScript, you have to know whether the value is an object. Assignment of an object requires a Set statement (Set a = b). Assignment of a non-object requires a Let statement (the Let keyword is generally omitted so it’s just a = b).
Use of the wrong assignment statement causes an error condition. So if we want to write a flexible assignment that doesn’t care whether the argument is an object or not, we can either (option 1) test first whether it is, or (option 2) try it one way and see whether it throws an error, and if so do it the other way. These are the options represented by the above two subroutines.
To test the performance of these, I wrote some code to use PerfTimer class to time the results of 1000 calls. The code:
Dim i%, pt As New perfTimer(10) pt.start "ifassign scalar" Do For i = 0 To 1000 ifassign i Next Loop Until pt.isdone pt.start "ifassign object" Do For i = 0 To 1000 ifassign Nothing Next Loop Until pt.isdone pt.start "oopsassign scalar" Do For i = 0 To 1000 oopsassign i Next Loop Until pt.isdone pt.start "oopsassign object" Do For i = 0 To 1000 oopsassign Nothing Next Loop Until pt.isdone MsgBox pt.results
0.000282 - ifassign scalar 0.000289 - ifassign object 0.005724 - oopsassign scalar 0.000261 - oopsassign object
The times are in seconds, but since I do 1000 iterations between each call to pt.isDone, these times represent milliseconds per iteration. Each testing scenario ran for 10 seconds, which for in-memory operations is a long time, but I wanted to make sure it ran long enough for the signal to overwhelm the noise of variable CPU resources assigned to the process.
List access versus error handling
Testing isObject is pretty fast — what about something that takes longer? Searching a List variable for a particular key is a lot slower than testing a scalar or accessing an array.
In this case I’m comparing two different approaches to reading a value from a list:
Class ColorIndex docsByColor List As NotesDocument Function getDocOfColor(ByVal color$) As NotesDocument If IsElement(docsByColor(color)) Then Set getDocOfColor = docsByColor(color) End If End Function Function getDocOfColor2(ByVal color$) As NotesDocument On Error Resume Next Set getDocOfColor2 = docsByColor(color) End Function End Class
The two methods in this class return identical results. The difference is in how they determine whether the requested key value was found — one tests with isElement, the other just tries it and traps the resulting error in case of failure.
The code that calls these functions to test their performance created a list of 1200 elements, then made 1200 calls for keys that did exist using each function, and 1200 calls for keys that did not exist, ditto. The results were:
0.00187899 - iselement succeed 0.00127097 - errortrap succeed 0.00169005 - iselement fail 0.00854701 - errortrap fail
The list takes longer to search if it contains more values — isElement on a 10,000 element list seems to take about .002ms, whereas from the above results we can see the isElement call added .0006s/1200 = .0005ms (comparing lines 1 and 2).
Analysis of results
It takes very little time for ifassign to test whether the argument is an object — comparing “ifassign object” to “oopsassign object” which does the same assignment without testing first, the difference is 0.000028ms.
On the other hand, oopsassign’s error trap takes an additional 0.0054ms to process the error condition caused by its use of Set to assign a non-object, compared to any of the other scenarios. It takes about 200 times longer to process the error condition, than to test for the situation that would cause the error. You would need 99.5% of calls to have an Object as argument, to make error trapping a better option performance-wise.
Other situations may take longer to test to prevent an error, so the tradeoff will vary. As we saw above with lists, the cost of an error even when we just tell it to ignore all errors is several times slower than any other scenario — on the other hand, it doesn’t take longer to trap an error with an immense list than with a smaller list. So if you expect a high rate of invalid keys or you have huge lists, the error trap could still make sense.
Or you may be less certain you’ve covered all the cases that might cause an error. If your main concern is reliability rather than performance, it may be better to use On Error just to be sure you don’t allow an unexpected error to slip through.
But if you do need to squeak out every ounce of performance, bear this result in mind.
Also worth noting that if the “if” statement has two branches, LotusScript will currently execute both regardless of whether the first is successful, so in scenarios where you need to check multiple things it could skew performance. Visual Basic added OrElse and AndAlso to avoid that.
For VoltScript we’ve added Try/Catch/Finally and are re-purposing Return, which will avoid let/set for function return values. Although it re-purposes normal error handling, I found TCF is sometimes quicker than On Error for scalars, but it seems random.
Re: Both branches are executed
Huh? So if I have two branches one simple, the other huge, the simple branch will be slow? That seems really weird to me. Plus I looked up the OrElse and AndAlso operators, and they pertain to the condition expression right after the If, not the code branches…
So as I understand it, if you build a complex expression, all parts are executed, but the code branches are only executed depending on the outcome of the result (three outcomes: False, True or an error)
Correct both parts of the if condition get validated. We know If and Else aren’t executed, because otherwise our error management would be much more complex.