Wonderful List datatype in LotusScript

The general term for it is “associative array” — a collection of values indexed by a string rather than a numeric index. You might be thinking you already know about the List datatype in LotusScript, but there are a few tricks you might not have thought of.

Basics

Declare a variable of type List As datatype — for instance,

Dim docsByColor List As NotesDocument

Read and set values as you would an array, but using a string as your index value.

Set docsByColor(doc.color(0)) = doc

In this example we’re building an in-memory collection of documents that we can search by the value of their color field, so:

Function getDocOfColor(ByVal color$) As NotesDocument
	If IsElement(docsByColor(color)) Then
		Set getDocOfColor = docsByColor(color)
	End If
End Function

Attempting to access an element with a key that’s not found in the list causes an error condition — you can either use On Error to trap the error, or use isElement as shown here to make sure the key is present before trying to access the associated value.

Note: my testing shows a call to isElement following by accessing the same element, is much faster than a call to isElement following by accessing some other element. LotusScript is doing something behind the scenes to make this common case execute faster, e.g. remembering the last-requested key and corresponding location to avoid repeating the search.

To iterate through the list elements, use the Forall statement:

Forall aDoc In docsByColor
	Print "tag=" & Listtag(aDoc) & ": " & aDoc.NoteID
End Forall

Forall returns in the order their keys were originally inserted. If you assign an element using the same key value, the new value goes into the old position for that key, not at the end.

Case and pitch sensitivity

The case sensitivity of searching for a list entry is established by the Option Compare statement in the LotusScript header — by default, it is case sensitive, so you can have list values whose keys differ only by their case.

If that’s what not what you want, you could use Option Compare Nocase, but that affects all the comparisons in the whole script module, which can have unexpected results. It’s usually better to convert all the key values to a standard case as you add and search for values. This is less prone to coding errors if the list is protected from direct access by callers through encapsulation. Put it into a class where there’s a limited amount of code that accesses it directly.

Class ColorIndex
	Private docsByColor List As NotesDocument
	
	Sub AddDoc(doc As NotesDocument)
		Set docsByColor(LCase(doc.color(0))) = doc
	End Sub

	Function getDocOfColor(ByVal color$) As NotesDocument
		color = LCase(color)
		If IsElement(docsByColor(color)) Then
			Set getDocOfColor = docsByColor(color)
		End If
	End Function
End Class

In the getDocOfColor method, the use of ByVal lets us modify the input argument (force it to lowercase) without having to worry about changing the value of a variable in the calling program.

If you need a list with pitch-insensitive keys, though, this approach doesn’t work, because there’s no pitch equivalent to the LCase and UCase functions. You really do have to use Option Compare NoPitch. To avoid this affecting other string comparisons in the same agent or library, create a separate script library that has just the “pitch insensitive list” datatype in it (and any other string operations where using the flags argument isn’t an option).

Option Public
Option Declare
Option Compare NoPitch

%REM
	Class PI_List
	Description: A wrapper for the List datatype that supports pitch-insensitive key values. 
	Constructor: New PI_List(ByVal caseflag As Boolean)
		caseflag: True if you want the keys to be case insensitive also.
%END REM
Class PI_List
	Private z_data List As Variant
	Private z_case As Boolean
	
	Sub New(ByVal caseInsensitive As Boolean)
		z_case = caseInsensitive
	End Sub
	Sub Set(ByVal key$, value)
		If z_case Then key = LCase(key)
		If IsObject(value) Then Set z_data(key) = value Else z_data(key) = value
	End Sub
	
	Function IsElement(ByVal key$) As Boolean
		me.IsElement = iselement(z_data(key))
	End Function
	
	Function Get(ByVal key$) As Variant
		If z_case Then key = LCase(key)
		me.Get = z_data(key)
	End Function
End Class

Lists of collections

Sometimes you have more than one item that matches a list key, and you want to keep all those values and be able to find them with that same key.

In earlier posts I talked about how we can keep lists of items. Since the data associated with a List can be any type, you can create a list of, say, LinkedList or Queue objects, or arrays, depending how you need to be able to access the contents.

It’s more difficult to manage the contents since you can no longer simply assign a list element without worrying about what’s already in there for that key. Once again, for greater reliability and code clarity, you may want to encapsulate this data structure to avoid random code poking its fingers in and messing it up.

%REM
	Class ListOfList
	By Andre Guirard
	Description: Similar to the List As Variant datatype except each key can be associated with a list of values
		instead of just one.
	Constructor: New ListOfList(defaultval)
		defaultval is the value returned by Get when there are no more values matching the key.
%END REM
Class ListOfList
	z_data List As LinkedList
	z_defaultval As Variant
	z_keyCount As Long
	
	Sub Put(ByVal key$, value)
		Dim el As LinkedList
		If IsElement(z_data(key)) Then
			Set el = z_data(key)
		Else
			Set el = New LinkedList(z_defaultval)
			Set z_data(key) = el
			z_keyCount = z_keyCount + 1
		End If
		el.Append value
	End Sub
	
	Function IsElement(ByVal key$) As Boolean
		me.IsElement = IsElement(z_data(key))
	End Function
	
	%REM
		Function getKeys
		Description: Returns a list of all the key values.
		Returns: array of strings (limited to 32,768 values, else error)
	%END REM
	Function getKeys
		Dim result$, cElems&, bHardWay As Boolean, tmp$
		Const DELIM = "▧"
		ForAll lis In z_data
			tmp = ListTag(lis)
			If InStr(tmp, DELIM) Then bHardWay = true
			result = result & "▧" & ListTag(lis)
			cElems = cElems + 1
		End ForAll
		If cElems > 32768 Then
			Error 20060, "Too many keys"
		ElseIf bHardWay Then
			ReDim arr(0 To cElems-1) As String
			cElems = 0
			ForAll lis In z_data
				arr(cElems) = ListTag(lis)
				cElems = cElems + 1
			End ForAll
			getKeys = arr
		Else
			getKeys = Split(Mid$(result, 2), "▧")
		End If
	End Function
	
	%REM
		Function getList
		Description: retrieve the list of values for a given key.
		Arguments: key: key value to retrieve
		Returns: Nothing if key not found, else the list of matches.
	%END REM
	Function getList(ByVal key$) As LinkedList
		If IsElement(z_data(key)) Then
			Set getList = z_data(key)
		End If
	End Function

	Private Sub assign(vari, valu)
		If IsObject(valu) Then Set vari = valu Else vari = valu
	End Sub

	%REM
		Function Get
		Description: Retrieve the next value associated with a given key.
		Arguments:
			key: the key value
		Returns: first value associated with the key, or defaultval if there's none.
			If called again with same key, will return the second value, and so on,
			or defaultval if there are none.
	%END REM
	Private z_lastList As Linkedlist
	Private z_lastkey As String
	Private z_bIsCurrent As Boolean
	Function Get(ByVal key$) As Variant
		Dim bFirst As Boolean
		If z_lastkey <> key Or Not z_bIsCurrent Then
			Set z_lastlist = getList(key)
			z_lastkey = key
			z_bIsCurrent = True
			bFirst = True
		End If
		If z_lastlist Is Nothing Then
			assign me.get, z_defaultval
		ElseIf bFirst Then
			z_lastList.first
			assign Me.Get, z_lastlist.Value
		ElseIf z_lastList.Next Then
			assign Me.Get, z_lastlist.Value
		Else
			assign Me.Get, z_defaultval
			ResetGet ' next get will retrieve a first value for the supplied key, even if it's this same key.
		End If
	End Function

	%REM
		Function isMore
		Description: Test whether there are more values associated with the key most recently passed to Get.
			You can also just keep calling Get until you receive the default value back (usually Null or Nothing).
		Returns: True if there are more values to fetch for the most recently used key.
	%END REM
	Function isMore As Boolean
		If Not (z_lastList Is Nothing) Then
			isMore = Not z_lastList.IsAtEnd
		End If
	End Function
	
	%REM
		Sub Remove
		Description: Delete the last-fetched value.
	%END REM
	Sub Remove
		If z_bIsCurrent Then
			If Not (z_lastList Is Nothing) Then
				If z_lastlist.Count = 1 Then
					Erase z_data(z_lastkey)
					ResetGet
					z_keyCount = z_keyCount - 1
				ElseIf z_lastList.IsAtEnd Then
					z_lastlist.Remove ' leaving us still at the end, so the next Get will return nothing.
				Else
					z_lastlist.Remove
					z_lastList.Prev ' back up to a previously read node so the next Get will return the node following the one we just deleted.
				End if
			End If
		End If
	End Sub

	%REM
		Sub ResetGet
		Description: Make sure the next call to Get returns the first value associated with the specified key.
	%END REM
	Sub ResetGet
		z_bIsCurrent = False
		Set z_lastList = Nothing
	End Sub
	
	Sub New(defaultval)
		z_defaultval = defaultval
		ResetGet
	End Sub
End Class

Does it need a Delete method?

As always when working with data structures, it’s important to consider whether we need to do anything special when the container object is deleted. We don’t want to leave “islands” of mutually referential objects that can’t be garbage collected even though no references remain to them from outside, because they’re each holding on to the other’s bootstraps.

In this case, the default deletion strategy should work without our help. The LinkedList class does have pairs of objects that contain references to each other, but it already has a programmed Delete method to take care of that. In general, as a best practice, make sure all your classes clean up their own messes when the object gets deleted.

When the ListOfList object is deleted, its members are also deleted, so every member of the internal List has its reference counter decremented and gets garbage collected. It’s possible the caller has used the getList method to create an additional reference to one of those LinkedList objects, so that list would hang around until the referring variable is erased — which is the desired behavior.

Determining whether a list is empty

None of the “is” functions can tell you whether a list contains any elements. This function does the job. Don’t use it while in a Forall loop of processing the list.

Function isListEmpty(aList) As Boolean
	Forall thing In aList
		Exit Function ' returning False
	End Forall
	isListEmpty = True
End Function

Erasing while iterating

Experimentation has shown that it does work to use the Erase statement on a list element while iterating through the list with Forall. It doesn’t freak out or skip elements.

Encapsulation and iteration

If the List variable is hidden away in a private class member, how is a caller supposed to get access to it to iterate through its contents with Forall?

Well… this is indeed a problem. In the example in the previous section, I tried to address it by providing a function that returns an array of all the keys so the caller can request the values for each of them in turn. But this solution is only reasonable for small data sets — it’s slow to compile that list and its size is limited to the size of an array.

I think slicker solutions to this will be a subject for a later article. For now: time for bed.

Notes

The getList method in the ListOfList class above is an alternate way for the caller to retrieve all the values associated with a given key with a single call. This is handy, but it’s also an encapsulation violation. The caller could alter the contents of the list in a way that would confuse the other methods in ListOfLists — specifically, by deleting all the values from it, which violates the assumption in the Get method that every list contains at least one value. Or by using the Delete method on the returned object, which leaves us internally with a list element whose value is Nothing, likely to cause invalid reference errors.

I leave it as an exercise to the reader to come up with a way to provide this function without exposing internal data to meddling by the caller.