Conversion to New ClassAds

This is part of ticket #187. See #227 for benchmark comparisons.

Analysis of HTCondor's use of old ClassAds

Additional conversion info

Compatibility Class

 To ease the transition to new ClassAds, we will create compatibility methods that mimic the interface of old ClassAds. We will put these methods in a child class to new ClassAds called CompatClassAd. This child class will be part of the HTCondor C++ utility library. This leaves the new ClassAds library completely independent of other HTCondor code.

Methods with more than a dozen or so callsites will be emulated in CompatClassAd when feasible. Some methods, particularly those dealing with ExprTrees, cannot be emulated easily, and the callsites will have to be modified. Once the initial transition is complete, we can update callsites at our leisure, with a large dose of student work, until CompatClassAd is no longer needed.

ClassAd Functions

There are some ClassAd functions that exist in old ClassAds but not new ClassAds. These are ones based on HTCondor's StringList class and provide a poor man's list functionality in old ClassAds, by allowing a single string to be interpreted as a comma-separated list of items. We should support these functions in new ClassAds.

Direct use of ExprTree

Many parts of the code do the following to evaluate an expression in the context of a ClassAd:

These need to be converted to the following sequence:

This work can probably be done mostly by a student. We can also write compatibility versions of Parse() and EvalTree().

In new ClassAds, there is no assignment operator inside an ExprTree. A ClassAd contains a list of attribute name and ExprTree pairs. Any code in HTCondor that's directly inserting or extracting ExprTree's from a ClassAd will need to be updated.

One way to do this update in stages is to add new functions to old ClassAds that hide the assignment operator. They would operate on the attribute name as a string and the attribute value as an ExprTree. The internal structures could remain the same. Then, the users of the old functions could be be switched over to use the new ones. The new ClassAds compatibility class would implement the new functions.

Cross-Ad References

In old ClassAds, when evaluating an expression in the context two ads, if variable in the expression isn't in the local ad, it is looked for in the remote ad as well. The scopes 'my' and 'target' can be used to look in only one of the two ads for a variable. New ClassAds include a MatchClassAd for evaluating expressions in the context of two ads. But references to attributes in the remote ad must use the 'target' scope.

AttrList

Several parts of the code use AttrList, which is a parent class to ClassAd in old ClassAds. This code can be converted to use ClassAd instead if the put() or initFromStream() methods are used. Where those methods are used, alternative functions (which exist) need to used which don't include special handling for MyType and TargetType.

ClassAdList

There is no equivalent to the ClassAdList class in new ClassAds. We propose writing a compatibility ClassAdList that's a simple wrapper around an STL vector of ClassAd pointers. We would emulate the interface of the old ClassAdList, but not the reference counting. Nowhere do we put the same ClassAd into multiple ClassAdLists.

In a couple places (matchmaker.cpp and condor_query.cpp), we move an ad from one list to another. For these locations, we'll need to add a method to allow movement without deletion.

Operands to Logical Operators

In new ClassAds, the logical operators ('&&', '||', and '!') require their operands to be booleans. In old Classads, operands that are integers or reals are converted to boolean (based on whether they are 0). At present, it's unclear how often numbers are used in logical operations in the wild. Supporting this usage would be very annoying without changing new ClassAds to allow it.

'T' and 'F' Symbols

The tokens T and F are synonyms for True and False, and cannot be used as attribute names in old ClassAds. This is not the case in new ClassAds. Supporting this behavior would be difficult to do without modifying new ClassAds directly. My belief is that no one is aware of these symbols, and removing their special meaning will not affect any users.

STL Exceptions

Do we need to worry about handling exceptions from STL objects?

Chained Ads

New ClassAds support chaining, but the iterators ignore them. The chained ad can be referenced explicitly by callers where it matters. It should be possible to write custom iterators that visit the chained ad in addition to the current ad.

Warnings

It would be nice to emit warnings whereever the compatibility functions are invoked, so that we don't forget to convert them eventually.

Invisible Attributes

Old ClassAds support the notion of private attributes that can be marked invisible when exporting an ad. The set of invisible attributes is static and invisibility is only done for the put() and dPrint() calls, so we can handle this strictly in the compatibility functions.

String Escaping

String escaping is different between new and old ClassAds. With a couple minor tweaks, the compatibility functions will use the old ClassAd escaping rules. The new parser would continue to use the new escaping rules. We'll have to be careful when changing existing code to call the new methods, especially when parsing expressions taken from a user.

String Classes

New ClassAds use std::string while old ClassAds use MyString. The compatibility functions can accept MyString where appropriate. But as new code is written or old code is converted, they will have to start using std::string.