-<html> -<center> -<p><h1>Condor's Imake Build System Coding Guidlines and Conventions</h1></p> -<p> +{section: Condor's Imake Build System Coding Guidlines and Conventions} Revised: Tue Jun 22 14:24:56 CDT 2004 -</p> -</center> -<p> -<h2>Background</h2> +{section: Background} I, Pete Keller, am writing this in May 2004. The Imake system has been a painful thorn in the side of Condor since anyone could remember. This document describes the work which I have done and a new coding style for @@ -20,9 +14,9 @@ whitespace sensitive language, i.e, Makefiles. Currently, things *should* work for a good many number of ports, but the headaches of "keeping everything just right" might get to much, and the evolution will continue. -</p> -<p> + + The biggest problem with using imake seemed to be a fundamental lack of understanding for how it was supposed to work in the first place. Poorly implemented rules which "worked" on old cpps were simply copied around @@ -31,353 +25,345 @@ that they had avoided for so long. However, many OSes vendor cpps tread into undefined territory, so I worked hard to find a common ground, the stuff that all cpps are supposed to implement. -</p> -<p> + + The main culprits for the failure of our Imake system to be resilient and robust comes down to two systemic problems. Serious misuse of whitespace and the '##' concatenation operator. These two problems alone caused the most headaches while trying to port Condor to a new compiler which happens often and will, if it had been left alone, cause problems for the rest of eternity. -</p> -<h3>Guidelines and Rules for the Use of Imake with Condor</h3> -<ul> -<li> - <h4>The CPP</h4> +{section: Guidelines and Rules for the Use of Imake with Condor} + + + +{subsection: The CPP} + + +The build system has been changed to use the same cpp that comes +with the distribution of the target OS. This would be the same +cpp that the X Consortium source code would need to compile. On +most machines it is /lib/cpp, or even /usr/lib/cpp, etc. It +doesn't matter the revision of the cpp since the rest of the +guildlines explain how to write generalized preprocessor macros. +There is a new script, ansi_cpp, in the imake/ directory which figures +out which cpp to use for each architecture. On some architectures, +we are forced to use the vendor compiler, and so that must be installed. +On other architectures, there are a lot of cpp programs to choose from +and we must choose the right one. At all times, I've tried to stay +cannon to what the X Consortium code does to compile X. + + + + +{subsection: TAB characters} +TAB characters passed through cpp and then imake are preserved in all +cases except an errant gcc 3.1 revision. This errant version was fixed +by hacking our imake code to invoke the cpp differently. + + + + +{subsection: #endif Comments} +Comments on the same line as the #endif, but after it, must be C style +comments, not bare text. + + + + +{subsection: #define [thing] [stuff]} +[thing] MUST be a true C Identifier: + +_CORRECT_: + +=#define IS_I386_LINUX_RH9 YES= + +_INCORRECT_: + +=#define IS_I386_LINUX-RH9 YES= + + + + + +{subsection: Generated Makefile Comments} +If you'd like comments in the generated Makefile, please use XCOMM in +the Imakefile like this: +_XCOMM This is a comment in the generated Makefile_ + + +XCOMM is an imake program built in. + + + + + +{subsection: Macro Names} +Macro names must not unintentionally appear in any XCOMM comments as +they can be possibly expanded by various revisions of cpps. + + + + +{subsection: Macro Definitions} +There can be no whitespace in the parameter list of a macro +definition. Also, no whitespace around the interior of the +parenthesis, which is a common idiom in Condor C/C++ code. +Here is an example: + + + +_CORRECT_: =#define Concat(x,y)x##y= + +_INCORRECT_: =#define Concat( x, y )x##y= + + +In addition, there should be no spaces after the closing parenthesis +and the start of the nonwhitespace text--especially in a one +line macro rule: + +_CORRECT_: =#define copy(x,y)cp x y= + +_INCORRECT_: =#define copy(x,y) cp x y= + + + +If you have a multiline rule where you use @@\ to end the line(but +continue the rule to the next line), +any whitespace from the @@\ to the previous nonwhitespace character +on that line is undefined for its preservation. Do not rely +on the existance of that particular sort of whitespace. In practice, +however, no rules rely on this and it would be out of the ordinary +if a rule does. + + + + +{subsection: Rule Invocation in an Imakefile} + +There must be NO WHITESPACE around the open/closing parenthesis and +commas in rule invocation since this whitespace is undefined in its +preservation between cpps. However, whitespace could exist in the +invocation rule if it is separating items within a parameter: + + + +_CORRECT_: + +=program_target(thingy,$(OBJS) foo.o bar.o,$(LIBS))= +Notice the whitespace in the second parameter, whitespace of this +specific kind is valid. + + +_INCORRECT_: + +=program_target( thingy, $(OBJS) foo.o bar.o, $(LIBS) )= + + +In addition, you may NOT use the line continuation character: +='\'= anywhere in the line when invoking a rule: + + +_INCORRECT_: - <p> - The build system has been changed to use the same cpp that comes - with the distribution of the target OS. This would be the same - cpp that the X Consortium source code would need to compile. On - most machines it is /lib/cpp, or even /usr/lib/cpp, etc. It - doesn't matter the revision of the cpp since the rest of the - guildlines explain how to write generalized preprocessor macros. - There is a new script, ansi_cpp, in the imake/ directory which figures - out which cpp to use for each architecture. On some architectures, - we are forced to use the vendor compiler, and so that must be installed. - On other architectures, there are a lot of cpp programs to choose from - and we must choose the right one. At all times, I've tried to stay - cannon to what the X Consortium code does to compile X. - </p> -</li> - -<li> - <h4>TAB characters</h4> - <p> TAB characters passed through cpp and then imake are preserved in all - cases except an errant gcc 3.1 revision. This errant version was fixed - by hacking our imake code to invoke the cpp differently. - </p> -</li> - -<li> - <h4>#endif Comments</h4> - <p> Comments on the same line as the #endif, but after it, must be C style - comments, not bare text. - </p> -</li> - -<li> - <h4>#define [thing] [stuff]</h4> - <p> [thing] MUST be a true C Identifier: </p> - - <p><em>CORRECT</em>:</p> - <p><tt>#define IS_I386_LINUX_RH9 YES</tt></p> - <p><em>INCORRECT</em>:</p> - <p><tt>#define IS_I386_LINUX-RH9 YES</tt></p> - </p> -</li> - - -<li> - <h4>Generated Makefile Comments</h4> - <p> If you'd like comments in the generated Makefile, please use XCOMM in - the Imakefile like this: - <p><em>XCOMM This is a comment in the generated Makefile</em></p> - </p> - <p> - XCOMM is an imake program built in. - </p> - -</li> - -<li> - <h4>Macro Names</h4> - <p>Macro names must not unintentionally appear in any XCOMM comments as - they can be possibly expanded by various revisions of cpps. - </p> -</li> - -<li> - <h4>Macro Definitions</h4> - <p>There can be no whitespace in the parameter list of a macro - definition. Also, no whitespace around the interior of the - parenthesis, which is a common idiom in Condor C/C++ code. - Here is an example: - </p> - - <p> - <p><em>CORRECT</em>:</p> - <p><tt>#define Concat(x,y)x##y</tt></p> - <p><em>INCORRECT</em>:</p> - <p><tt>#define Concat( x, y )x##y</tt></p> - </p> - - <p> In addition, there should be no spaces after the closing parenthesis - and the start of the nonwhitespace text--especially in a one - line macro rule: - - <p><em>CORRECT</em>:</p> - <p><tt>#define copy(x,y)cp x y</tt></p> - <p><em>INCORRECT</em>:</p> - <p><tt>#define copy(x,y) cp x y</tt></p> - - </p> - - <p> If you have a multiline rule where you use @@\ to end the line(but - continue the rule to the next line), - any whitespace from the @@\ to the previous nonwhitespace character - on that line is undefined for its preservation. Do not rely - on the existance of that particular sort of whitespace. In practice, - however, no rules rely on this and it would be out of the ordinary - if a rule does. - </p> -</li> - -<li> - <h4>Rule Invocation in an Imakefile</h4> - <p> - There must be NO WHITESPACE around the open/closing parenthesis and - commas in rule invocation since this whitespace is undefined in its - preservation between cpps. However, whitespace could exist in the - invocation rule if it is separating items within a parameter: - </p> - - - <p><em>CORRECT</em>:</p> - <p> - <p><tt>program_target(thingy,$(OBJS) foo.o bar.o,$(LIBS))</tt></p> - Notice the whitespace in the second parameter, whitespace of this - specific kind is valid. - </p> - - <p><em>INCORRECT</em>:</p> - - <p><tt>program_target( thingy, $(OBJS) foo.o bar.o, $(LIBS) )</tt></p> - - <p> - In addition, you may NOT use the line continuation character: - <tt>'\'</tt> anywhere in the line when invoking a rule: - </p> - - <p><em>INCORRECT</em>:</p> - - <p> - <pre>program_target(thingy,$(OBJS) foo.o bar.o,\ - $(LIBS)) - </pre> - </p> - - <p> - Different pre-processors preserve or don't preserve the whitespace - up to the <tt>$(LIBS)</tt> lexeme and that can cause trouble - depending how your rules are layed out. Since it COULD be possible to - use the <tt>'\'</tt> character in some places and not others validly, - I'm striking out the use of it altogether so we don't have to - worry about the details of when you can or can't use it. - </p> - -</li> - -<li> - <h4>Use of the concatenation operator: ##</h4> - <p> - This operator is the main reason for the problems we've experienced - using the Imake system. People had been using ## - incorrectly in the sense that they were compressing - out whitespace preserved by cpp during incorrect Rule - Invocations but, in doing so, produced invalid C identifier - tokens with respect to the usage of ## according to the - specifications in the ANSI C manual. - </p> - - <p> - Since using ## to produce a non-C token was "undefined" - according to the specification, many, but not all, cpp - programmers honored the pasting together of two lexemes - for the reason of whitespace removal. But ultimately, - it was bad behavior and since the specification has - been tightened to reject arbitrary lexeme pastings by - many modern cpp versions. - </p> - - <p> - If you do need to use ##, then use one of the <tt>Concat*()</tt> - rules which exist for this purpose that are found - in Global.h. These rules embody special behavior for - certain cpp programs where ## can be replaced with /**/ - because ## wasn't supported correctly or even available - at all in the cpp. - </p> - - <p> - Since we now FORCE, by the creation of this document, that ANY and - ALL rule invocations must not have any extraneous whitespace, we can - consign the use of ## back to where it is supposed to be, in the - <tt>Concat*()</tt> rules. - </p> - - <p> This is the <em>ONLY VALID USE </em> - of the ## operator: </p> - <p> c_identifier1##c_identifer2 -> c_identifier1c_identifier2 <p> - - <p>These are examples of <em>INCORRECT</em> uses of the - ## operator:</p> - - <p> - <tt>path##/##to##/##some##/##dir##</tt><br> - <tt>directory##.static</tt><br> - <tt>##make_target##: stuff things other</tt><br> - <tt>perl condor_scripts/make_both_tarball -cmd "$(TAR_CMD)" strip##, name##, files##, contribfiles</tt><br> - <tt>DEPEND_C_SRC := $(##obj_list##:.o=.c)</tt><br> - </p> - - <p> Now, here is the important thing: </p> - - <p> - <span class="good"><em> - YOU WILL NEVER NEED TO USE THE ## OPERATOR EVER! USE ONE OF THE - <tt>Concat*()</tt> RULES INSTEAD AND ONLY WHEN PRODUCING A TRUE - C IDENTIFIER. - </em></span> - </p> - -</li> - -<li> - <h4> <tt>#</tt> and Preprocessor Directives </h4> - <p>When using preprocessor directives like <tt>#define</tt>, etc, - please be sure to place the <tt>#</tt> character in the - FIRST column of the text window. However, there can be - whitespace between the hash mark and the preprocessor - directive like this: - <pre> + +{code}program_target(thingy,$(OBJS) foo.o bar.o,\ +$(LIBS)) +{endcode} + + + +Different pre-processors preserve or don't preserve the whitespace +up to the =$(LIBS)= lexeme and that can cause trouble +depending how your rules are layed out. Since it COULD be possible to +use the ='\'= character in some places and not others validly, +I'm striking out the use of it altogether so we don't have to +worry about the details of when you can or can't use it. + + + + + +{subsection: Use of the concatenation operator: ##} + +This operator is the main reason for the problems we've experienced +using the Imake system. People had been using ## +incorrectly in the sense that they were compressing +out whitespace preserved by cpp during incorrect Rule +Invocations but, in doing so, produced invalid C identifier +tokens with respect to the usage of ## according to the +specifications in the ANSI C manual. + + + +Since using ## to produce a non-C token was "undefined" +according to the specification, many, but not all, cpp +programmers honored the pasting together of two lexemes +for the reason of whitespace removal. But ultimately, +it was bad behavior and since the specification has +been tightened to reject arbitrary lexeme pastings by +many modern cpp versions. + + + +If you do need to use ##, then use one of the =Concat*()= +rules which exist for this purpose that are found +in Global.h. These rules embody special behavior for +certain cpp programs where ## can be replaced with /**/ +because ## wasn't supported correctly or even available +at all in the cpp. + + + +Since we now FORCE, by the creation of this document, that ANY and +ALL rule invocations must not have any extraneous whitespace, we can +consign the use of ## back to where it is supposed to be, in the +=Concat*()= rules. + + +This is the _ONLY VALID USE _ +of the ## operator: +c_identifier1##c_identifer2 -> c_identifier1c_identifier2 + +These are examples of _INCORRECT_ uses of the +## operator: + +{code} +=path##/##to##/##some##/##dir##=<br> +=directory##.static=<br> +=##make_target##: stuff things other=<br> +=perl condor_scripts/make_both_tarball -cmd "$(TAR_CMD)" strip##, name##, files##, contribfiles=<br> +=DEPEND_C_SRC := $(##obj_list##:.o=.c)=<br> +{endcode} + +Now, here is the important thing: + + +*YOU WILL NEVER NEED TO USE THE ## OPERATOR EVER! USE ONE OF THE +=Concat*()= RULES INSTEAD AND ONLY WHEN PRODUCING A TRUE +C IDENTIFIER.* + + + + + +{subsection: =#= and Preprocessor Directives } +When using preprocessor directives like =#define=, etc, +please be sure to place the =#= character in the +FIRST column of the text window. However, there can be +whitespace between the hash mark and the preprocessor +directive like this: +{code} # define FOO BAR - </pre> - </p> -</li> - -</ul> - -<p> <h3>Conventions in the Imake Build Files</h3> </p> - -<ul> - -<li> - <h4>Macro Namespace</h4> - - <p>The name of a macro can only be used during the invocation of a macro. - </p> - <p> - For example, suppose we have a macro called - <tt>BUILD(x,y)</tt>. Now in an Imakefile rule, we have a - cleanup rule for a target which looks like this, <tt>rm - -rf BUILD</tt>--which is completely seperate from the - invocation of the macro rule <tt>BUILD(x,y)</tt>. In - this case the cleanup rule is an illegal use of the - macro <tt>BUILD(x,y)</tt>. - </p> - <p> In short, do not mix the namespaces of the macro names and the - entities the generated Makefile will manage. - </p> - -</li> - -<li> - <h4>Code Reuse</h4> - <p>Suppose in a rule you'd like to have _static appended to - a directory name in multiple places or something - similar. Instead of using Concat(name,_static) everywhere, - write a simple rule like this (toward the beginning of - the Imake.rules file): - </p> - - <p><tt>#define sufstatic(x)Concat(x,_static)</tt></p> - <p> And now whenever you want foo_static, you write sufstatic(foo) instead. - This will greatly help in producing maintainable and defensive code. - </p> - -</li> - -<li> - <h4>Writing New Imake Rules</h4> - <p> New rules should be written in this sort of style: </p> - <pre> - #ifndef release_target - #define release_target(file,dir,mode) @@\ - XCOMM Begin translation of func(release_target) @@\ - $(RELEASE_DIR)/dir/file: file @@\ - /bin/rm -f $(RELEASE_DIR)/dir/file @@\ - cp file $(RELEASE_DIR)/dir @@\ - chmod mode $(RELEASE_DIR)/dir/file @@\ - release:: $(RELEASE_DIR)/dir/file @@\ - XCOMM End translation of func(release_target) - #endif /* release_target */ - </pre> - - <p> - - Notice the XCOMM comments dictating that we are beginning - and ending the translation of this particular function. The - macro <tt>func()</tt> is defined in Imake.rules which - expands(even in the comment) the given function name into a nice piece - of text in the Makefile which describes the line of translation of the - associated function in the original Imakefile. While these are not - required for the rule to function, I <span class="good">STRONGLY - ENCOURAGE</span> you to follow it. Obviously, for very simple rules - like the <tt>sufstatic()</tt> rule above, you don't need it, but - for anything more complicated you should strive to put it in. - </p> -</li> - -<li> - <h4>Converting older style Imake rules to New Imake Rules</h4> - - <p>If you see any function (simple or complex) - that does not follow the conventions outlined in this document then - update the function to follow the guidlines. This is more of - a concern while merging as these changes propogate through the various - branches. But if you are editing a source file which - looks like it is adhering to these conventions, then assume it is - adhering to these conventions and change it accordingly. - </p> -</li> - -<li> - <h4>The Imake processor is the - <em>C Preprocessor</em>:</h4> - <p>The usual caveats to this applies, e.g.:</p> - <p>If you need to do something like this:</p> - - <p><tt>#define cat(x,y)x##y</tt></p> - <p>and you want to use it like this: <tt>cat(cat(1,2),3)</tt> it will - do the wrong thing and produce <tt>cat(1,2)3</tt>. If you want to do - it right, then you'd need an additional rule like this:</p> - <p><tt>#define xcat(x,y)cat(x,y)</tt></p> - <p>which would then produce the correct concatenation sequence. - Remember that section A.12 in the K&R C book applies.</p> - - - <p>In the case of the Condor build system, - <em>this specific example is already implemented</em> - in the <tt>Concat*()</tt> rules written in - <tt>Global.h</tt>.</p> +{endcode} + + + + + +{section: Conventions in the Imake Build Files} + + + + +{subsection: Macro Namespace} + +The name of a macro can only be used during the invocation of a macro. + + +For example, suppose we have a macro called +=BUILD(x,y)=. Now in an Imakefile rule, we have a +cleanup rule for a target which looks like this, =rm +-rf BUILD=--which is completely seperate from the +invocation of the macro rule =BUILD(x,y)=. In +this case the cleanup rule is an illegal use of the +macro =BUILD(x,y)=. + +In short, do not mix the namespaces of the macro names and the +entities the generated Makefile will manage. + + + + + +{subsection: Code Reuse} +Suppose in a rule you'd like to have _static appended to +a directory name in multiple places or something +similar. Instead of using Concat(name,_static) everywhere, +write a simple rule like this (toward the beginning of +the Imake.rules file): + + +=#define sufstatic(x)Concat(x,_static)= +And now whenever you want foo_static, you write sufstatic(foo) instead. +This will greatly help in producing maintainable and defensive code. + + + + + +{subsection: Writing New Imake Rules} +New rules should be written in this sort of style: +{code} +#ifndef release_target +#define release_target(file,dir,mode) @@\ +XCOMM Begin translation of func(release_target) @@\ +$(RELEASE_DIR)/dir/file: file @@\ +/bin/rm -f $(RELEASE_DIR)/dir/file @@\ +cp file $(RELEASE_DIR)/dir @@\ +chmod mode $(RELEASE_DIR)/dir/file @@\ +release:: $(RELEASE_DIR)/dir/file @@\ +XCOMM End translation of func(release_target) +#endif /* release_target */ +{endcode} + + + +Notice the XCOMM comments dictating that we are beginning +and ending the translation of this particular function. The +macro =func()= is defined in Imake.rules which +expands(even in the comment) the given function name into a nice piece +of text in the Makefile which describes the line of translation of the +associated function in the original Imakefile. While these are not +required for the rule to function, I *STRONGLY +ENCOURAGE* you to follow it. Obviously, for very simple rules +like the =sufstatic()= rule above, you don't need it, but +for anything more complicated you should strive to put it in. + + + + +{subsection: Converting older style Imake rules to New Imake Rules} + +If you see any function (simple or complex) +that does not follow the conventions outlined in this document then +update the function to follow the guidlines. This is more of +a concern while merging as these changes propogate through the various +branches. But if you are editing a source file which +looks like it is adhering to these conventions, then assume it is +adhering to these conventions and change it accordingly. + + + + +{subsection: The Imake processor is the C Preprocessor:} +The usual caveats to this applies, e.g.: +If you need to do something like this: -</li> +=#define cat(x,y)x##y= +and you want to use it like this: =cat(cat(1,2),3)= it will +do the wrong thing and produce =cat(1,2)3=. If you want to do +it right, then you'd need an additional rule like this: +=#define xcat(x,y)cat(x,y)= +which would then produce the correct concatenation sequence. +Remember that section A.12 in the K&R C book applies. -</ul> -</html> +In the case of the Condor build system, +_this specific example is already implemented_ +in the =Concat*()= rules written in +=Global.h=.