-{section: Autoclustering: SIGNIFICANT_ATTRIBUTES and Automatic Significant Attributes}
+{section: Debugging Smashed Stacks}
 
-The following is old documentation for the
-_SIGNIFICANT_ATTRIBUTES_ ("SA")
-setting in Condor.  It's here for two reasons.  1. Some customers are using
-SA, and you might need to deal with their RUST.  2. The automatic SA
-detection (appearing in 6.7.15) basically does this automatically, so this
-is a good introduction to the general principles. 3. If the automatic SA
-generation fails, you might need to call back on explicit SA.
-
-When attempting to match jobs, the negotiator basically asks the schedd,
-"Let me see the next job", repeating until the schedd says, "No more jobs".
-As an optimization, prior to SA if the negotiator could not match a given
-job, the schedd would not return any more processes in that given cluster
-for the current negotiation cycle.  So if 12.4 failed to match, the schedd
-would simply not type to match 12.5, 12.6 and so on.  The theory is that
-other procs in the same cluster likely have similar attributes and
-Requirements, so if one fails to match the others likely will fail to match
-as well, so why bother the negotiator with them?
-
-Problem 1: Sometimes different procs in a cluster had different
-attributes.  For example, 12.4 might fail to match because it's run (and
-was interrupted) once and has generated a monstrous ImageSize.  Problem 2:
-Identical jobs in different clusters would never be optimized this way.
-
-Solution: Autoclustering.  Automatically create clusters of jobs that
-are "identical" for purposes of matchmaking. The initial implementation was
-_SIGNIFICANT_ATTRIBUTES_, in which the administrator had to identify
-which attributes of a job were important for deciding that two jobs were,
-for matchmaking purposes, identical.
-
-The remainder of this document is heavily based on the original rough
-documentation given to users of _SIGNIFICANT_ATTRIBUTES_.  There is
-some out of date information, but the core design remains accurate.
-
-
-_SIGNIFICANT_ATTRIBUTES_ is a list of attributes from a schedd's
-job ads that define different jobs to use in negotiation. The
-schedd maintains a list of "autoclusters", and then tries to put
-each job into one of those autoclusters. For each job, it looks
-at the values of the attributes listed in _SIGNFICANT_ATTRIBUTES_.
-If all of those values match those of an existing autocluster, it
-adds the job to an existing autocluster. If not, it creates a new
-autocluster for that tuple of values.
+Debugging a smashed stack can be difficult because the backtrace is nearly useless. A few tips:
 
-To get the pre 6.7.15 behavior, just set
+{subsection: The Zen of Core Files}
 
-{code}SIGNIFICANT_ATTRIBUTES = ClusterId{endcode}
+(Section added Aug 1, 2006, adesmet)
 
-To make significant attributes useful, use the following formula:
-
-1: Collect all _START_ expressions from the pool. Find all job
-attributes that the _START_ expression references. ie:
-
-
-{code}
-Start = ((TARGET.ImageSize <= ((My.Memory - 15) * 1024))
-
-SIG_ATTRS_FROM_STARTD = ImageSize
-{endcode}
-
-
-2: Now add anything from the job ad referenced by the Central
-Manager's _PreemptionRequirements_.
-
-(_MY_ in _preemption_requirements_ is the startd ad of the machine
-we're considering for preemption, _Target_ is the job ad of that
-we're considering putting on the machine. Anything you want from
-the job that's currently running on the machine you have to
-export into the machine ad with _STARTD_JOB_EXPRS_)
-
-
-{code}
-Preemption_Requirements = (MY.ResearchGroupOfMachine == Target.ResGroupOfJob)
-
-SIG_ATTRS_FROM_PREEMPTION = ResGroupOfJob
-{endcode}
-
-
-3: At the schedd, we're always going to want to have jobs with
-different requirements be part of different AutoClusters.  (The
-schedd may do this automatically in the future.)
+Pete's core dump technique for finding buffer overruns on the stack:
 
+Check the actual addresses in the backtrace. If a few stack frames in a row fail to have the high bit set in any of the bytes of the address, chances are that an ASCII string has walked off the end of the buffer. An example from a real core dump:
 {code}
-SIG_ATTRS_OBVIOUSLY_NEEDED = Requirements
+#0  0x0818d216 in WriteCoreDump ()
+#1  0x0817db6c in linux_sig_coredump ()
+#2  0xffffe500 in ?? ()
+#3  0x0000000b in ?? ()
+#4  0x00000063 in ?? ()
+#5  0x00000000 in ?? ()
+#6  0x0000002b in ?? ()
+#7  0x0000002b in ?? ()
+#8  0x0854f47c in ?? ()
+#9  0x08554aa8 in ?? ()
+#10 0xffff8ed8 in ?? ()
+#11 0xffff8ed8 in ?? ()
+#12 0x00000330 in ?? () from /lib/libm.so.6
+#13 0x00002c0c in ?? () from /lib/libm.so.6
+#14 0xffff8f80 in ?? ()
+#15 0x2e303956 in ?? ()
+#16 0x0000000e in ?? ()
+#17 0x00000004 in ?? ()
+#18 0x0819e716 in List<char>::AtEnd ()
+#19 0x0819dee3 in List<char>::Next ()
+#20 0x081fd963 in GenericQuery::makeQuery ()
+#21 0x65646f6e in ?? ()
+#22 0x2e303932 in ?? ()
+#23 0x7361646c in ?? ()
+#24 0x7469632d in ?? ()
+#25 0x67696c2e in ?? ()
+#26 0x61632e6f in ?? ()
+#27 0x6365746c in ?? ()
+#28 0x64652e68 in ?? ()
+#29 0x00292275 in ?? ()
+#30 0x30323735 in ?? ()
+#31 0x2e303330 in ?? ()
+#32 0x08540030 in ?? ()
+#33 0x0062f8b8 in ?? ()
+#34 0x084b3800 in vtable for Daemon ()
+#35 0x00000000 in ?? ()
+#36 0x00000000 in ?? ()
+#37 0x00000000 in ?? ()
+#38 0x0ac87610 in ?? ()
+#39 0x00000000 in ?? ()
+#40 0x00000000 in ?? ()
+#41 0x00000000 in ?? ()
+#42 0x00000000 in ?? ()
+#43 0x00000001 in ?? ()
+#44 0x00000000 in ?? ()
+#45 0x0ac8c020 in ?? ()
 {endcode}
+If you think you might have this case, "strings core" can give some clues. After that, open the core file in less. Jump to the end (G), At the very end, or near it you'll find the environment. Jump backward a few pages (Ctrl+B) looking for a large string. That's probably the string responsible for overflowing the buffer on the stack.
 
+{subsection: gcc's -fstack-protector}
 
-4: Now, find all references in the requirements of the job ads
-for other things in the job ad, so two jobs that have the _same
-expression_ for requirements but _different values_ for that
-expression get placed in different autoclusters:
-
-
-{code}
-Requirements = (Target.HasSwapCkpt == TRUE &amp;&amp; (TARGET.Arch == "INTEL") \
-    &amp;&amp; (TARGET.OpSys == "LINUX") &amp;&amp; (TARGET.Disk &gt;= MY.DiskUsage)
-
-SIG_ATTRS_FROM_REQUIREMENTS = DiskUsage
-{endcode}
+(Section added Aug 1, 2006, adesmet)
 
+Greg Thain offers the following on August 1, 2006:
 
-We're now ready to create the list of significant attributes for the
-schedd:
-
-{code}
-SIGNIFICANT_ATTRIBUTES = $(SIG_ATTRS_FROM_STARTD),\
-                         $(SIG_ATTRS_FROM_PREEMPTION), \
-                         $(SIG_ATTRS_OBVIOUSLY_NEEDED), \
-                         $(SIG_ATTRS_FROM_REQUIREMENTS)
-{endcode}
+Gcc 4.1 has a new option that can help find these much, much faster. There's a new option, -fstack-protector, which puts a guard word on the stack of each function that has an array on the stack, and checks the guard at function exit. If the guard has changed, it asserts right then and there, before returning. This means that even though the stack is garbage, the innermost frame is still visible and debuggable in gcc. It also means that most buffer overflow attacks won't work.
 
+I've fixed a bunch of the condor source code to allow us to compile with this new gcc, but not all of the source code -- in particular, not the externals. Still, all of the daemons can be compiled with this option.
 
-Note, if you make your significant attributes too restrictive
-(you cluster jobs in ways that don't actually matter), you'll end
-up with extra negotiation.  This is harmless, but reduces the
-benefits of _SIGNIFICANT_ATTRIBUTES_.   "_SIGNIFICANT_ATTRIBUTES =
-ClusterId_" is the extreme version of this.
-
-If your significant attributes are too loose (you cluster
-dissimilar jobs), Condor may skip over jobs that can in fact be
-run.  If you follow the system above, you shouldn't run into
-this.
-
-Given the above, note that putting certain things in your
-_SIGNIFICANT_ATTRIBUTES_ will lead to worst case behavior: every
-single job gets its own cluster and zero optimization is performed.  This
-includes any unique or highly individual attribute.  Things
-that probably never belong in SA include _QDate, ClusterId, and
-ProcId_.  Note that the automatical significant attribute finder in
-6.7.15 will happily mark these as significant if they are present in the
-expressions above.  Thus, avoid putting unique or highly unique things in
-your machine expressions and the like.  For example, giving startds a
-"_RANK=2000000000-QDate_" to try and prioritize older jobs will
-cause worst case behavior.  It will work correctly, but slowly.
+We can't just switch to this new gcc because of standard universe and the externals, but I'd recommend to folks that if you suspect a stack buffer overflow, and are sending a debug binary out to a user, that you recompile with this option, it leaves a slime trail of awesomeness in its wake.