Adding Statistics Values

Not logged in

• Browse
• Help
• Home
• Login
• Reports
• Search
• Timeline
• Wiki

• Contents
• No-Diff
• Text

 {subsection: Adding statistics values to an existing collection}
 To add a statistics value where there is already an existing collection of values
 
-*: Add a member of type =stats_entry_xxx= to the stats collection class. the most common type is =stats_entry_recent=. {linebreak}
+*: Declare a member of type =stats_entry_xxx= and add it to the stats collection class. the most common type is =stats_entry_recent=.  The collection is a member of type {quote: StatisticsPool} and is usually
+a named Pool
 *: Set or increment the variable in the code to be measured.
-*: Find the =::Publish= method of the collection class, it will refer to a =static const= array of =GenericStatsPubItem=.  Add one or more entries in this array that refer to your new member. for =stats_entry_recent= type members, you will typically add a =_STATS_PUB= entry to publish the overall value and a =_STATS_PUB_RECENT= one to publish the Recent value.
-
-This array is used by the code that publishes the collection as well as code that advances the recent buffers and sets their sizes.
 
 {subsection: Adding a statistics value when there isn't a collection}
 At the most basic level, use generic statistics by:
@@ -17,10 +15,11 @@
 *: Add member variables of type =stats_entry_xxx= to your class or as a static or global
 *: Set the size of the recent buffer if the type is =stats_entry_recent= (you can do this in the constructor)
 *: Set or increment the variable in the code to be measured.
-*: Periodically call the =Advance= method to rotate the recent buffer
+*: Periodically call the =Advance= or =AdvanceBy= method to rotate the recent buffer
 *: call the =Publish= method to write the value as a {quote: ClassAd} attribute.
 
 {code}
+
 class MyObject {
     ...
     // declare a statistics counter
@@ -54,10 +53,14 @@
 // publish the counter value(s) in another worker method
 void MyObject::MyPublishStatisticsMethod(ClassAd & ad) {
 
+    if (publish_all) {
+       myObjectRuntime.Publish(ad, "ObjectRuntime");
+    } else {
     // publish the overall value
-    myObjectRuntime.PublishValue(ad, "ObjectRuntime");
+      myObjectRuntime.Publish(ad, "ObjectRuntime", myObjectRuntime.PubValue);
     // publish the recent value
-    myObjectRuntime.PublishRecent(ad, "RecentObjectRuntime");
+      myObjectRuntime.Publish(ad, "RecentObjectRuntime", myObjectRuntime.PubRecent);
+    }
 
 }
 {endcode}
@@ -70,18 +73,19 @@
 
 {subsection: Creating a new statistics collection}
 
-It is common to have multiple statistics values to measure a subsystem that are intended to be published together. When this is the case is most convenient to gather then up into a container class/struct and then use the helper functions in generic_stats.cpp to Clear, Publish and Advance the collection of counters as a unit.
+It is common to have multiple statistics values to measure a subsystem that are intended to be published together. When this is the case is most convenient to gather then up into a container class/struct and then add a {quote: StatisticsPool} member to the class so that you can use it's methods to Clear, Publish and Advance the collection of counters as a unit.
 
-As a general rule a statistics collection defined by where and when the data will be gathererd rather rather than by how and where the data is published.
-If there is a need to publish the same value in more than one {quote: ClassAd} with a different name in each add, this can be accomplished by using more than one GenericStatsPubItem table and multiple calls to the helper functions.
+As a general rule a statistics collection is defined by where and when the data will be gathererd rather rather than by how and where the data is published.
 
-To take advantage of these helper functions for working with collections of predefined statistics you will.
+If there is a need to publish the same value in more than one {quote: ClassAd} with a different name in each add, this can be accomplished by calling the Publish methods of the counters explicitly or adding the counters to the {quote: StatisticsPool} multiple times with different names.
 
-*: create a structure or class to hold your counters {linebreak}
-*:: use =stats_entry_recent<type>= for counters that should publish recent values (i.e. number of jobs that have finished){linebreak}
-*:: use =stats_entry_abs<type>= for counters that are instantaneous values for which you may want to publish a max value (i.e. number of shadow currently alive){linebreak}
-*:: use =stats_entry_probe<type>= for counters that publish min,max,avg,stdev{linebreak}
-*: give your stats class methods for Init, Clear, Tick and Publish, and have those methods call the corresponding generic_stats_ functions.
+*: create a structure or class to hold your probes {linebreak}
+*:: use =stats_entry_recent<type>= for probes that should publish recent values (i.e. number of jobs that have finished){linebreak}
+*:: use =stats_entry_abs<type>= for probes that are instantaneous values for which you may want to publish a max value (i.e. number of shadow currently alive){linebreak}
+*:: use =stats_recent_counter_timer= for probes that accumulate runtime
+*:: use =stats_entry_probe<type>= for probes that publish min,max,avg,stdev{linebreak}
+*:: use =stats_entry_recent<Probe>= for probes that publish min,max,avg,stdev and also keep a recent buffer of the same.{linebreak}
+*: give your stats class methods for Init, Clear, Tick and Publish, and have those methods call the corresponding methods on the {quote: StatisticsPool} member
 
 {code}
 class MyStats {
@@ -96,40 +100,34 @@
     stats_entry_recent<double> JobsRunningTime;
     ... etc
 
+    StatisticsPool  Pool;
+
     void Init();
     void Clear();
     void Tick();
-    void SetRecentWindowSize(int window); // in sec
     void Publish(ClassAd & ad) const;
     void Unpublish(ClassAd & ad) const;
 
 } MyStats;
 {endcode}
 
-Then in your cpp file for the =MyStats= class, define a static const array of =GenericStatsPubItem=.  This array is used by the =generic_stats_= helper functions.
-
-the =GENERIC_STATS_PUB_xxx= macros make initializing the array with standard values easy.  If you use these macros, the attribute names that are used to publish the counters will be derived from the counter variable names themselves.  In the example below, =MYSTATS_PUB(JobsSubmitted, AS_COUNT)= results in a attribute name of =MyStat_JobsSubmitted=.
+Then in your cpp file for the =MyStats= class
 
 {code}
-#define MYSTATS_PUB(name, as)         GENERIC_STATS_PUB(MyStats, "MYStat_", name, as)
-#define MYSTATS_PUB_RECENT(name, as)  GENERIC_STATS_PUB_RECENT(MyStats, "MYStat_", name, as)
-#define MYSTATS_PUB_TYPE(name, T, as) GENERIC_STATS_PUB_TYPE(MyStats, "MYStat_", name, as, T)
-
+void MyStats::Init()
+{
+   const int recent_window = 60*20;
+   const int window_quantum = 60*4;
 
-static const GenericStatsPubItem MyStatsPub[] = {
-   MYSTATS_PUB_TYPE(Lifetime,       time_t,  AS_RELTIME),
-   MYSTATS_PUB_TYPE(LastUpdateTime, time_t,  AS_ABSTIME),
-
-   MYSTATS_PUB(JobsSubmitted,          AS_COUNT),
-   MYSTATS_PUB_RECENT(JobsSubmitted,   AS_COUNT),
-   MYSTATS_PUB(JobsTimeToStart,        AS_RELTIME),
-   MYSTATS_PUB_RECENT(JobsTimeToStart, AS_RELTIME),
-   MYSTATS_PUB(JobsRunningTime,        AS_RELTIME),
-   MYSTATS_PUB_RECENT(JobsRunningTime, AS_RELTIME),
+   Pool.AddProbe(JobsSubmitted,   "JobsSubmitted");
+   Pool.AddProbe(JobsTimeToStart, "JobsTimeToStart");
+   Pool.AddProbe(JobsRunningTime, "JobsRunningTime");
+   Pool.SetRecentMax(recent_window, window_quantum);
+}
 
 void MyStats::Clear()
 {
-   generic_stats_Clear(MyStatsPub, COUNTOF(MyStatsPub), (char*)this);
+   Pool.Clear();
    this->InitTime = time(NULL);
    this->StatsLifetime = 0;
    this->StatsLastUpdateTime = 0;
@@ -139,12 +137,12 @@
 
 void MyStats::Publish(ClassAd & ad) const
 {
-   generic_stats_PublishToClassAd(ad, MyStatsPub, COUNTOF(MyStatsPub), (const char *)this);
+   Pool.Publish();
 }
 
 void MyStats::Unpublish(ClassAd & ad) const
 {
-   generic_stats_DeleteInClassAd(ad, MyStatsPub, COUNTOF(MyStatsPub), (const char *)this);
+   Pool.Unpublish();
 }
 
 void MyStats::Tick()
@@ -153,8 +151,6 @@
    const int my_quantum = 4*60;  // in seconds
 
    int cAdvance = generic_stats_Tick(
-      StatsTick, COUNTOF(StatsTick),
-      (char*)this,
       my_window,
       my_quantum,
       this->InitTime,
@@ -162,77 +158,62 @@
       this->RecentTickTime,
       this->Lifetime,
       this->RecentLifetime);
+   if (cAdvance)
+      Pool.Advance(cAdvance);
 }
 {endcode}
 
 {subsection: Adding statistics values at runtime}
 
-Note: this section is speculative, the code does not yet work (or work like this..)
-
-Your statistics class can also contain an instance of =stats_pool= if you need to handle values that are allocated and named at runtime rather than compile time.
+Probes can be added to the {quote: StatisticsPool} at runtime.  They can also be queried out of the pool, athough this is more expensive and error-prone than merely referencing a member variable.
 
 {code}
-class MyStats : public stats_pool {
-    time_t                  InitTime
-    time_t                  Lifetime;
-    time_t                  LastUpdateTime;
-    time_t                  RecentLifetime;
-    time_t                  RecentTickTime;
+class MyStats {
+    ...
 
-    void Tick();
-    void NewProbe(const char * category, const char * name, int cls);
-    void Sample(const char * name, int val);
-    void Sample(const char * name, double val);
+    StatisticsPool  Pool;
+
+    ...
+
+    void NewProbe(const char * category, const char * ident, int as_type);
+    void Sample(const char * ident, int val);
+    void Sample(const char * ident, double val);
 
 } MyStats;
 
-void MyStats::NewProbe(const char * category, const char * name, int as_type)
+// create a new probe with name name
+//
+void MyStats::NewProbe(const char * category, const char * ident, int as_type)
 {
     MyString attr;
-    attr.sprintf("MYStat_%s_%s", category, name);
-    MakeCleanAttribNameString(attr);
+    attr.sprintf("MYStat_%s_%s", category, ident);
+    cleanStringForUseAsAttr(attr);
 
     int cRecent = my_window / my_quantum;
     switch (as_type)
     {
     case STATS_ENTRY_TYPE_INT32 | AS_RECENT:
-       {
-          stats_entry_recent<int>* probe = new stats_entry_recent<int>(cRecent);
-          this->AddProbe(name, as_type, probe, attr.Value());
-       }
+       Pool.NewProbe< stats_entry_recent<int> >(ident, attr.Value(), 0);
+       break;
+
+    case STATS_ENTRY_TYPE_DOUBLE | AS_RECENT:
+       Pool.NewProbe< stats_entry_recent<double> >(ident, attr.Value(), 0);
        break;
     }
 }
 
-void MyStats::Sample(const char * name, int val)
+void MyStats::Sample(const char * ident, int val)
 {
-    stats_entry_recent<int>* probe = this->Get<stats_entry_recent<int>*>(name);
-    if (probe) probe->Add(val);
+    stats_entry_recent<int>* probe = Pool.Get< stats_entry_recent<int> >(ident);
+    if (probe)
+       probe->Add(val);
 }
 
 void MyStats::Sample(const char * name, double val)
 {
-    stats_entry_recent<double>* probe = this->Get<stats_entry_recent<double>*>(name);
-    if (probe) probe->Add(double);
-}
-
-void MyStats::Tick()
-{
-   const int my_window  = 20*60; // seconds
-   const int my_quantum = 4*60;  // in seconds
-
-   int cAdvance = generic_stats_Tick(
-      StatsTick, COUNTOF(StatsTick),
-      (char*)this,
-      my_window,
-      my_quantum,
-      this->InitTime,
-      this->LastUpdateTime,
-      this->RecentTickTime,
-      this->Lifetime,
-      this->RecentLifetime);
-
-   Advance(cAdvance); // call the stats_pool Advance method.
+    stats_entry_recent<double>* probe = Pool.Get< stats_entry_recent<double> >(ident);
+    if (probe)
+       probe->Add(double);
 }
 
 {endcode}

This work is supported by NSF under Cooperative Agreement OAC-2030508 as part of the PATh Project. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF. Site built using CVSTrac.