Using the python-condor library

python-condor is a wrapper module containing mechanisms for interacting with the HTCondor system. It is included by default in 7.9.4.

Basics - Job Submission

1: Begin by importing the condor and classad modules from a python interpreter. Note: some distributions require you import the module as htcondor 1: Declare a collector with an IP address/hostname. 1: Locate a scheduler by applying the 'locate' method on the collector object. 1: Parse a classad file with the classad library's .parse() method. (Note: these are raw classad files. They are similar to submit files, but vary in interesting ways. condor_q -l xx.xx will display the classad data for a particular job.) 1: Using the .submit method to submit the file to the appropriate scheduler. {code} [bbockelm@example python-condor]$ python Python 2.6.6 (r266:84292, Jun 18 2012, 09:57:52) [GCC 4.4.6 20110731 (Red Hat 4.4.6-3)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import condor >>> import classad >>> coll = condor.Collector("red-condor.unl.edu") >>> results = coll.query(condor.AdTypes.Startd, "true", ["Name"]) >>> len(results) 3812 >>> results[0] [ Name = "slot1@red-d20n35"; MyType = "Machine"; TargetType = "Job" ] >>> scheddAd = coll.locate(condor.DaemonTypes.Schedd, "red-gw1.unl.edu") >>> scheddAd["ScheddIpAddr"] '<129.93.239.132:53020>' >>> schedd = condor.Schedd(scheddAd) >>> results = schedd.query('Owner =?= "cmsprod088"', ["ClusterId", "ProcId"]) >>> len(results) 63 >>> results[0] [ MyType = "Job"; TargetType = "Machine"; ServerTime = 1356722353; ClusterId = 674143; ProcId = 0 ] >>> condor.param["COLLECTOR_HOST"] 'hcc-briantest.unl.edu' >>> schedd = condor.Schedd() # Defaults to the local schedd. >>> results = schedd.query() >>> results[0]["RequestMemory"] ifthenelse(MemoryUsage isnt undefined,MemoryUsage,( ImageSize + 1023 ) / 1024) >>> results[0]["RequestMemory"].eval() 1L >>> ad=classad.parse(open("test.submit.ad")) >>> print schedd.submit(ad, 2) # Submits two jobs in the cluster; edit test.submit.ad to preference. 110 >>> print schedd.act(condor.JobAction.Remove, ["111.0", "110.0"])' [ TotalNotFound = 0; TotalPermissionDenied = 0; TotalAlreadyDone = 0; TotalJobAds = 2; TotalSuccess = 2; TotalChangedAds = 1; TotalBadStatus = 0; TotalError = 0 ] >>> print schedd.act(condor.JobAction.Hold, "Owner =?= \"bbockelm\"")' [ TotalNotFound = 0; TotalPermissionDenied = 0; TotalAlreadyDone = 0; TotalJobAds = 2; TotalSuccess = 2; TotalChangedAds = 1; TotalBadStatus = 0; TotalError = 0 ] >>> schedd.edit('Owner =?= "bbockelm"', "Foo", classad.ExprTree('"baz"')) >>> schedd.edit(["110.0"], "Foo", '"bar"') >>> coll = condor.Collector() >>> master_ad = coll.locate(condor.DaemonTypes.Master) >>> condor.send_command(master_ad, condor.DaemonCommands.Reconfig) # Reconfigures the local master and all children >>> condor.version() '$CondorVersion: 7.9.4 Jan 02 2013 PRE-RELEASE-UWCS $' >>> condor.platform() '$CondorPlatform: X86_64-ScientificLinux_6.3 $' >>> {endcode}