Page History

Turn Off History

Using the python-condor library

python-condor is a wrapper module containing mechanisms for interacting with the HTCondor system. It is included by default in 7.9.4.

Basics - Job Submission

  1. Begin by importing the condor and classad modules from a python interpreter.
    Note: some distributions require you import the module as htcondor

  2. Declare a collector with an IP address/hostname.
  3. Locate a scheduler by applying the 'locate' method on the collector object.
  4. Parse a classad file with the classad library's .parse() method. (Note: these are raw classad files. They are similar to submit files, but vary in interesting ways. condor_q -l xx.xx will display the classad data for a particular job.)
  5. Using the .submit method to submit the file to the appropriate scheduler. 
    [bbockelm@example python-condor]$ python
Python 2.6.6 (r266:84292, Jun 18 2012, 09:57:52)
[GCC 4.4.6 20110731 (Red Hat 4.4.6-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import condor
>>> import classad
>>> coll = condor.Collector("red-condor.unl.edu")
>>> results = coll.query(condor.AdTypes.Startd, "true", ["Name"])
>>> len(results)
3812
>>> results[0]
[ Name = "slot1@red-d20n35"; MyType = "Machine"; TargetType = "Job" ]
>>> scheddAd = coll.locate(condor.DaemonTypes.Schedd, "red-gw1.unl.edu")
>>> scheddAd["ScheddIpAddr"]
'<129.93.239.132:53020>'
>>> schedd = condor.Schedd(scheddAd)
>>> results = schedd.query('Owner =?= "cmsprod088"', ["ClusterId", "ProcId"])
>>> len(results)
63
>>> results[0]
[ MyType = "Job"; TargetType = "Machine"; ServerTime = 1356722353; ClusterId = 674143; ProcId = 0 ]
>>> condor.param["COLLECTOR_HOST"]
'hcc-briantest.unl.edu'
>>> schedd = condor.Schedd() # Defaults to the local schedd.
>>> results = schedd.query()
>>> results[0]["RequestMemory"]
ifthenelse(MemoryUsage isnt undefined,MemoryUsage,( ImageSize + 1023 ) / 1024)
>>> results[0]["RequestMemory"].eval()
1L
>>> ad=classad.parse(open("test.submit.ad"))
>>> print schedd.submit(ad, 2) # Submits two jobs in the cluster; edit test.submit.ad to preference.
110
>>> print schedd.act(condor.JobAction.Remove, ["111.0", "110.0"])'

    [
        TotalNotFound = 0;
        TotalPermissionDenied = 0;
        TotalAlreadyDone = 0;
        TotalJobAds = 2;
        TotalSuccess = 2;
        TotalChangedAds = 1;
        TotalBadStatus = 0;
        TotalError = 0
    ]
>>> print schedd.act(condor.JobAction.Hold, "Owner =?= \"bbockelm\"")'

    [
        TotalNotFound = 0;
        TotalPermissionDenied = 0;
        TotalAlreadyDone = 0;
        TotalJobAds = 2;
        TotalSuccess = 2;
        TotalChangedAds = 1;
        TotalBadStatus = 0;
        TotalError = 0
    ]
>>> schedd.edit('Owner =?= "bbockelm"', "Foo", classad.ExprTree('"baz"'))
>>> schedd.edit(["110.0"], "Foo", '"bar"')
>>> coll = condor.Collector()
>>> master_ad = coll.locate(condor.DaemonTypes.Master)
>>> condor.send_command(master_ad, condor.DaemonCommands.Reconfig) # Reconfigures the local master and all children
>>> condor.version()
'$CondorVersion: 7.9.4 Jan 02 2013 PRE-RELEASE-UWCS $'
>>> condor.platform()
'$CondorPlatform: X86_64-ScientificLinux_6.3 $'
>>>