-{subsection: Igor's Thoughts re Pilot Factory Requirements} +{section: Flow-based grid submissions, aka Streaming CE} -{subsubsection: Server side requirements} +Currently, grid CEs based around technologies like Globus GRAM, CREAM, or Condor-C essentially are services that submit a job into a local site scheduler. -I have seen several people discussing the need for a "Pilot Factory", or "flow submission". -So here is my take on the requirements for a service that would submit flows of jobs; +This page contains thoughts re a "Streaming CE" (pseudonyms incl "Pilot Factory", or "flow submission"). +What we mean is a service that would submit flows of jobs; in the sense of "keep X jobs running at all times". -Before I begin, let me state that I discussed this with the Wisconsin group some time ago, -and they convinced me that this is really "Resource provisioning". -And I think they are right. -If you are asking to "keep X single core jobs running at all times" +When combined with the concept of glideins/pilot jobs, the idea of a streaming CE is to transform the job interface provided by site schedulers into a "resource provisioning" interface. +If you are asking to "keep X single core jobs running at all times", this is equivalent to saying "provision me X CPUs". The fact that I may need to move from one node to another is just a nuisance. -Said that, let me start with the description of what we are trying to solve: +{subsection: Requirements} +[this Requirements section is via emails written by Igor Sfiligoi] + +{subsubsection: Server side requirements} + + 1) Design a resource provisioning service to be deployed at Grid sites. For purely explanatory reasons, an example provisioning request would be "I need X single CPU slots with 3G of memory".