design pipeline scheduling and the dag, covering data aware triggering that waits for data rather than for a clock, sensors and arrival timeouts, retry policy and non retryable failure classes, concurrency pools and queueing at peak, sla miss conditions and notification, catch up and backfill contro
design pipeline scheduling and the dag, covering data aware triggering that waits for data rather than for a clock, sensors and arrival timeouts, retry policy and non retryable failure classes, concurrency pools and queueing at peak, sla miss conditions and notification, catch up and backfill controls with partition bounds and concurrency caps, run isolation between backfill and scheduled runs, an