AImageLab-HPC

Job Priority

Last updated: March 31, 2026


SLURM uses the multifactor priority plugin to assign a numeric priority to every pending job. A higher priority means the job is considered sooner for scheduling. Understanding the factors that compose this number helps you choose the right partition and QOS, and set realistic expectations about queue wait times.

Priority Formula

The effective priority of a job is the sum of several weighted components:

Priority = (PartitionWeight × PartitionFactor)
         + (QOSWeight      × QOSPriority)
         + (AgeWeight      × AgeFactor)
         + SiteFactor
Component Weight Notes
Partition factor 10,000,000 Based on the partition’s PriorityJobFactor
QOS priority 100,000 Based on the QOS Priority value
Age 1,000 Grows linearly; reaches maximum after 7 days in the queue
Site factor Budget-based fairshare (see below)
Fairshare (built-in) 0 Disabled — replaced by the site factor
Job size 0 Disabled

Priority Factors

QOS

The QOS you request is the most impactful lever available to you day-to-day. Each QOS carries a fixed priority value:

QOS Priority Max walltime Notes
all_qos_dbg 100 1 h Debug — highest priority; 1 GPU max; available on both production partitions
all_serial, all_usr_prod, boost_usr_prod, all_qos_lprod, qos_biz_lprod 50 varies Standard priority
all_qos_sprod, boost_qos_sprod, all_qos_tprod, boost_qos_tprod 40 24 h* Students and thesis students (auto-assigned)
all_qos_bio, all_qos_biol 0 varies Biomedical research team (restricted)

* boost_qos_sprod and boost_qos_tprod have no explicit QOS walltime limit; the partition’s 24 h cap applies.

Students and thesis students are automatically placed on their respective lower-priority QOS (all_qos_sprod/all_qos_tprod on all_usr_prod, boost_qos_sprod/boost_qos_tprod on boost_usr_prod) by the job submission system. This is intentional and ensures that student workloads do not displace research jobs.

Use all_qos_dbg when you need immediate or near-immediate access for a quick test — its priority 100 (vs. the standard 50) gives it a strong head start in the queue, but it is capped at 1 GPU and 1 hour. It is available on both all_usr_prod and boost_usr_prod.

See Scheduler and Job Submission for the full QOS reference including walltime limits, GPU caps, and partition compatibility.

Partition

Each partition has a PriorityJobFactor that is multiplied by the partition weight (10,000,000):

Partition PriorityJobFactor
all_serial 10
boost_usr_prod 5
all_usr_prod 0

all_serial (login nodes) has the highest factor because serial/interactive jobs are lightweight and expected to start promptly. Within the GPU partitions, boost_usr_prod carries a higher factor than all_usr_prod, reflecting the premium hardware it provides access to.

Age

Every minute a job waits in the queue its age factor grows, up to a maximum reached after 7 days. The age contribution at maximum is 1,000 × 1 = 1,000, which is small compared to the QOS and partition terms but becomes relevant when many jobs with the same QOS and partition are competing.

Budget Fairshare (Site Factor)

The scheduler includes a custom site factor plugin that adjusts each job’s priority based on the project’s recent budget consumption relative to its allocation. This is the mechanism described in the Budget and Accounting page:

  • Projects (and users within a project) that have consumed less than their expected monthly share recently receive a positive priority boost.
  • Projects that have consumed more receive a negative adjustment.

This is a soft mechanism: jobs are never blocked or cancelled due to budget, only reordered in the queue. The fairshare adjustment is continuous and recalculated regularly.

The built-in SLURM fairshare (PriorityWeightFairshare) is disabled (weight = 0). Fairshare is implemented entirely through the site factor plugin to allow tighter integration with the budget tracking system.

Inspecting Job Priority

Use sprio to see the individual priority components for your pending jobs:

sprio -u $USER

Example output:

  JOBID PARTITION   PRIORITY        AGE  FAIRSHARE        QOS  PARTITION
1234567 all_usr_p    5000201          1          0     500000          0
1234568 boost_usr    5540086        86          0    5000000     500000

To see all pending jobs sorted by priority:

sprio -S -y

Practical Recommendations

  • Use all_qos_dbg for quick tests. Its priority 100 means it almost always beats standard jobs (priority 50) in the queue, and the 1-hour cap keeps it from blocking others.
  • Don’t request more walltime than you need. While walltime does not directly affect priority, it constrains which jobs can backfill around yours, indirectly affecting wait time.
  • Spread work evenly over time. The site factor rewards projects that consume budget at a steady rate. Submitting large bursts after a quiet period will temporarily depress your fairshare bonus relative to projects with more consistent usage.
  • Check your budget share. If your project has consumed a large fraction of its monthly allocation recently, your jobs will compete at a disadvantage. Use susage to monitor consumption (see Budget and Accounting).