Waiting for a runner to pick up this job: what to check

“Waiting for a runner to pick up this job” means GitHub has a job and no runner has taken it yet. Usually it lasts a few seconds. When it stays, the cause is on a short list, and which list depends on whose runners the job asked for.

First: is it GitHub?

Look at githubstatus.com. When Actions has an incident, jobs wait everywhere, and nothing on your side will help.

On GitHub’s own runners

A label no runner has. A typo in runs-on, or a label for a runner your plan does not have, does not fail the job. GitHub’s reference for self-hosted runners says:

If a job is labeled for a certain type of runner, but none matching that type are available, the job does not immediately fail at the time of queueing. Instead, the job will remain queued until the 24 hour timeout period expires.

Check the label against GitHub’s list of runner labels letter by letter.

Your plan’s limit on jobs at once. Jobs over the limit wait for a running one to end. GitHub’s limits for its standard runners:

A matrix of 30 jobs on the Free plan runs 20 and keeps 10 waiting. GitHub Support can raise the limits.

Larger runners. GitHub’s larger runners reference lists why their jobs wait:

• the runner’s own limit on jobs at once, set in its autoscaling settings;
• the repository not being allowed to use the runner’s group, which enterprise runners are not by default;
• no credit card on file (adding one “can take up to 10 minutes” to take effect), or a spending limit of zero;
• GitHub’s fair use throttling;
• a cold pool: “a warm VM may not be ready to pick up a job on first request as the pools for these machines are smaller”, so GitHub may have to create one.

Billing. Without a valid payment method, GitHub’s billing page says, “usage is blocked once you use up your quota”, and larger runners are blocked until you add one.

On your own runners

GitHub routes a job to “an online and idle runner that matches the job’s runs-on labels and groups”. If there is none, “the job will remain queued until a runner comes online”, and after 24 hours it fails. So the questions are the ones in that sentence.

Is there a runner with every label? A runner must have all of the job’s labels, not one of them. Compare the job’s runs-on with the labels under Settings → Actions → Runners.

Is it online? That page shows each runner as Idle, Active or Offline. GitHub’s troubleshooting guide gives three reasons for Offline: the machine is down, the runner program is not running, or it “cannot communicate with GitHub”. The last one is common in private subnets with no way out: a runner needs to reach GitHub over the internet, through a NAT gateway or a proxy.

Is it idle? A runner takes one job at a time. Ten jobs and two runners means eight waiting.

Is it too old? A runner that has not taken an update for 30 days gets no jobs: “If you do not perform a software update within 30 days, the GitHub Actions service will not queue jobs to your runner.”

May this repository use it? Runners belong to runner groups, and a group can be limited to some repositories. Public repositories are left out unless you say otherwise: “By default, only private repositories can access runners in a runner group” (runner groups).

When runners are started for each job

With an autoscaler, no runner exists until the job does, so “waiting” also covers the time to start one. A few things can stop that:

• The autoscaler never heard of the job. It learns of jobs from GitHub’s workflow_job webhook. Look at the webhook’s or the GitHub App’s recent deliveries: a failed delivery is a job nobody will start a runner for.
• It heard, but it did not recognise the label. CodeBuild, for one, needs the project’s name in the label. AWS’s tutorial warns that if it does not match, “CodeBuild will not process the webhook and the GitHub Actions workflow might hang”.
• The cloud said no. A spot machine with no capacity, a vCPU quota reached, or a new AWS account allowed no GPU machines at all. The autoscaler’s own log says which.
• The machine started but its runner never connected. The network again: a machine with no route to GitHub boots and waits forever.

How long is normal

On GitHub’s standard runners, seconds: GitHub keeps them warm. On larger runners, longer when the pool is cold. With a machine started for each job, the time to start it: SuperCI’s machines take about 15 seconds on AWS and about 7 on Cloudflare. Minutes are a problem, and an hour is never a slow start.

How SuperCI handles it

SuperCI, which we make, starts a fresh machine in your own AWS, Cloudflare or Modal account for each job. It tries to make “waiting” mean only the start:

• A job no provider can run fails at once, with the reason on the job’s page, instead of waiting 24 hours. A label it does not understand fails the same way.
• A provider that cannot start the machine (no spot capacity, a quota, an error) passes the job to the next one in your order.
• Jobs GitHub never told it of are found. It also asks GitHub for queued jobs, so one lost webhook does not lose a job.
• The dashboard shows each job’s wait, and the median for the day.

Troubleshooting has what to check when a SuperCI job waits.