Troubleshooting

What to check when a job waits, fails at once or is refused, and where SuperCI says why.

SuperCI tries to say why, where you will see it: a job it cannot run fails on GitHub with a message at the top of its log, and the dashboard shows the same reason beside the job.

A job stays queued

  • The label is not SuperCI’s. runs-on must be your control plane’s label (superci unless you chose another), alone or with key=value pairs after it.
  • The repository is not connected. On GitHub, the App must be installed on it; on GitLab, the project must be turned on.
  • Every provider is full. The dashboard’s Runners page says what jobs wait for (“Cloudflare runs 10 at once”). Raise the limit or add a provider.
  • It waits for a spot machine. With on-demand turned off and no other provider that fits, a job waits until AWS has a spot machine.
  • Your AWS account’s limit is in use. When your own jobs are using all the CPUs your account may run, the next job waits for one of them to end. See Spot machines.

For jobs that wait on GitHub’s runners or on other self-hosted runners, Waiting for a runner to pick up this job has the causes.

A job was slow to start

superci jobs retrieve ID says how its start went, step by step: how long GitHub took to tell SuperCI of the job, every request to AWS for its machine and what AWS answered, the machine being made, its boot, and its runner reaching GitHub and being given the job. With --json the same is under start, in milliseconds.

A runner that took more than ten seconds to reach GitHub also says what it logged meanwhile: each wait of two seconds or more, with the line before and after, and its warnings and errors (start.runner_log with --json).

A job fails at once

The first line of its log on GitHub says why. Common ones:

  • Nothing can run its machine. The label asks for something no provider you added has: macOS (coming soon), a GPU without AWS or Modal, a size above your limit.
  • Something in the label is not understood. The message says which key or value, and lists what a label can say.
  • A public repository that is not allowed. See Public repositories.
  • AWS allows no GPU machines. New accounts start with a GPU quota of zero. The message links to the quota to raise.

A job was interrupted

AWS took its spot machine back, or Modal preempted its Function. The job stops within seconds, and is run again by itself once its run has finished. See Spot machines.

A job stops after six hours

That is the longest a job may run unless you set another: in the dashboard under Settings → Limits → Longest job, or with superci limits update --max-hours=24. The workflow needs the same in its timeout-minutes, since GitHub ends a job after six hours by itself. Up to five days on AWS and a day on Modal; on Cloudflare six hours is the longest a container lives, so send longer jobs to AWS with runs-on: superci/provider=aws. The GitHub Actions 6-hour limit has the other limits around it, GITHUB_TOKEN’s among them.

The dashboard asks to review permissions

A newer version needs something the control plane was not given yet (a permission on AWS, on the GitHub App, or a token scope). Control plane → Permissions lists what is needed and why, with one button to allow it.

A sign-in expired

A sign-in to one cloud that has ended stops nothing in the others. SuperCI says which it was and goes on with what the other sign-ins reach; superci login cloudflare (or aws, or modal) signs in again. Your control plane and runners keep working meanwhile.

The control plane is out of date

The dashboard offers Update when its version is newer than the control plane’s, and lists what the update brings. Updating takes a minute; jobs that are running are not touched.

Sending jobs back to GitHub’s runners

GitHub fixes a job’s runner when the job is queued, so a workflow cannot say “or else GitHub’s own”. A variable can. Put the label in an Actions variable of the repository or the organization, and name the variable in the workflow:

runs-on: ${{ vars.CI_RUNS_ON || 'ubuntu-latest' }}

With CI_RUNS_ON set to superci, jobs run on your machines. Change it to ubuntu-latest, or delete it, and the next jobs run on GitHub’s runners, with no commit. Change it back the same way.