Home / CI/CD / Fix GitLab CI Pipeline Stuck in Pending

CI/CD

Fix GitLab CI Pipeline Stuck in Pending

Resolve GitLab CI pipelines stuck in pending state by checking runner availability, job limits, and concurrency settings.

Published: Apr 5, 20269 min readBy FixWikiHub Editorial Team

Abstract illustration for a troubleshooting knowledge base category.

Introduction

GitLab CI pipeline jobs remain stuck in "pending" state, never being picked up by runners. Pipelines queue indefinitely, blocking deployments and causing development delays.

Symptoms

Pipeline stuck pending:

```bash # In GitLab UI: Pipeline #123 - pending - Created 1 hour ago

Job 1: build - pending (stuck) Job 2: test - pending (waiting for build) Job 3: deploy - pending (waiting for test) ```

Job details:

bash

This job is stuck, because the project doesn't have any runners online assigned to it.

Runner status:

```bash $ gitlab-runner list

Configured runners: my-runner Token=xxx URL=https://gitlab.com Status=offline ```

Common Causes

1.No runners available - All runners offline or paused
2.Runner capacity full - concurrent limit reached
3.Tag mismatch - Job tags don't match runner tags
4.Runner paused - Runner explicitly paused
5.Project not shared - Runner not assigned to project
6.Job limit exceeded - Project job limit reached

Step-by-Step Fix

Step 1: Check Runner Status

```bash # List runners in GitLab UI # Admin Area > CI/CD > Runners

# Or via API: curl --header "PRIVATE-TOKEN: <token>" "https://gitlab.com/api/v4/runners"

# Check specific runner curl --header "PRIVATE-TOKEN: <token>" "https://gitlab.com/api/v4/runners/<runner-id>"

# Check runner status locally gitlab-runner status

# List configured runners gitlab-runner list

# Verify runner is running systemctl status gitlab-runner docker ps | grep gitlab-runner ```

Step 2: Verify Runner is Online

```bash # Check runner process ps aux | grep gitlab-runner

# Check runner logs journalctl -u gitlab-runner -f docker logs gitlab-runner -f

# Common issues: # - Runner crashed # - Network connectivity to GitLab # - Token expired # - Config file missing

# Restart runner systemctl restart gitlab-runner docker restart gitlab-runner

# Verify runner connects gitlab-runner verify

# Output should show: # Runner my-runner is alive ```

Step 3: Check Runner Tags

```bash # Check job tags in .gitlab-ci.yml cat .gitlab-ci.yml

build: stage: build tags: - docker - production

# Check runner tags # In GitLab UI: Admin > Runners > <runner> > Tags # Or API: curl --header "PRIVATE-TOKEN: <token>" "https://gitlab.com/api/v4/runners/<id>" | jq '.tag_list'

# Job will only run on runner with matching tags # If job needs "docker" tag, runner must have "docker" tag

# Add tags to runner via API: curl --request PUT --header "PRIVATE-TOKEN: <token>" \ "https://gitlab.com/api/v4/runners/<id>" \ --data "tag_list[]=docker&tag_list[]=production"

# Or in config.toml: [[runners]] name = "my-runner" tags = ["docker", "production"] ```

Step 4: Check Runner Concurrency

```bash # Check runner concurrent limit cat /etc/gitlab-runner/config.toml

concurrent = 4 # Max jobs this runner can run

# If concurrent = 4 and 4 jobs running, new jobs stay pending

# Increase concurrent limit concurrent = 10

# Restart runner systemctl restart gitlab-runner

# Check current running jobs curl --header "PRIVATE-TOKEN: <token>" \ "https://gitlab.com/api/v4/runners/<id>/jobs" | jq '.[] | select(.status=="running")'

# Check runner capacity # In GitLab UI: Admin > Runners > shows "Active jobs" ```

Step 5: Check Project Runner Assignment

```bash # Check project runners curl --header "PRIVATE-TOKEN: <token>" \ "https://gitlab.com/api/v4/projects/<project-id>/runners"

# If runner is "shared", it's available to all projects # If runner is "specific", it must be explicitly assigned

# Assign runner to project curl --request POST --header "PRIVATE-TOKEN: <token>" \ "https://gitlab.com/api/v4/projects/<project-id>/runners" \ --data "runner_id=<runner-id>"

# Enable shared runners for project # In GitLab UI: Project > Settings > CI/CD > Runners > Enable shared runners

# Check if project has paused runners curl --header "PRIVATE-TOKEN: <token>" \ "https://gitlab.com/api/v4/projects/<project-id>/runners" | jq '.[] | select(.paused==true)' ```

Step 6: Unpause Runner

```bash # Check if runner is paused curl --header "PRIVATE-TOKEN: <token>" \ "https://gitlab.com/api/v4/runners/<id>" | jq '.paused'

# Unpause runner via API curl --request PUT --header "PRIVATE-TOKEN: <token>" \ "https://gitlab.com/api/v4/runners/<id>" \ --data "paused=false"

# Or in GitLab UI: Admin > Runners > <runner> > Unpause

# Verify runner status curl --header "PRIVATE-TOKEN: <token>" \ "https://gitlab.com/api/v4/runners/<id>" | jq '.status' # Should be "online" ```

Step 7: Check Job Queue Limit

```bash # Check project CI/CD settings # In GitLab UI: Project > Settings > CI/CD > General pipelines

# Check "Maximum jobs per pipeline" limit # If exceeded, jobs stay pending

# Check "CI/CD minutes quota" for group # Group > Settings > CI/CD > Quota

# If minutes quota exceeded, jobs cannot run

# Check via API: curl --header "PRIVATE-TOKEN: <token>" \ "https://gitlab.com/api/v4/projects/<project-id>" | jq '.ci_cd_settings'

# For group quota: curl --header "PRIVATE-TOKEN: <token>" \ "https://gitlab.com/api/v4/groups/<group-id>" | jq '.shared_runners_minutes_limit' ```

Step 8: Register New Runner

```bash # If no runners available, register new one

# Get registration token # In GitLab UI: Project > Settings > CI/CD > Runners > New project runner # Or: Admin > Runners > New instance runner

# Register runner gitlab-runner register \ --non-interactive \ --url https://gitlab.com \ --registration-token <token> \ --name "my-runner" \ --tag-list "docker,production" \ --executor "docker" \ --docker-image "alpine:latest"

# Verify runner registered gitlab-runner list

# Or use docker: docker run -d --name gitlab-runner --restart always \ -v /srv/gitlab-runner/config:/etc/gitlab-runner \ -v /var/run/docker.sock:/var/run/docker.sock \ gitlab/gitlab-runner:latest register \ --non-interactive \ --url https://gitlab.com \ --registration-token <token> \ --executor docker \ --docker-image alpine:latest ```

Step 9: Check Runner Executor Issues

```bash # Runner executor may have issues

# Check executor type cat /etc/gitlab-runner/config.toml | grep executor

# Common executor types: shell, docker, docker+machine, kubernetes

# For docker executor, check docker daemon: docker ps docker info

# For kubernetes executor, check cluster: kubectl get pods -n gitlab-runner

# For shell executor, check shell access: ssh user@runner-host

# Check runner can pull docker images docker pull alpine:latest

# Check runner disk space df -h

# Check runner memory free -h ```

Step 10: Monitor Runner and Pipeline

```bash # Create monitoring script cat << 'EOF' > monitor_gitlab_ci.sh #!/bin/bash TOKEN="<private-token>" PROJECT="<project-id>" GITLAB="https://gitlab.com"

echo "=== Project Runners ===" curl -s --header "PRIVATE-TOKEN: $TOKEN" \ "$GITLAB/api/v4/projects/$PROJECT/runners" | jq '.[] | {name, status, active, paused}'

echo "" echo "=== Pending Jobs ===" curl -s --header "PRIVATE-TOKEN: $TOKEN" \ "$GITLAB/api/v4/projects/$PROJECT/pipelines?status=pending" | jq '.[] | {id, status, created_at}'

echo "" echo "=== Runner Status (local) ===" gitlab-runner status

echo "" echo "=== Runner Process ===" ps aux | grep gitlab-runner EOF

chmod +x monitor_gitlab_ci.sh

# Monitor in loop watch -n 30 ./monitor_gitlab_ci.sh

# Enable GitLab CI/CD monitoring # Admin > Settings > Metrics and profiling > Prometheus ```

GitLab CI Runner Checklist

Check	Command	Expected
Runner online	gitlab-runner status	Running
Runner paused	API .paused	false
Tags match	.gitlab-ci.yml tags	Runner has tags
Concurrent limit	config.toml concurrent	Not exceeded
Project assigned	API /runners	Runner listed
Quota available	Group quota	Not exceeded

Verification

```bash # After fixing runner issues

# 1. Check runner is online curl --header "PRIVATE-TOKEN: <token>" "https://gitlab.com/api/v4/runners/<id>" | jq '.status' // Should be "online"

# 2. Verify job starts # Check pipeline in GitLab UI // Jobs should change from "pending" to "running"

# 3. Monitor job execution gitlab-runner logs // Shows job being processed

# 4. Check pipeline completes curl --header "PRIVATE-TOKEN: <token>" "https://gitlab.com/api/v4/projects/<project-id>/pipelines/<pipeline-id>" | jq '.status' // Should be "success" or "failed" (not "pending")

# 5. Verify runner picks future jobs # Trigger new pipeline // Should be picked up immediately ```

Prevention

To prevent GitLab CI pipelines from getting stuck in pending state, implement these proactive measures:

1. Monitor Runner Capacity

yaml

groups:
- name: gitlab-runner
  rules:
  - alert: GitLabRunnerOffline
    expr: gitlab_runner_online == 0
    for: 1m
    labels:
      severity: critical
    annotations:
      summary: "GitLab Runner {{ $labels.runner }} is offline"

2. Configure Runner Autoscaling

```toml # In /etc/gitlab-runner/config.toml: concurrent = 10

[[runners]] executor = "docker+machine" limit = 20

[runners.machine] IdleCount = 2 IdleTime = 1800 MaxBuilds = 100 ```

3. Set Up Health Checks

```bash # Runner health monitoring cat << 'EOF' > /usr/local/bin/monitor_runners.sh #!/bin/bash curl -s -H "Private-Token: $TOKEN" \ "$GITLAB_URL/api/v4/runners/all" | jq -r '.[] | select(.status == "offline") | .id' EOF

chmod +x /usr/local/bin/monitor_runners.sh ```

4. Configure Resource Limits

```yaml # In .gitlab-ci.yml: default: retry: max: 2 when: - runner_system_failure - stuck_or_timeout_failure

job: timeout: 1h resource_group: production ```

Best Practices Checklist

[ ] Monitor runner capacity with alerts
[ ] Configure runner autoscaling
[ ] Set up runner health checks
[ ] Add resource limits to pipelines
[ ] Implement pipeline notifications
[ ] Document maintenance procedures
[ ] Test runner recovery procedures

[Fix GitLab CI Runner Not Picking Jobs](/articles/fix-gitlab-ci-runner-not-picking-jobs)
[Fix GitLab CI Pipeline Stuck](/articles/fix-gitlab-ci-pipeline-stuck)
[Fix GitLab Runner Timeout](/articles/fix-gitlab-runner-timeout)

[Technical troubleshooting: Fix Cicd Artifact Upload Failed Storage Issue in C](cicd-artifact-upload-failed-storage)
[Technical troubleshooting: Fix Cicd Code Quality Gate Failed Sonarqube Issue ](cicd-code-quality-gate-failed-sonarqube)
[Technical troubleshooting: Fix Cicd Deployment Failed Health Check Issue in C](cicd-deployment-failed-health-check)
[Technical troubleshooting: Fix Cicd Github Actions Workflow Queue Timeout in ](cicd-github-actions-workflow-queue-timeout)
[Technical troubleshooting: Fix Cicd Gitlab Runner Stuck Pending Issue in CI/C](cicd-gitlab-runner-stuck-pending)

Was this guide helpful?

Related search paths

People also search for

If the symptom is close but not identical, these search paths usually surface the right neighboring fixes faster than scrolling the full archive.

GitLab CI Pipeline Stuck in Pending GitLab CI Pipeline Stuck in Pending CI/CD GitLab CI Pipeline Stuck in Pending troubleshooting GitLab CI Pipeline Stuck in Pending fix Resolve GitLab CI pipelines stuck in pending state by checking runner availability, job limits, and concurrency settings CI/CD Resolve GitLab CI pipelines stuck in pending state by checking runner availability, job limits, and concurrency settings

Explore Related Topics

Browse Guides from Other Categories

Discover troubleshooting guides from related categories to expand your knowledge.

FAQ

CI/CD Troubleshooting FAQs

Common questions about troubleshooting and preventing similar issues

How do I know if this cicd-errors troubleshooting guide applies to my situation?

This guide is designed for cicd-errors issues. If you're experiencing similar symptoms described in the article, follow the step-by-step instructions. Start with the most common causes and work through the diagnostic process.

Is it safe to follow these cicd-errors troubleshooting steps?

Yes, all steps are designed to be safe and non-destructive. We recommend creating backups before making significant changes and testing each step before proceeding to the next.

How long does it typically take to resolve this type of cicd-errors issue?

Most cicd-errors issues can be resolved within 30 minutes to 2 hours, depending on the complexity and root cause. Follow the troubleshooting flow to identify and fix the problem efficiently.

How can I prevent this cicd-errors issue from happening again?

Regular maintenance, monitoring, and following best practices for cicd-errors configuration can help prevent recurrence. Consider implementing automated checks and alerts for early detection.

Written by

FixWikiHub Editorial Team

Our editorial team consists of experienced DevOps engineers, systems administrators, and cloud architects with hands-on experience in production environments across AWS, Azure, GCP, and on-premises infrastructure.

Every guide undergoes technical review for accuracy and is updated when software versions, commands, or best practices change.

Last updated: Apr 5, 2026

About our team

Important Notice

Disclaimer & Safety Guidelines

The troubleshooting steps in this guide are provided for educational and informational purposes. Before applying any changes to production systems:

Test in a staging environment first — Always verify commands and configurations in a non-production environment before deploying to live systems.
Create backups — Ensure you have current backups of databases, configurations, and critical files before making changes.
Understand the impact — Review how each step may affect your specific environment, dependencies, and users.
Consult official documentation — This guide supplements, but does not replace, official vendor documentation and best practices.

FixWikiHub is not responsible for any damages arising from the use of this content. See our Terms of Use for more information.

Resources

Official Documentation & Further Reading

For authoritative information, consult the official documentation for the technologies discussed in this guide. Our troubleshooting content supplements, but does not replace, vendor documentation.

AWS Documentation — Official Amazon Web Services guides and API references
Kubernetes Documentation — Official Kubernetes documentation
Nginx Documentation — Official Nginx web server documentation
Apache Documentation — Official Apache HTTP Server documentation
Docker Documentation — Official Docker container documentation

Fix GitLab CI Pipeline Stuck in Pending

Introduction

Symptoms

Common Causes

Step-by-Step Fix

Step 1: Check Runner Status

Step 2: Verify Runner is Online

Step 3: Check Runner Tags

Step 4: Check Runner Concurrency

Step 5: Check Project Runner Assignment

Step 6: Unpause Runner

Step 7: Check Job Queue Limit

Step 8: Register New Runner

Step 9: Check Runner Executor Issues

Step 10: Monitor Runner and Pipeline

GitLab CI Runner Checklist

Verification

Prevention

1. Monitor Runner Capacity

2. Configure Runner Autoscaling

3. Set Up Health Checks

4. Configure Resource Limits

Best Practices Checklist

People also search for

Browse Guides from Other Categories

WordPress

SSL

DNS

CI/CD Troubleshooting FAQs

FixWikiHub Editorial Team

Disclaimer & Safety Guidelines

Official Documentation & Further Reading

Fix GitLab CI Pipeline Stuck in Pending

Introduction

Symptoms

Common Causes

Step-by-Step Fix

Step 1: Check Runner Status

Step 2: Verify Runner is Online

Step 3: Check Runner Tags

Step 4: Check Runner Concurrency

Step 5: Check Project Runner Assignment

Step 6: Unpause Runner

Step 7: Check Job Queue Limit

Step 8: Register New Runner

Step 9: Check Runner Executor Issues

Step 10: Monitor Runner and Pipeline

GitLab CI Runner Checklist

Verification

Prevention

1. Monitor Runner Capacity

2. Configure Runner Autoscaling

3. Set Up Health Checks

4. Configure Resource Limits

Best Practices Checklist

Related Issues

Related Articles

People also search for

Share this guide

More CI/CD Troubleshooting Guides

Browse Guides from Other Categories

CI/CD Troubleshooting FAQs

FixWikiHub Editorial Team

Disclaimer & Safety Guidelines

Official Documentation & Further Reading