As modern DevOps practices evolve to meet the needs of dynamic software development lifecycles, Continuous Integration and Continuous Deployment (CI/CD) systems must scale efficiently. Jenkins, one of the most popular CI tools, can leverage Kubernetes to dynamically provision build agents, known as Kubernetes Executors, allowing teams to scale Jenkins pipelines on demand. This article dives into how to set up Jenkins with Kubernetes Executors, and how it improves scalability and resource efficiency. 

What is Jenkins? 

Jenkins is an open-source automation server widely used to implement continuous integration and continuous delivery (CI/CD) for software projects. It automates the building, testing, and deploying of applications and integrates with nearly every tool in the DevOps. 

What are Kubernetes Executors? 

A Kubernetes Executor in Jenkins refers to a dynamically provisioned Jenkins agent (or slave) that runs in a Kubernetes pod. These agents are ephemeral—they spin up when a job starts and are terminated after job completion. This behavior is ideal for reducing resource usage and ensuring isolated, reproducible builds. 

Architecture Overview 

Jenkins-Kubernetes Integration Architecture 

The Jenkins-Kubernetes integration follows a controller-agent model with the following components: 

Jenkins Controller (Master) 

• Location Options: Can be deployed either inside the Kubernetes cluster as a Deployment/StatefulSet or outside the cluster on a separate server 

• Responsibilities: Handles web UI, job scheduling, API interactions, and orchestrates the entire CI/CD process 

• Communication: Uses the Kubernetes API to communicate with the cluster for agent provisioning 

Kubernetes Cluster 

• Role: Provides the infrastructure to launch ephemeral pods (agents) on-demand 

• Components: Worker nodes where Jenkins agent pods are dynamically created and destroyed 

• Scaling: Automatically scales based on workload demands and available resources 

Jenkins Kubernetes Plugin 

• Function: Acts as the bridge between Jenkins Controller and Kubernetes API 

• Responsibilities: Handles pod creation, lifecycle management, and termination of agents 

• Communication: Makes API calls to create, monitor, and delete agent pods 

Architecture Diagram Flow 

[Jenkins Controller]  

        ↓ (API Calls) 

[Kubernetes API Server] 

        ↓ (Pod Creation) 

[Worker Nodes] 

        ↓ (Contains) 

[Jenkins Agent Pods] → [Build Execution] → [Pod Termination] 

Jenkins Infrastructure Setup Scenarios 

Scenario 1: Jenkins Controller Inside Kubernetes Cluster 

Recommended for: Cloud-native environments, better resource utilization, simplified networking 

Benefits: 

• Faster communication between controller and agents 

• Simplified service discovery and networking 

• Better resource sharing and utilization 

• Native Kubernetes health checks and recovery 

Setup Requirements: 

• Jenkins deployed as StatefulSet with persistent storage 

• Service account with necessary RBAC permissions 

• Same namespace for controller and agents (recommended) 

Scenario 2: Jenkins Controller Outside Kubernetes Cluster 

Recommended for: Hybrid environments, existing Jenkins installations, compliance requirements 

Benefits: 

• Easier migration from existing Jenkins setups 

• Separation of concerns between CI/CD orchestration and build execution 

• Reduced cluster resource consumption for controller operations 

Setup Requirements: 

• External network access to Kubernetes API 

• Kubernetes service account tokens for authentication 

• Proper firewall rules for communication 

Scaling Logic and Agent Provisioning 

How Jenkins Decides When to Scale 

The Jenkins Kubernetes plugin implements intelligent scaling based on several factors: 

Scale-Out Triggers: 

1. Job Queue Length: When build jobs are queued and no available agents exist 

2. Agent Availability: When all existing agents are busy with running builds 

3. Pod Template Matching: When a job requires specific capabilities defined in pod templates 

4. Resource Constraints: Based on defined resource limits and cluster capacity 

Scale-In Triggers: 

1. Build Completion: Agents are terminated immediately after job completion (default behavior) 

2. Idle Timeout: Configurable timeout for agent termination when idle 

3. Pod Retention Policies: “Always”, “On Failure”, or “Never” retention strategies 

Dynamic Agent Provisioning Process 

Step 1: Job Trigger 

• Jenkins receives a build request through webhook, schedule, or manual trigger 

Step 2: Agent Requirement Analysis 

• Jenkins analyzes the job requirements (labels, resources, tools needed) 

• Matches requirements against available pod templates 

Step 3: API Call to Kubernetes 

• Jenkins Kubernetes plugin makes API call to create pod with specified template 

• Pod includes environment variables with Jenkins server details and secrets 

Step 4: Agent Registration 

• Agent pod starts and connects back to Jenkins using JNLP protocol 

• Agent registers itself and becomes available for job execution 

Step 5: Build Execution 

• Jenkins assigns the job to the newly created agent 

• All pipeline steps execute within the isolated pod environment 

Step 6: Cleanup 

• Upon job completion, pod is automatically terminated (unless retention policy specifies otherwise) 

• Resources are freed for other workloads 

Components Involved 

Jenkins Controller (Master) 

• Runs inside Kubernetes as a Deployment or StatefulSet 

• Handles web UI, job scheduling, and API interactions 

• Requires persistent storage for job history and configurations 

Kubernetes Plugin for Jenkins 

• Enables Jenkins to dynamically create pods as build agents 

• Communicates with the Kubernetes API to launch and manage pods 

• Handles pod lifecycle management and cleanup 

Pod Templates 

• Define what the build agent should look like (image, command, resource limits) 

• Can include pre-installed tools like Maven, Node.js, Docker, etc. 

• Support for multiple containers per pod for complex build requirements 

Step-by-Step Setup Guide 

1. Install Jenkins on Kubernetes 

• Use Helm for a quick install: 

2. Install Kubernetes Plugin in Jenkins 

• Go to: Manage Jenkins → Plugin Manager 

• Install “Kubernetes” Plugin 

3. Configure Kubernetes Cloud in Jenkins 

• Go to Manage Jenkins → Configure System → Cloud 

• Add a new cloud of type Kubernetes 

• Configure: 

1. Kubernetes URL (leave blank if Jenkins is in-cluster) 

2. Namespace: same as Jenkins deployment 

3. Credentials: use Jenkins ServiceAccount with necessary RBAC 

4. Add Pod Templates (e.g., Maven builder) 

5. Pod Retention Strategy: “Always”, “On failure”, or “Never” 

4. Define Pod Templates 

Define Pod Templates either in the Jenkins UI or in code using YAML or Groovy. For example, in YAML: 

5. Create a Pipeline Using a Kubernetes Agent 

• Example Jenkinsfile:-  

Best Practices 

• Use Labels: Label pod templates to distinguish environments (e.g., nodejs-agent, docker-agent). 

• Limit Pod Lifetimes: Always use ephemeral agents unless persistent state is needed. 

• Use RBAC: Restrict Jenkins’ permissions using Kubernetes RBAC roles. 

• Resource Requests & Limits: Define CPU and memory limits to prevent over-provisioning. 

• Secrets Management: Use Kubernetes Secrets for credentials or tools in the build process. 

Metrics Monitoring & Autoscaling  

• Integrate Jenkins with monitoring tools like Prometheus and Grafana to visualize: 

1. Build times 

2. Queue wait times 

3. Executor utilization 

4. Pod provisioning delays 

• Set up Horizontal Pod Autoscaler (HPA) for Jenkins master if needed. 

• Use Cluster Autoscaler for node-level elasticity based on pod demand. 

Conclusion 

Scaling Jenkins pipelines with Kubernetes Executors offers dynamic scalability, resource efficiency, and operational flexibility. It enables teams to meet modern DevOps demands with containerized, ephemeral build agents tailored for each task. By integrating Jenkins with Kubernetes, organizations can unlock powerful automation workflows that are resilient, scalable, and optimized for cloud-native applications.