How to Run Git Workflows on the Same Runner: A Guide

Coordinating GitHub Workflows on Shared Runners

It might be difficult to manage several processes in GitHub Actions, particularly if you want them to execute on the same self-hosted runner. It might be challenging to make sure that distinct YAML files for distinct workflows, such as codeql.yml and snyk-zap.yml, run on the same runner from a certain group.

In order to prevent conflicts with other workflows, the same runner will be used for both workflows without being specifically named. This tutorial will examine potential approaches to effectively accomplish this synchronization while preserving the procedures in different YAML files.

Command	Description
jq	A portable and adaptable command-line JSON processor that is employed in Bash scripts to parse JSON output.
head -n 1	Produces the first line of the result, which is then used to choose the first runner ID that is available.
curl	A Bash script use this command-line utility to communicate with GitHub's API and send data via URLs.
os.getenv()	Obtains Python environment variables, which are needed to obtain the repository name and GitHub token.
requests.get()	Enables the Python script to retrieve available runners from GitHub's API by sending a GET request to a given URL.
os.path.exists()	Determines whether the runner ID file is already present in the Python script by checking to see if a given path exists.
with open()	The runner ID can be read from and written to a file using the Python context manager for file operations.

Managing Workflows Using Common Runners

The included scripts control the assignment of runners for GitHub workflows. First, the Bash script determines whether a runner ID has previously been saved in a temporary file. If not, it queries GitHub's API using to find available runners, parses the JSON result using , chooses the first idle runner, and saves its ID. The Python script obtains comparable functionality by retrieving runner data from GitHub's API using the method. Next, using os.path.exists(), the script determines whether a runner ID has already been saved and, if not, saves it.

By using the saved runner ID, both scripts guarantee that a runner will be utilized for subsequent jobs once it has been assigned. The Python script uses to safely handle file operations, and to retrieve environment variables for the GitHub token and repository. These scripts are flexible and effective in managing workflow execution since they assist coordinate many workflows and make sure they all execute on the same runner without hardcoding the runner name.

# A script to manage runner assignment
#!/bin/bash

# Check if a runner is already assigned
RUNNER_ID=$(cat /tmp/runner_id)

if [ -z "$RUNNER_ID" ]; then
  # No runner assigned yet, pick one and save its ID
  RUNNER_ID=$(curl -s -H "Authorization: token $GITHUB_TOKEN" \
               https://api.github.com/repos/$GITHUB_REPOSITORY/actions/runners |
               jq -r '.runners[] | select(.status=="online" and .busy==false) | .id' | head -n 1)
  echo $RUNNER_ID > /tmp/runner_id
fi

echo "Using runner $RUNNER_ID"

# Proceed with the workflow using the assigned runner

import requests
import os

# GitHub API token and repository
GITHUB_TOKEN = os.getenv('GITHUB_TOKEN')
REPO = os.getenv('GITHUB_REPOSITORY')

# Function to get an available runner
def get_runner():
    url = f"https://api.github.com/repos/{REPO}/actions/runners"
    headers = {'Authorization': f'token {GITHUB_TOKEN}'}
    response = requests.get(url, headers=headers)
    runners = response.json()['runners']
    for runner in runners:
        if runner['status'] == 'online' and not runner['busy']:
            return runner['id']
    return None

# Check if a runner is already assigned
if not os.path.exists('/tmp/runner_id'):
    runner_id = get_runner()
    with open('/tmp/runner_id', 'w') as f:
        f.write(str(runner_id))
else:
    with open('/tmp/runner_id', 'r') as f:
        runner_id = f.read()

print(f"Using runner {runner_id}")

Effective Runner Control in GitHub Tasks

A crucial factor to take into account in situations where many workflows must operate on a single self-hosted runner is guaranteeing runner availability and reducing conflicts. As seen in the earlier scripts, using a shared runner method guarantees that once a runner is allocated to a work, they are used for subsequent jobs. This can be especially helpful in intricate CI/CD pipelines when it's critical to preserve state or make use of cached resources.

Maximizing the use of runners is an additional factor to think about. Organizations can more effectively manage their resources by dynamically assigning and choosing runners according to their availability. By putting such strategies into practice, processes become more efficient and spend less time waiting for a runner to become available. This method is adaptable to different CI/CD systems and tools, making it a flexible answer for a range of automation requirements.

1. How can I make sure a particular runner is used every time?
2. You can define the specific runner name or runner group in your YAML file by using the key.
3. Is it possible to dynamically assign runners to tasks?
4. Yes, by allocating them dynamically and querying available runners with scripts.
5. In a crowded setting, how should I resolve disagreements between runners?
6. To efficiently manage runner allocation, prioritize workflows or put in place a queue system.
7. What occurs if there aren't any runners available?
8. Workflows will wait in line until a runner is free. To cut down on wait times, maximize runner utilization.
9. Are these scripts compatible with any other CI/CD platforms?
10. Absolutely, other platforms having runner management API access can modify the logic.
11. How can I keep my state intact in between workflows?
12. Make sure related jobs are completed by the same runner, and when feasible, use caching techniques.
13. Which permissions do these scripts require?
14. Make sure the scopes and , among others, are present on your GitHub token.
15. Is it possible to run more than one workflow at once on a single runner?
16. Usually not. One task is completed at a time by each runner. Employ several runners to achieve concurrency.
17. How can I keep an eye on my runners' progress and usage?
18. Track runner activity and performance using external services or the built-in monitoring tools in GitHub.

Conclusion:

For consistency and efficiency, managing GitHub processes to execute on the same self-hosted runner is essential. The Python and Bash scripts that have been discussed offer a strong answer by dynamically allocating runners and guaranteeing that the same runner is used for all subsequent jobs. This method works well for complicated CI/CD pipelines since it reduces conflicts and maximizes resource usage. By putting these strategies into practice, businesses can improve productivity and preserve a smooth development process by streamlining workflow execution and cutting wait times.