How to send automation test reports on Testrail dashboard and using GitHub Actions

4 min readAug 22, 2023

TestRail is a web-based test case management tool that allows software development and QA teams to organize, manage, and track software testing efforts. It provides a centralized platform for creating, documenting, executing, and reporting test cases, making it easier to manage the testing process and ensure the quality of software products.

Key features of TestRail, I will talk about today is “Integration with Automation”

Firstly, I need to set up on my Testrail website

Go to ADMINISTRATION and Click at “Site Setting” then Enable API then click Save setting same as the photo below,

2. I need to Customizations my Case feilds for my automation test results

Go to ADMINISTRATION
Click Customization
Tab CASE FIELDS
Click Add Field

5. After going to Add Field then Create information at Label, System Name and choose Type is String then Save

After you set up on Testrail website then let’s go to work on GitHub action

On your project to integration run CI test on GitHub, I created with .ymal file and you can follow up my antoher blog as I talked about How to setup CI GitHub action to run robot framework

Using the TestRail CLI in GitHub workflow files

Automated processes can be configured on GitHub by using workflow configuration files. These are YAML files. The easiest way to integrate TestRail with GitHub Actions is by configuring your workflow files to use the TestRail CLI to send automated test results to TestRail looks like this

name: Run-Test

on:
  push:
    branches:
      - master
  pull_request:
  workflow_dispatch:

jobs:
    Run-Test:
      runs-on: ubuntu-20.04
      steps:
        - name: Checkout repository
          uses: actions/checkout@v1
        - uses: actions/cache@v2
          id: cache
          with:
            path: ~/.cache/pip
            key: ${{ runner.os }}-pip-${{ hashFiles('**/requirement.txt') }}
        - name: Install pip dependencies
          run: |
            pip install --upgrade pip
            pip install -r requirement.txt
        - name: Run Test
          run: python -m robot --outputdir results testcases/functional_tests
        - name: Run Test and create report
          run: python -m robot -d reports -x junit-report.xml "testcases/functional_tests"
        - name: Run Robot Framework tests and generate report
          run: |
             echo "ROBOT_RC=$?" >> "$GITHUB_ENV" 
        - name: Publish test results
          uses: actions/upload-artifact@v2
          with:
            name: robot-test-results
            path: results
        - name: Setup Python
          if: always()
          uses: actions/setup-python@v4
          with:
            python-version: '3.10' 
        - name: Execute TestRail CLI to upload results
          if: always()
          run: |
            pip install trcli 
            trcli -y \
              -h "https://<your testrail>.testrail.io" \
              --project "<your project name>" \
              --username "<your Username>" \
              --password "<your Password>" \
              parse_junit \
              --title "Robot Framework Automated Tests" \
              -f "reports/junit-report.xml"\

Check Out Repository

- name: Checkout repository
  uses: actions/checkout@v1

2. Execute automated tests.

This can be accomplished either by using a GitHub Action or by inserting your custom execution script.

name: Execute Automated tests
  uses: ...

3. Install Python on the agent

The Python setup uses the actions/setup-python@v3 GitHub Action to configure Python on the agent. Because the TestRail CLI is a tool developed in Python, it is required that it is installed on the environment before you are able to install and run it.

- name: Setup Python
          if: always()
          uses: actions/setup-python@v4
          with:
            python-version: '3.10'

4. Install the TestRail CLI and upload test results

The TestRail CLI upload results step runs a custom shell script to install and call the TestRail CLI in order to parse the JUnit report generated by your test automation framework and send the results to TestRail.

Install Python Package we simply execute the command pip install trcli.

- name: Execute TestRail CLI to upload results
          if: always()
          run: |
            pip install trcli 
            trcli -y \
              -h "https://<your testrail>.testrail.io" \
              --project "<your project name>" \
              --username "<your Username>" \
              --password "<your Password>" \
              parse_junit \
              --title "Robot Framework Automated Tests" \
              -f "reports/junit-report.xml"\

5. Uupdate to your GitHub repository

Afer Github Action had run it looks like this