Skip to content

ci: Add automated test workflow and manual test environment - #9

Open
ohto-ai with Copilot wants to merge 4 commits into
masterfrom
copilot/add-ci-test-workflow
Open

ohto-ai with Copilot wants to merge 4 commits into
masterfrom
copilot/add-ci-test-workflow

Conversation

Copilot AI commented Apr 3, 2026 •

Copy link
Copy Markdown
Contributor

Adds two GitHub Actions workflows to enable automated test execution and on-demand human testing of the running application.

test.yaml — Test CI (workflow_dispatch)

  • Triggered manually via workflow_dispatch (avoids duplicate push/PR runs alongside ci.yaml)
  • Matrix over ubuntu-22.04, ubuntu-latest, and windows-latest
  • On Linux: installs openssh-server (with apt-get update), enables password auth, and creates a test user with a cryptographically random password (openssl rand -hex 8)
  • On Windows: installs the built-in OpenSSH.Server capability, starts sshd, and creates a local test user with a cryptographically secure password (RandomNumberGenerator.Fill)
  • Exports TEST_SSH_HOST/PORT/USER/PASS env vars so that ssh_connection.test.cpp integration tests have a real SSH target (they skip gracefully when TEST_SSH_HOST is unset)
  • Builds webssh_cpp_tests (Debug) and runs ctest --output-on-failure
  • Job timeout-minutes: 60

manual_test.yaml — Manual test environment (workflow_dispatch)

  • Accepts a password input for the test SSH account (webssh_tester); leave it blank to auto-generate a random password — the value is passed via an environment variable to prevent shell injection
  • Accepts a duration input (minutes, default 30, max 60) for how long to keep the environment alive
  • Builds webssh_cpp (Release) and starts it on port 8080
  • Provides interactive access via action-tmate (detached, restricted to the workflow triggerer via limit-access-to-actor: true) — no public tunnel; testers use SSH port-forwarding from the tmate session to reach http://localhost:8080/
  • Prints SSH credentials to enter in the webssh_cpp web terminal (127.0.0.1:22)
  • Job timeout-minutes: 65 enforces a hard ceiling

Both jobs use permissions: contents: read for least-privilege GITHUB_TOKEN access.

Copilot AI linked an issue Apr 3, 2026 that may be closed by this pull request
Copilot AI changed the title [WIP] Add CI workflow for testing existing code ci: Add automated test workflow and manual test environment Apr 3, 2026
Copilot AI requested a review from ohto-ai April 3, 2026 01:18
@ohto-ai
ohto-ai requested a review from Copilot April 3, 2026 03:23

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds GitHub Actions workflows to run the project’s test suite in CI (including SSH integration tests) and to spin up an on-demand manual testing environment for the running webssh_cpp app.

Changes:

  • Added test.yaml to build and execute webssh_cpp_tests on Ubuntu runners, provisioning a local SSH server/user for integration tests.
  • Added manual_test.yaml (workflow_dispatch) to build and run webssh_cpp, then expose it via a public tunnel for human testing for a configurable duration.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.

File Description
.github/workflows/test.yaml New CI workflow to provision local SSH and run ctest for webssh_cpp_tests on Ubuntu matrix.
.github/workflows/manual_test.yaml New manually triggered workflow to run webssh_cpp and expose it over a public tunnel for interactive testing.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/test.yaml Outdated
Comment on lines +4 to +19
push:
paths:
- '.github/workflows/test.yaml'
- 'src/**'
- 'inc/**'
- 'test/**'
- 'CMakeLists.txt'
- '*.cmake'
pull_request:
paths:
- '.github/workflows/test.yaml'
- 'src/**'
- 'inc/**'
- 'test/**'
- 'CMakeLists.txt'
- '*.cmake'

Copilot AI Apr 3, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This workflow overlaps with the existing .github/workflows/ci.yaml triggers (same push/PR path filters) and OS matrix, so pushes/PRs will run two near-identical CI jobs and increase total CI time. Consider consolidating by re-enabling/adding the test step in ci.yaml instead of introducing a separate workflow, or adjusting triggers so only one of them runs for a given change set.

Suggested change
push:
paths:
- '.github/workflows/test.yaml'
- 'src/**'
- 'inc/**'
- 'test/**'
- 'CMakeLists.txt'
- '*.cmake'
pull_request:
paths:
- '.github/workflows/test.yaml'
- 'src/**'
- 'inc/**'
- 'test/**'
- 'CMakeLists.txt'
- '*.cmake'
workflow_dispatch:

Copilot uses AI. Check for mistakes.

- name: Set up SSH server for integration tests
shell: bash
run: |

Copilot AI Apr 3, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

apt-get install is run without an apt-get update, which can intermittently fail on GitHub-hosted runners when package indexes are stale. Adding an update step (or using apt-get update immediately before install) will make the workflow more reliable.

Suggested change
run: |
run: |
sudo apt-get update

Copilot uses AI. Check for mistakes.
Comment thread .github/workflows/test.yaml Outdated
sudo systemctl start ssh
# Create a temporary test user with a random password
TEST_USER=webssh_testuser
TEST_PASS=$(openssl rand -base64 12 | tr -dc 'A-Za-z0-9' | head -c 16)

Copilot AI Apr 3, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Password generation can produce fewer than 16 characters after the tr -dc filter (or even an empty string), which would make SSH auth flaky. Use a generation method that guarantees length/charset (e.g., openssl rand -hex ... or loop until the desired length is reached).

Suggested change
TEST_PASS=$(openssl rand -base64 12 | tr -dc 'A-Za-z0-9' | head -c 16)
TEST_PASS=$(openssl rand -hex 8)

Copilot uses AI. Check for mistakes.
- name: Set up SSH server and test account
id: setup
shell: bash
run: |

Copilot AI Apr 3, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

apt-get install is run without an apt-get update, which can intermittently fail on GitHub-hosted runners when package indexes are stale. Adding an update step (or using apt-get update immediately before install) will make the workflow more reliable.

Suggested change
run: |
run: |
sudo apt-get update

Copilot uses AI. Check for mistakes.
Comment thread .github/workflows/manual_test.yaml Outdated
Comment on lines +56 to +94
- name: Set up public tunnel
shell: bash
run: |
# Start a reverse SSH tunnel via localhost.run (no account required)
ssh -o StrictHostKeyChecking=no \
-o ServerAliveInterval=30 \
-o ServerAliveCountMax=10 \
-R 80:localhost:8080 \
nokey@localhost.run 2>&1 | tee /tmp/tunnel.log &
echo "TUNNEL_PID=$!" >> "$GITHUB_ENV"
# Wait for the tunnel URL to appear in the log
for i in $(seq 1 15); do
TUNNEL_URL=$(grep -oP 'https?://[^\s]+' /tmp/tunnel.log | head -1)
if [ -n "$TUNNEL_URL" ]; then
break
fi
sleep 2
done
echo "TUNNEL_URL=${TUNNEL_URL:-http://localhost:8080 (tunnel unavailable)}" >> "$GITHUB_ENV"

- name: Print test environment info
shell: bash
run: |
echo "============================================"
echo " WebSSH Manual Test Environment"
echo "============================================"
echo ""
echo " 1. Open the web interface in your browser:"
echo " $TUNNEL_URL"
echo ""
echo " 2. In the web terminal, connect with these SSH credentials:"
echo " Host : 127.0.0.1"
echo " Port : 22"
echo " Username : ${{ steps.setup.outputs.test_user }}"
echo " Password : ${{ steps.setup.outputs.test_pass }}"
echo ""
echo " Environment will stay up for ${{ inputs.duration }} minutes."
echo " Cancel this workflow run to shut it down early."
echo "============================================"

Copilot AI Apr 3, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This workflow exposes a publicly reachable tunnel to the runner and prints the tunnel URL plus SSH credentials into the build logs. In a public repo, that effectively creates an unauthenticated remote shell environment that anyone who sees the logs can use while the job is running, which is a significant security and abuse risk. Consider replacing the public tunnel with an authenticated/approved access mechanism (e.g., GitHub environments with required reviewers, or a GitHub-provided debugging channel like action-tmate), and avoid emitting reusable credentials into logs.

Copilot uses AI. Check for mistakes.
Comment on lines +96 to +105
- name: Wait for manual testing
shell: bash
run: |
DURATION_INPUT="${{ inputs.duration }}"
# Validate that the input is a positive integer; fall back to 30 if not.
if ! [[ "$DURATION_INPUT" =~ ^[0-9]+$ ]] || [ "$DURATION_INPUT" -le 0 ]; then
DURATION_INPUT=30
fi
DURATION_SECONDS=$(( DURATION_INPUT * 60 ))
echo "Keeping environment alive for ${DURATION_INPUT} minute(s)..."

Copilot AI Apr 3, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The duration input is only validated as a positive integer and has no upper bound, so a dispatch can keep a runner allocated up to the GitHub job timeout (hours), increasing cost/queue pressure. Add a reasonable maximum duration (and/or set timeout-minutes on the job) and clamp the input to that max.

Copilot uses AI. Check for mistakes.
@ohto-ai

ohto-ai commented Apr 3, 2026

Copy link
Copy Markdown
Owner

@copilot apply changes based on the comments in this thread, and add windows test

@ohto-ai
ohto-ai marked this pull request as ready for review April 3, 2026 05:42

Copilot AI commented Apr 3, 2026

Copy link
Copy Markdown
Contributor Author

@copilot apply changes based on the comments in this thread, and add windows test

Applied all review feedback and added Windows test support in aa58e62:

test.yaml:

  • Changed trigger to workflow_dispatch only — eliminates the duplicate push/PR overlap with ci.yaml
  • Added apt-get update before apt-get install for reliability
  • Password generation changed to openssl rand -hex 8 (guaranteed 16 hex chars, no filtering)
  • Added windows-latest to the OS matrix with a dedicated PowerShell step: installs the built-in OpenSSH.Server capability, starts sshd, creates a local test user with a cryptographically secure password (RandomNumberGenerator.Fill), and exports the same TEST_SSH_* env vars as the Linux step
  • Added timeout-minutes: 60 to the job
  • Build-directory path normalized with sed for Windows compatibility (matching the pattern in ci.yaml)

manual_test.yaml:

  • Added apt-get update before apt-get install
  • Replaced the public localhost.run tunnel with action-tmate running in detached: true mode with limit-access-to-actor: true — the tmate session is only accessible to the workflow triggerer; connection info is used for SSH port-forwarding to reach localhost:8080
  • Added timeout-minutes: 65 on the job as a hard ceiling
  • Duration input now capped at 60 minutes in the validation block

@ohto-ai

ohto-ai commented Apr 3, 2026

Copy link
Copy Markdown
Owner

@copilot
人工测试我需要输入密码, 请将账户密码作为手动测试的输入项, 如果可以提供默认值就更好了

Copilot AI commented Apr 3, 2026

Copy link
Copy Markdown
Contributor Author

@copilot
人工测试我需要输入密码, 请将账户密码作为手动测试的输入项, 如果可以提供默认值就更好了

Done in 313dade. Added a password input to the workflow_dispatch trigger. Leave it blank to auto-generate a random password, or enter your own. The value is passed to the shell via an environment variable (not inline interpolation) to avoid injection issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI添加Test

3 participants