upskill-event-manager/docs/staging-workflow-plan.md

Staging Server Testing Workflow Plan

Status: Proposed Date: 2025-04-08 Scope: Transitioning development and testing workflow to utilize a Cloudways staging server.

Goal

Modify the development workflow to use the Cloudways staging server for End-to-End (E2E) testing and as the primary source for synchronizing data to the local development environment. Unit and Integration tests will continue to run locally within Docker against this synced data.

Overview & Key Decisions

This plan addresses the need to test against a more production-like environment (Cloudways staging) while accommodating SSH access restrictions on the staging server.

• Data Source: The Cloudways staging server will replace the production server as the source for data synced to the local development environment via a new script (bin/sync-staging.sh).
• Unit/Integration Testing: These tests will remain local, executed within the Docker environment using bin/run-tests.sh. This avoids the complexity and potential risks of setting up and running PHPUnit on the restricted Cloudways staging server. Tests will run against data synced from staging.
• E2E Testing: Playwright tests will be executed locally but will target the staging server URL (UPSKILL_STAGING_URL).
• Deployment to Staging: A new script (bin/deploy-plugin.sh) and configuration (deploy-config-staging.sh) will be created to handle deploying the plugin code (hvac-community-events) from the local machine to the staging server using the provided restricted SSH user (UPSKILL_STAGING_SSH_USER).
• Staging <-> Production Sync: Synchronization between the staging and production environments will continue to be managed manually via the Cloudways platform UI, as per current practice.

Workflow Diagram

graph TD
    subgraph Local Machine
        direction LR
        Dev[Developer] -- Edits Code --> CodeRepo[Plugin Code: hvac-community-events]
        Dev -- Runs --> SyncScript(./bin/sync-staging.sh)
        SyncScript -- Pulls Data --> StagingServer(Cloudways Staging)
        SyncScript -- Stores Data --> LocalBackup(./backups/)
        Dev -- Runs --> SetupScript(./bin/setup-from-backup.sh)
        SetupScript -- Uses --> LocalBackup
        SetupScript -- Sets up --> DockerEnv[Local Docker WP Env]

        Dev -- Runs --> DeployScript(./bin/deploy-plugin.sh --config deploy-config-staging.sh)
        DeployScript -- Pushes Code --> StagingServer

        Dev -- Runs --> TestScript(./bin/run-tests.sh)
        TestScript -- Unit/Integration --> DockerEnv
        TestScript -- E2E --> Playwright(Local Playwright)
        Playwright -- Tests --> StagingServer
    end

    subgraph Cloudways Staging Server
        StagingServer -- Contains --> StagingWP[Staging WP Install @ UPSKILL_STAGING_PATH]
        StagingWP -- Uses --> StagingDB[Staging Database]
        StagingServer -- Accessible via --> StagingURL[UPSKILL_STAGING_URL]
    end

    CodeRepo --> DeployScript

Detailed Plan

Phase 1: Adapt Data Synchronization

1. Create Staging Sync Script (bin/sync-staging.sh):

   • Duplicate wordpress-dev/bin/sync-production-fixed.sh to wordpress-dev/bin/sync-staging.sh.
   • Modify sync-staging.sh:
     • Source environment variables from .env and dev-env.conf (ensure dev-env.conf is sourced or its vars are in .env).
     • Replace all PROD_* variable references with the corresponding UPSKILL_STAGING_* variables (e.g., PROD_HOST -> UPSKILL_STAGING_IP, PROD_SSH_USER -> UPSKILL_STAGING_SSH_USER, PROD_PATH -> UPSKILL_STAGING_PATH, PROD_DB_NAME -> UPSKILL_STAGING_DB_NAME).
     • Update log messages, comments, and the generated manifest.txt content to refer to "Staging" instead of "Production".
     • Verify the rsync command correctly targets the staging path and user.
     • Verify the ssh command for wp db export executes correctly using the staging path and user.
   • Make the script executable: chmod +x wordpress-dev/bin/sync-staging.sh.

2. Update Documentation:

   • wordpress-dev/README.md:
     • Add a new section "Syncing Data from Staging" explaining the use of bin/sync-staging.sh.
     • Update the "Development Environment Setup from Backup" section to clarify that backups restored via setup-from-backup.sh now originate from staging (when sync-staging.sh is used).
   • memory-bank/activeContext.md / progress.md: Add entries reflecting the start and progress of this workflow transition task.

Phase 2: Configure E2E Tests for Staging

1. Update Playwright Configuration (wordpress-dev/tests/e2e/playwright.config.ts):

   • Modify the baseURL option. Instead of hardcoding, attempt to read UPSKILL_STAGING_URL from an environment variable (preferred) or directly use the URL: 'https://wordpress-974670-5399585.cloudwaysapps.com/'.
   • Ensure environment variables used for baseURL are available when running tests (e.g., load .env/dev-env.conf before running Playwright).
   • Review test files (e.g., personas.ts, specific tests) for hardcoded credentials (TEST_USER, TEST_PASSWORD from dev-env.conf) and ensure these users/passwords exist and are valid on the Cloudways staging environment.

2. Update Documentation:

   • wordpress-dev/testing.md:
     • Update the "E2E Tests" > "Environment" section to state tests run locally against the staging URL (UPSKILL_STAGING_URL).
     • Add a prerequisite note about ensuring test users/credentials are configured on the staging server.

Phase 3: Adapt Deployment Process (Code to Staging)

1. Create Staging Deployment Configuration (wordpress-dev/deploy-config-staging.sh):

   • Create this new file.
   • Populate with necessary variables for the deployment script, sourcing from dev-env.conf where possible:

     #!/bin/bash
     # Staging Deployment Configuration
     
     # Source shared config if needed
     # source dev-env.conf
     
     REMOTE_HOST="${UPSKILL_STAGING_IP}"
     REMOTE_USER="${UPSKILL_STAGING_SSH_USER}" # Restricted user
     REMOTE_PATH_BASE="/home/974670.cloudwaysapps.com/uberrxmprk/public_html" # Base path
     PLUGIN_SLUG="hvac-community-events"
     REMOTE_PLUGIN_PATH="${REMOTE_PATH_BASE}/wp-content/plugins/${PLUGIN_SLUG}/"
     LOCAL_PLUGIN_PATH="../wp-content/plugins/${PLUGIN_SLUG}/" # Adjust relative path as needed from bin/
     
     # Add other variables needed by deploy script (e.g., WP_CLI_PATH, cache flags)
     # WP_CLI_PATH="wp" # If needed and available for REMOTE_USER
     # PURGE_BREEZE_CACHE=false # Example
     

2. Create Deployment Script (wordpress-dev/bin/deploy-plugin.sh):

   • Create this new script.
   • Implement the following logic:
     • Accept a --config argument pointing to a configuration file (like deploy-config-staging.sh).
     • Source the specified configuration file.
     • Use rsync -avz --delete (or similar) to synchronize the LOCAL_PLUGIN_PATH to the REMOTE_PLUGIN_PATH via SSH using REMOTE_USER@REMOTE_HOST. Handle SSH key authentication or password prompting securely. Exclude files like .git, node_modules, etc.
     • Optional: If needed and possible with the restricted user, add steps to clear caches via SSH commands (e.g., ssh $REMOTE_USER@$REMOTE_HOST "cd $REMOTE_PATH_BASE && wp cache flush" if WP-CLI is usable).
     • Include error checking and informative output messages.
   • Make the script executable: chmod +x wordpress-dev/bin/deploy-plugin.sh.

3. Update Documentation:

   • docs/deployment.md:
     • Add a "Deploying to Staging" section.
     • Document the deploy-config-staging.sh file.
     • Explain how to use the new bin/deploy-plugin.sh script (e.g., ./bin/deploy-plugin.sh --config deploy-config-staging.sh).
   • memory-bank/decisionLog.md: Record the decision to keep Unit/Integration tests local due to SSH restrictions on the staging server.

Phase 4: Refine Local Testing Workflow

1. Update Test Runner Script (wordpress-dev/bin/run-tests.sh):

   • No functional changes required for test execution logic.
   • Optional: Add a check at the beginning to see how old the backups/latest/manifest.txt is and warn the user if it's old, suggesting they run sync-staging.sh.

2. Update Documentation:

   • wordpress-dev/testing.md:
     • Clearly reiterate that Unit/Integration tests run locally in Docker.
     • Add a prominent section emphasizing the need to run bin/sync-staging.sh regularly before local testing to ensure data relevance.
     • Review and update the troubleshooting section for PHPUnit issues, ensuring it reflects the local Docker execution context.

Prerequisites & Assumptions

• SSH access to the Cloudways staging server is configured for the user running the scripts (key-based authentication preferred).
• The UPSKILL_STAGING_* variables in dev-env.conf are correct.
• rsync, ssh, scp, and wp-cli (for remote execution via SSH) are available on the local machine and potentially on the staging server within the restricted user's environment (specifically wp-cli for DB export).
• The local Docker environment setup (docker-compose.yml, etc.) remains functional.

Next Steps

• Implement the script changes (sync-staging.sh, deploy-plugin.sh).
• Create the configuration file (deploy-config-staging.sh).
• Update the Playwright configuration.
• Update all relevant documentation files (README.md, testing.md, deployment.md, Memory Bank).
• Test the new workflow thoroughly.