Batch Command and Recipe Reference

Introduction

The batch submit command allows you to define and execute complex testing scenarios through YAML files called recipes. These recipes consist of steps, each containing specific actions, options, and arguments that correspond to coordinator commands.

Recipe Structure

Basic Format

version: 1
steps:
  - description: Brief description of what this step does
    action: command.subcommand
    args:
      argName: value
    options:
      optionName: value

Keywords Reference

Top-Level Keywords

version: (Required) Recipe format version. Currently must be 1.
targets: (Optional) Define custom targets for use within the recipe.
steps: (Required) Array of steps to execute in order.

Target Definition

When using targets, you can define reusable target configurations:

targets:
  - name: target_name          # Unique name for this target
    filter: regex_pattern      # Regular expression to match ports
    tags:                      # Existing tags to include
      - tag1
      - tag2

Step Definition

Each step must contain:

description: (Required) Human-readable description of the step
action: (Required) The coordinator command to execute (format: command.subcommand)
args: (Optional) Object containing command arguments
options: (Optional) Object containing command options
allowFailure: (Optional) Boolean. If true, recipe continues even if step fails

Available Actions

The batch system supports most coordinator commands. Here are the commonly used actions:

Configuration Commands

# Set username
- action: config.username
  options:
    username: "User Name"

# Load configuration from file
- action: config.load
  options:
    input: "config-file.yaml"
    force: true           # Optional: overwrite existing data

# Dump current configuration
- action: config.dump
  options:
    output: "backup.yaml"

Host Management Commands

# Add a host
- action: host.add
  args:
    host: "192.168.1.100"
  options:
    u: username           # Username
    p: password           # Password
    t: tag_name           # Tag to assign
    frameworkFile: "framework.tgz"  # Optional: framework file
    rpc: true             # Optional: install as RPC service

# Update host framework
- action: host.update
  args:
    target: "host_or_tag"
  options:
    v: "version"          # Version (or "edge")
    frameworkFile: "path/to/framework.tgz"

# Reserve hosts
- action: host.reserve
  options:
    limit: 2              # Number of hosts to reserve
    tag: "target_tag"     # Tag to assign to reserved hosts

# List hosts
- action: host.list

# Remove host
- action: host.remove
  args:
    target: "host_or_tag"

Port Management Commands

# List ports
- action: port.list
  options:
    i: true               # Include detailed info
    t: "tag_filter"       # Filter by tag

# Unreserve ports
- action: port.unreserve
  args:
    target: "target_name"

# Add port tags
- action: port.tag.add
  args:
    target: "port_name"
    tags: "new_tag"

# Remove port tags
- action: port.tag.remove
  args:
    target: "port_name"
    tags: "tag_to_remove"

Test Commands

# Run tests
- action: test.run
  args:
    target: "port_or_tag"
    script: "path/to/test.js"
  options:
    period: 30            # Monitoring period
    wait: true            # Wait for completion

# Check test status
- action: test.status
  args:
    target: "port_or_tag"
  options:
    failIfNotZero: true   # Fail if any tests failed
    f:                    # Filter options
      - "status=FAILED"

Log Commands

# Read log files
- action: log.read
  args:
    target: "port_name"
    dirName: "task-123"
    fileName: "default.log"
  options:
    n: 10                 # Number of lines to read

Variable Substitution

Recipes support environment variable and argument substitution using ${VARIABLE_NAME} syntax.

Setting Variables

Use the batch submit command with -e for environment variables and -a for custom arguments:

# Environment variables
coordinator batch submit recipe.yaml -e HOST -e USERNAME -e PASSWORD

# Custom arguments  
coordinator batch submit recipe.yaml -a TARGET=tester -a LIMIT=2

# Combined
coordinator batch submit recipe.yaml \
  -e HOST -e USERNAME -e PASSWORD \
  -a TARGET=tester -a SCRIPT_PATH=test.js

Using Variables in Recipes

version: 1
steps:
  - description: Setting username to ${USERNAME}
    action: config.username
    options:
      username: ${USERNAME}
      
  - description: Adding host ${HOST}
    action: host.add
    args:
      host: ${HOST}
    options:
      u: ${USERNAME}
      p: ${PASSWORD}
      t: ${TARGET}

Error Handling

Allow Failure

Use allowFailure: true to continue execution even if a step fails:

steps:
  - description: This step might fail but recipe continues
    action: host.add
    args:
      host: "192.168.1.100"
    options:
      u: "user"
      p: "pass"
    allowFailure: true
    
  - description: This step runs regardless of previous step result
    action: host.list

Conditional Execution Patterns

Since recipes don't support when conditions, use separate recipe files for different scenarios:

For existing hosts (functional_test.yaml):

version: 1
steps:
  - description: Load existing configuration
    action: config.load
    options:
      input: ${CONFIG_STATE_FILE}
  # ... rest of steps

For new VMs (functional_test_vm.yaml):

version: 1
steps:
  - description: Add new host
    action: host.add
    args:
      host: ${REGRESSION_HOSTS}
    options:
      u: ${REGRESSION_USERNAME}
      p: ${REGRESSION_PASSWORD}
  # ... rest of steps

Complete Examples

Example 1: Simple Test Execution

version: 1
steps:
  - description: Set coordinator username
    action: config.username
    options:
      username: "CI Bot"
      
  - description: Run functional test
    action: test.run
    args:
      target: "test-port-01"
      script: "test/functional_test.js"
    options:
      wait: true
      period: 30
      
  - description: Check test results
    action: test.status
    args:
      target: "test-port-01"
    options:
      failIfNotZero: true

Example 2: Multi-Host Test with Cleanup

version: 1
steps:
  - description: Load host configuration
    action: config.load
    options:
      input: "hosts.yaml"
      
  - description: Reserve test hosts
    action: host.reserve
    options:
      limit: 3
      tag: "batch-test"
      
  - description: Update framework on reserved hosts
    action: host.update
    args:
      target: "batch-test"
    options:
      v: "edge"
      
  - description: Run distributed tests
    action: test.run
    args:
      target: "batch-test"
      script: "test/distributed_test.js"
    options:
      wait: true
      period: 60
      
  - description: Collect test results
    action: test.status
    args:
      target: "batch-test"
    options:
      failIfNotZero: true
      
  - description: Cleanup - unreserve hosts
    action: port.unreserve
    args:
      target: "batch-test"
    allowFailure: true

Example 3: Using Targets

version: 1
targets:
  - name: primary_ports
    filter: "host-[0-9]+-p0[0-3]"
    tags:
      - "primary-cluster"
  - name: secondary_ports  
    filter: "host-[0-9]+-p0[4-7]"
    tags:
      - "secondary-cluster"
      
steps:
  - description: Test primary cluster
    action: test.run
    args:
      target: primary_ports
      script: "test/primary_test.js"
    options:
      wait: true
      
  - description: Test secondary cluster
    action: test.run
    args:
      target: secondary_ports
      script: "test/secondary_test.js" 
    options:
      wait: true
      allowFailure: true

Best Practices

Use Descriptive Names: Make step descriptions clear and specific.
Handle Failures Gracefully: Use allowFailure: true for non-critical steps.
Organize with Targets: For complex scenarios, define targets to avoid repetition.
Variable Naming: Use consistent, descriptive variable names like ${HOST_LIMIT} instead of ${L}.
Separate Scenarios: Create different recipe files for different test scenarios rather than trying to use complex conditional logic.
Cleanup Steps: Always include cleanup steps with allowFailure: true to ensure resources are released.
Test Incrementally: Start with simple recipes and gradually add complexity.

Command Line Usage

# Basic execution
coordinator batch submit recipe.yaml

# With environment variables
coordinator batch submit recipe.yaml -e HOST -e USERNAME -e PASSWORD

# With custom arguments
coordinator batch submit recipe.yaml -a TARGET=tester -a LIMIT=2

# Dry run (show what would be executed)
coordinator batch submit recipe.yaml --dryrun

# Quiet mode (less output)
coordinator batch submit recipe.yaml --quiet

# Save output to file
coordinator batch submit recipe.yaml -o results.json

Integration with CI/CD

Recipes are commonly used in GitLab CI/CD pipelines:

test-framework:
  script:
    - coordinator batch submit recipe/functional_test.yaml
        -e COORDINATOR_USER
        -e CONFIG_STATE_FILE
        -a TARGET=tester
        -a SCRIPT_PATH=test/functional_test.js

test-vm:
  script:
    - coordinator batch submit recipe/functional_test_vm.yaml
        -e COORDINATOR_USER
        -e REGRESSION_USERNAME
        -e REGRESSION_PASSWORD
        -e REGRESSION_HOSTS
        -a TARGET=tester
        -a FRAMEWORK_FILE=iobuster.tgz

This allows for flexible, maintainable test automation that can adapt to different environments and requirements.

Batch Command and Recipe Reference

Introduction​

Recipe Structure​

Basic Format​

Keywords Reference​

Top-Level Keywords​

Target Definition​

Step Definition​

Available Actions​

Configuration Commands​

Host Management Commands​

Port Management Commands​

Test Commands​

Log Commands​

Variable Substitution​

Setting Variables​

Using Variables in Recipes​

Error Handling​

Allow Failure​

Conditional Execution Patterns​

Complete Examples​

Example 1: Simple Test Execution​

Example 2: Multi-Host Test with Cleanup​

Example 3: Using Targets​

Best Practices​

Command Line Usage​

Integration with CI/CD​