Skip to main content
A Private Runner runs your test runs in your own AWS account instead of on Empirical’s hosted infrastructure. It is Empirical’s BYOC (bring your own cloud) option, available on the Custom plan. You install it with a CloudFormation template from the dashboard. Empirical still schedules runs, and reports still show up in the dashboard as usual.

​
Prerequisites

  • AWS account ID (12 digits). The setup link only works in that account. One runner per account per region.
  • A supported region: us-east-1 or ap-south-1. Projects that use the runner move to its region.
  • Permission to create a CloudFormation stack with named IAM roles. AdministratorAccess is enough.
  • A VPC with subnets that reach the internet: public subnets (0.0.0.0/0 → igw-…, as in the default VPC), or private subnets with 0.0.0.0/0 → nat-….
    • Pick subnets in two or more Availability Zones, all in the same VPC.
    • Keep one free IP per shard that runs at the same time.
    • No inbound rules are needed. Workers only make outbound connections.
    • With a NAT gateway, add an S3 gateway endpoint (com.amazonaws.<region>.s3, free) to the private subnets’ route tables to keep image pulls and report uploads off the NAT.
  • Fargate vCPU quota (Service Quotas, On-Demand and Spot) of about vCPU per shard × shards at once. Shards use 4–16 vCPU each, depending on machine size.

​
Set up a private runner

Sign in with empirical login (these endpoints need a signed-in user, not a project API key). If you have access to more than one project, add -p <project-slug> to each command.
  1. Create the runner and a setup link:
  2. Open url while signed in to that AWS account. Choose the VPC and subnets, acknowledge IAM resources, and create the stack.
  3. Wait for the runner to become active (about 5 minutes):
    The project that started the setup switches to the runner on its own.
  4. Use the runner for another project in the organization, or switch back with null:
  • Link expired or the stack rolled back: empirical api api/runner-backends/project/<id>/registration -X POST gives a new link.
  • Re-run the checks: empirical api api/runner-backends/project/verify -X POST.
If the stack fails a check, it rolls back. The reason is in the stack’s Events tab in CloudFormation.

​
Reach private apps

Workers run in your subnets, so they can reach anything those subnets can route to, using the VPC’s DNS.
  • App in the same VPC: allow inbound traffic from the runner’s security group (the SecurityGroupId stack output).
  • App in another VPC or on-premises: use VPC peering, Transit Gateway or Site-to-Site VPN, and add routes in the runner subnets.
  • App behind NetBird or Tailscale: you can’t run a VPN client inside the workers (Fargate), so use a routing peer:
    • Run an EC2 instance in the runner VPC as a routing peer (NetBird network route with masquerade, or a Tailscale subnet router).
    • Turn off its source/destination check.
    • In each runner subnet’s route table, route the app’s private CIDR to the instance.
    • Make the app’s hostname resolve in the VPC (a Route 53 private hosted zone or a Resolver forwarding rule), or use IPs.

​
Limitations

  • A project runs everything in one place. Once a project uses a private runner, all its test runs go there, from every trigger and environment. You can’t send only some runs to it. To split runs, use a separate project.