Skip to main content

Distributed Testing

When a single machine can't generate enough load (CPU or memory maxes out before reaching the target thread count), you distribute the load across multiple machines. JMeter has built-in support for this using a controller-worker architecture.

When You Need Distributed Testing​

Signs that a single machine is not enough:

  • CPU usage on the load generator is consistently above 80% during the test

  • JMeter runs out of memory (OutOfMemoryError)

  • You can't reach the target thread count without the machine slowing down

  • Response times are inflated because the load generator itself is the bottleneck, not the server

Rule of thumb: A single machine can typically handle 300-1000 threads depending on the script complexity, hardware specs, and whether you're making lightweight API calls or heavy page loads. Monitor your load generator machine during tests.


Architecture​

┌──────────────────┐
│ Controller │
│ (your machine) │
│ │
│ Sends .jmx │
│ Collects results│
└──────┬───────────┘
│
┌────────────┼────────────┐
│ │ │
┌──────▼───┐ ┌──────▼───┐ ┌──────▼───┐
│ Worker 1 │ │ Worker 2 │ │ Worker 3 │
│ (remote) │ │ (remote) │ │ (remote) │
│ │ │ │ │ │
│ Runs the │ │ Runs the │ │ Runs the │
│ test │ │ test │ │ test │
└──────────┘ └──────────┘ └──────────┘
  • Controller - the machine that sends the test plan and collects results. Does not generate load itself (by default)

  • Workers - remote machines that actually run the test and generate load

  • Each worker runs the same test plan with the same thread count

  • If you configure 100 threads and have 3 workers, you get 300 total threads


Prerequisites​

Distributed testing fails in ways that look like bugs in your script - an empty .jtl, a test that hangs, an RMI error with no obvious cause. Almost all of it traces back to a handful of things being wrong before the test ever runs. Get these right first and most of the mystery disappears.

Matching Versions on Every Machine​

Must matchWhy it matters
JMeter versionThe controller serializes the test plan and the workers deserialize it. A mismatch produces silent serialization failures, not a helpful error message
Java major versionSame reason - the objects crossing the wire must agree on both ends
PluginsAny plugin your .jmx depends on must be installed on every worker, or the plan fails to load there while the controller looks fine

Important: Version mismatch is the most common cause of "it connects, but nothing happens". Check it first, every time.

Ports to Open​

JMeter needs RMI in both directions: controller → worker to send the test plan, and worker → controller to send results back. The return path is the one people forget, and it is exactly why a test can start cleanly and still leave you with an empty .jtl.

PortLives onUsed forPropertyDefault
1099WorkerController connects here firstserver_port1099
50000 (example)WorkerThe worker's RMI engineserver.rmi.localport4000
50000-50002 (example)ControllerWorkers send results back hereclient.rmi.localport0 - random

Two things to notice:

  • client.rmi.localport defaults to 0, meaning a random high port. You cannot write a firewall rule for a port that changes every run, which is why the return path silently fails. Pin it.
  • The controller opens up to three consecutive ports starting at client.rmi.localport, so open a small range rather than a single port.

On each worker:

server_port=1099
server.rmi.localport=50000

On the controller:

client.rmi.localport=50000

Then open them. On a Linux worker using firewalld:

sudo firewall-cmd --permanent --add-port=1099/tcp
sudo firewall-cmd --permanent --add-port=50000-50100/tcp
sudo firewall-cmd --reload
sudo firewall-cmd --list-ports
# Expected: 1099/tcp 50000-50100/tcp

If the workers are cloud VMs, the provider's own firewall (security list, security group, NSG) has to allow the same ports - that is a separate layer from the OS firewall, and both must agree.

RMI SSL​

Since JMeter 4.0, RMI uses SSL by default - server.rmi.ssl.disable ships as false - and expects a keystore at bin/rmi_keystore.jks. You have two choices:

Internal test network (simplest): turn it off on both sides.

# jmeter.properties, on the controller AND every worker
server.rmi.ssl.disable=true

Otherwise: generate a keystore with bin/create-rmi-keystore and copy bin/rmi_keystore.jks to every worker and the controller.

Important: Whichever you choose, it must match on both ends. Disabling SSL on only one side produces connection errors that say nothing about SSL.

Network Reachability​

Four separate paths have to work, and it is worth confirming each before blaming JMeter:

  1. Controller → worker on the RMI port
  2. Worker → controller on the return ports (see above)
  3. Worker → target application - the workers generate the load, so they need access to the system under test, not just the controller
  4. No NAT between controller and workers - if the controller sits behind a home or office router, workers have no address to call back to. See NAT breaks distributed testing below

Verify before starting, from the controller:

# Is the worker's RMI port reachable?
nc -zv worker1-ip 1099

Important: If a worker has more than one IP - a cloud VM with a private and a public address, or any multi-NIC host - it announces itself to the controller using whichever one Java picks, and that may not be the one the controller can reach. Pin it explicitly when starting the worker:

jmeter-server -Djava.rmi.server.hostname=<the IP the controller uses>

Pre-Flight Checklist​

  • Same JMeter version on controller and all workers
  • Same Java major version everywhere
  • Required plugins installed on every worker
  • server_port and server.rmi.localport set on workers, ports open in the OS firewall
  • client.rmi.localport set on the controller, small port range open
  • Cloud firewall rules match the OS firewall rules
  • server.rmi.ssl.disable=true on both sides, or the keystore copied everywhere
  • Workers can reach the target application
  • Controller is not behind NAT relative to the workers
  • Test data files present on every worker at the expected path

Tip: Prove the setup with a throwaway plan against a public endpoint before involving your real script - see Use a dummy test plan for setup validation below. It separates "my distributed setup is broken" from "my script is broken", which are very different afternoons.


Setting Up Remote Machines​

With the prerequisites satisfied, each worker needs the test plan's data on disk and the JMeter server process running.

Test data files (CSV) must be copied to the same path on every machine - or use the same filename with different content per machine, so each worker drives a distinct slice of the data. See Section 14.

Start the JMeter Server on Each Worker​

On each remote machine, run:

jmeter-server

Or on Windows:

jmeter-server.bat

This starts the JMeter server process, listening for connections from the controller. Default RMI port is 1099.


Configuring JMeter for Distributed Mode​

On the controller machine, edit jmeter.properties (in the JMeter bin/ folder):

remote_hosts=worker1-ip:1099,worker2-ip:1099,worker3-ip:1099

Replace worker1-ip, worker2-ip, etc. with the actual IP addresses of your worker machines.

RMI Configuration​

The RMI ports and SSL settings both machines need are covered in Prerequisites - see Ports to Open and RMI SSL. If you skipped ahead, that is the section to go back to when a worker refuses to connect.


Running Distributed Tests​

Run on all configured remote hosts:

jmeter -n -t test-plan.jmx -l results.jtl -r

The -r flag tells JMeter to run on all remote hosts listed in remote_hosts.

To run on specific workers only:

jmeter -n -t test-plan.jmx -l results.jtl -R worker1-ip:1099,worker2-ip:1099

What Happens During Execution​

  1. Controller sends the .jmx file to all workers
  2. Each worker starts the test with the configured thread count
  3. Workers send results back to the controller in real-time
  4. Controller aggregates all results into the single .jtl file
  5. When the test ends, the controller generates the report (if -e -o flags were used)

Important Considerations​

Thread Count is Per Worker​

If the test plan has 100 threads and you have 3 workers, the total is 300 threads. Adjust your Thread Group accordingly:

Target Total UsersWorkersThreads Per Worker (in .jmx)
3003100
5005100
10004250

CSV Data Distribution​

If each user needs unique data (e.g., unique login credentials), you need to split the CSV data so each worker gets different rows. Two approaches:

  1. Same filename, different content - each worker has a file named users.csv but with different rows. This is the cleanest approach (see Section 14)

  2. Thread Group offset - use the same full CSV on all machines and configure CSV Data Set Config sharing mode

Files Are Not Automatically Distributed​

JMeter sends the .jmx file to workers, but not supporting files like:

  • CSV data files
  • JAR files for plugins
  • External scripts

You must copy these to each worker machine manually or via a script (see Section 15).

Timers and Think Time​

Timers work the same in distributed mode. Each worker applies think time independently.

Backend Listener in Distributed Mode​

If using a Backend Listener for Grafana monitoring, each worker sends data directly to InfluxDB. The results are automatically merged in Grafana since they share the same application and testTitle fields.


Practical Lessons Learned​

These lessons came from hands-on distributed testing with Linux VMs and Windows machines. To work through the setup yourself, see Section 13 — Try Distributed Testing on Your Own Machine, which walks the whole thing end to end on free local VMs. For the cloud-specific version, see the archived OCI Linux Slave Setup.

RMI SSL must be disabled on both sides​

The configuration is in RMI SSL above; what that section cannot convey is how unhelpful the failure is. JMeter 5.x enables RMI SSL by default and expects rmi_keystore.jks. Since we don't generate certificates for internal testing, disable it on both controller and slave:

# Controller: jmeter.properties (or -Jserver.rmi.ssl.disable=true)
server.rmi.ssl.disable=true
# Slave: in start script
jmeter-server -Dserver.rmi.ssl.disable=true

If you only disable one side, you'll get cryptic connection errors.

NAT breaks distributed testing​

JMeter requires bidirectional RMI: controller → slave (send test plan) and slave → controller (send results). If your controller is behind NAT (home router, office firewall), slaves can't send results back.

Symptom: Test starts on slaves, but JTL is empty and console shows summary = 0.

Solutions: Use same-LAN machines, port forwarding, VPN, or run the controller in the cloud alongside the slaves.

Console summary = 0 is often a display quirk​

In distributed mode, the console summariser sometimes shows summary = 0 even when results are being collected. Always check the actual JTL file — it usually has the data.

-J vs -G for properties​

  • -J sets properties on the controller only

  • -G sets properties on all slaves (sent via RMI)

  • Use -G for things like thread count and duration that slaves need: -Gthreads=10 -Gduration=60

Enterprise antivirus is a silent blocker​

Corporate security tools (Symantec, CrowdStrike, etc.) can silently drop JMeter RMI traffic. If distributed testing works on a personal PC but not on a work PC, the antivirus is likely the cause. Ask IT for exceptions on java.exe and JMeter ports.

Use a dummy test plan for setup validation​

Before testing with your real application scripts, create a lightweight test plan hitting a public endpoint like httpbin.org. This isolates network/configuration issues from application-specific problems. See test_plan/Dummy-HTTP-Test.jmx in the project.


Tips​

  • Test with 1 worker first - validate the distributed setup works before adding more workers

  • Monitor worker machines - check CPU and memory on workers during the test to ensure they're not overloaded

  • Keep JMeter versions in sync - mismatched versions between controller and workers cause silent failures

  • Use the same network - controller and workers should ideally be on the same network or low-latency connection. High latency between controller and workers can affect result collection

  • Automate worker setup - when you have many workers, manually copying files and starting jmeter-server is tedious. Use batch scripts (see Section 15)

  • Check firewall rules - the most common distributed testing issue is connectivity, in both directions. Work through the Pre-Flight Checklist before blaming the script

  • RMI hostname matters — on cloud VMs, hostname -I returns the private IP but external controllers connect via the public IP. Use the public IP for -Djava.rmi.server.hostname when controller is external, private IP when in the same VCN