5.6. What is "agent VM" mode, and why would I want to use it?

It's like "same VM" mode, but better. By default, tests will run in the same JVM. In between tests, jtreg will try to reset the JVM to a standard state, and if it cannot, it will discard the JVM and start another.