PRTT - Performance Regression Testing Toolkit

PRTT makes Unreal performance testing repeatable: record a gameplay path once, then replay it across builds to catch regressions.
Instead of “fly around and eyeball FPS”, you get a consistent player path and camera view every run.
PRTT is designed for single-player style flows and integrates cleanly with CI via packaged builds and ExecCmds.

Features

• Repeatable replays: Record a movement/camera path in PIE and replay it consistently across runs and builds.
• Unreal Insights ready: Every test run captures a .utrace file for deep CPU/GPU analysis.
• Editor UI: Dedicated PRTT window for start/stop recording and replay, with automatic asset discovery.
• CI integration: prtt.TestCaseStartCI lets packaged builds auto-run a test case and exit for automation.

Table of Contents

• Quick Start
• Trace Setup
• Recording Test Cases
• Testing in Packaged Builds
• Console Commands
• CI Integration
• How It Works
• Security
• Limitations
• Troubleshooting
• Support

Quick Start

1. Record: Create test cases in Play-In-Editor (PIE) using the editor tool
2. Package: Build your game in Test/Development configuration
3. Test: Run test cases in packaged builds and analyze the resulting traces in Unreal Insights

Verify setup

After setup, PRTT recording assets under /Game/PRTT/Recordings/ will:

• Be discovered at runtime via AssetRegistry
• Be cooked into Development/Test builds by the plugin’s cook configuration
• Be available at runtime via PRTT console commands

In edge cases where assets still don’t appear in packaged builds (see Troubleshooting), you can fall back to explicitly referencing them via a DataAsset.

Trace Setup

PRTT captures utraces during replay via Trace.File. PRTT does not automatically enable trace channels. For meaningful Unreal Insights analysis, enable trace channels before starting a replay. PRTT will warn (PRTT: Failed to start utrace capture) if Trace.File fails.

Minimal recommended trace channels: CPU, GPU, LoadTime.

In editor: Open the Trace tab (bottom of editor, next to Output Log) → Channels (in Trace Data) → enable Cpu, Gpu, and LoadTime in the dropdown.

Packaged builds (recommended flags):

• For basic tracing needed by PRTT, you can still use:
  • -trace=cpu,gpu,loadtime
• For CI/perf runs with richer data (recommended):
  • -trace=cpu,gpu,frame,bookmark,log,loadtime,file
  • -statnamedevents

If Trace.File fails, ensure your build includes trace support. See Unreal Insights Reference for command-line options.

Recording Test Cases

Recommended: Using the Editor Tool

The easiest way to record and manage test cases in PIE is using the PRTT Editor Window:

1. Open your game map in the editor
2. Start Play-In-Editor (PIE)
3. Open the PRTT Editor Window: Tools → PRTT - Performance Regression Testing Toolkit
4. In the PRTT window:
   • Click Start Recording
   • Play your test scenario
   • Click Stop Recording when done
5. The recording is automatically saved as a Data Asset in /Game/PRTT/Recordings/

The editor tool provides:

• Visual list of all recorded test cases
• Easy start/stop recording controls
• Connection status to PIE
• Ability to replay test cases directly in the editor

Alternative: Using Console Commands

You can also use console commands in PIE:

1. Open your game map in the editor

2. Start Play-In-Editor (PIE)

3. Open the console (press ~)

4. Start recording:

   prtt.RecordStart <test_case_name>
   

   Example: prtt.RecordStart combat_sequence_01

   If you don't provide a name, a timestamped name will be generated automatically.

5. Play your test scenario

6. Stop recording:

   prtt.RecordStop
   

Verify Recording

1. Open Content Browser
2. Navigate to /Game/PRTT/Recordings/
3. You should see your test case asset (e.g., combat_sequence_01)
4. You can also view it in the PRTT Editor Window: Tools → PRTT

Testing in Packaged Builds

Step 1: Package Your Game

Package your game in Test or Development configuration (PRTT is disabled in Shipping builds).

Recommended: Use Test builds for performance testing (closer to Shipping than Development in most setups). Development builds include debug symbols and assertions, which can add overhead and skew results - making it harder to spot real regressions or compare fairly across builds.

You can package using:

• File > Package Project > Windows > Windows (64-bit)
• Select Test or Development configuration
• Or use UAT (Unreal Automation Tool) with your project's build scripts

Step 2: Run the Packaged Game

Launch your packaged game executable. For performance/CI runs, launch it with trace + stats flags, for example:

YourGame.exe -log -trace=cpu,gpu,frame,bookmark,log,loadtime,file -statnamedevents

Step 3: List Available Test Cases

1. Open the console (press ~)
2. List all available test cases:

   prtt.ListTestCases
   

3. You should see output like:

   PRTT: Found 2 recording asset(s):
     - combat_sequence_01
     - exploration_path_main
   

Step 4: Run a Test Case

1. Make sure you're on the correct map (the one used during recording)

2. Start replay:

   prtt.TestCaseStart <test_case_name>
   

   Example: prtt.TestCaseStart combat_sequence_01

3. The replay will:

   • Load the recording
   • Replay the exact movement path
   • Capture a utrace (Unreal trace file)

Step 5: View results

After the replay completes, a utrace file is saved to:

Saved/PRTT/Utraces/

Files are named <test_case_name>_<timestamp>.utrace. Open them with Unreal Insights or other trace tools.

Console Commands

PRTT exposes several console commands for recording and replaying test cases.
All commands are prefixed with prtt. and work in both editor and packaged builds (except Shipping).

Recording Commands

Replay Commands

Utility Commands

Editor-Only Commands

CI Integration

PRTT can be integrated into CI pipelines by launching a packaged build with ExecCmds and automatically exiting once a test case finishes.
It is designed to be easy to run in CI against a packaged Development/Test build.

Example CI PowerShell snippet

This example uses a packaged Test/Development build, runs a specific test case via prtt.TestCaseStartCI, and exits with the game process exit code:

# Path to your packaged Test/Development build (executable typically lives under Binaries\Win64)
$Exe = "C:\Builds\MyGame\Binaries\Win64\MyGame-Win64-Test.exe"
$Dir = Split-Path $Exe

# ExecCmds: run CI test case (game auto-exits after test via TestCaseStartCI)
$ExecCmds = 'prtt.TestCaseStartCI multiple_spot_light_corridor'

$CommandArgs = @(
    "-log",
    "-trace=cpu,gpu,frame,bookmark,log,loadtime,file",
    "-statnamedevents",
    "-ExecCmds=""$ExecCmds"""
)

$p = Start-Process -FilePath $Exe -ArgumentList $CommandArgs -PassThru -Wait -WorkingDirectory $Dir
exit $p.ExitCode

You can adapt this snippet to your own game executable, map, and test case names as needed.

How It Works

Automatic Asset Management

PRTT works automatically:

1. Asset Discovery: PRTT uses AssetRegistry to discover PRTTRecordingAsset assets at runtime
2. Runtime Enumeration: Assets can be listed and loaded at runtime via AssetRegistry
3. Performance Measurement: Replays capture Unreal trace (.utrace) files you can analyze in Unreal Insights

Note: For assets to be available in packaged builds, they must be cooked.

Asset cooking (easy mode)

PRTT configures the Asset Manager and cook pipeline to ensure PRTTRecordingAsset assets under /Game/PRTT/Recordings/ are included automatically in Development/Test builds (via plugin DefaultGame.ini and a cook delegate). In most cases, simply keeping your recordings in that folder is enough.

Asset Storage

• Location: /Game/PRTT/Recordings/
• Format: Data Assets (.uasset files)
• Content: Player movement path, camera settings, timing data
• Naming: Use descriptive names (e.g., combat_sequence_01, exploration_path_main, boss_encounter_phase1)

Build Configuration Support

Security

Disabled in Shipping builds (no runtime overhead). Test assets are not intended to ship.

Shipping Builds

• Compile-time: Code excluded via #if UE_BUILD_SHIPPING
• Runtime: Subsystem doesn't initialize in Shipping
• Assets: Test case assets are never cooked into Shipping packages
• Console commands: Not available in Shipping builds

Best Practices

1. Never reference PRTT assets in Shipping-critical code
2. Use Test/Development builds for performance testing
3. Keep test cases in /Game/PRTT/Recordings/ (excluded from Shipping)
4. Verify Shipping builds don't include PRTT assets (check package size/logs)

Limitations

• Same map: Replay must run on the same map used during recording
• Same pawn/controller: Requires a compatible pawn and player controller
• Single-player focus: Optimized for single-player; multiplayer support is limited
• Trace dependencies: Utrace capture requires Trace.File/Trace.Stop (Development/Test builds)

Troubleshooting

Problem: prtt.ListTestCases shows no assets in packaged build

Possible causes:

1. Assets weren't cooked (most common)
2. Assets not referenced anywhere in the project
3. Assets not in /Game/PRTT/Recordings/

Solutions:

1. ✅ Ensure assets are referenced somewhere in your project (e.g., in a DataAsset) so they get cooked
2. ✅ Ensure assets exist in Content Browser under /Game/PRTT/Recordings/
3. ✅ Check packaging logs for asset cooking errors
4. ✅ Repackage after ensuring assets are referenced

Problem: Replay doesn't work

Possible causes:

1. Wrong map loaded
2. Asset not found
3. Recording has no frames

Solutions:

1. ✅ Load the same map used during recording
2. ✅ Verify asset name is correct (case-sensitive)
3. ✅ Check console for error messages
4. ✅ Verify recording has frames: prtt.ListTestCases should show the asset

Problem: Assets appear in editor but not in packaged build

Cause: Assets are not being cooked because they're not referenced anywhere in the project.

Solution:

1. ✅ Reference assets in a DataAsset/Blueprint that's already cooked
2. ✅ Or create a DataAsset that references all PRTT recording assets and ensure it's referenced in your project

Problem: Recording doesn't save

Possible causes:

1. Not in PIE (Play-In-Editor)
2. No pawn/player controller
3. Editor-only operation

Solutions:

1. ✅ Ensure you're in Play-In-Editor, not standalone game
2. ✅ Verify you have a pawn/player controller in the world
3. ✅ Check console for error messages

Problem: Utrace file not found

Possible causes:

1. Replay didn't complete
2. Output directory permissions
3. Trace.File / Trace.Stop not available in your build

Solutions:

1. ✅ Let replay complete fully (wait for automatic exit or completion message)
2. ✅ Check Saved/PRTT/Utraces/ directory exists and is writable
3. ✅ Review console output for errors; ensure Trace.File started (PRTT logs the path when capture begins)

Support

For any issues or questions:

• Discord: Join the Infraction Labs server (https://discord.gg/WUynA27SHq)
• Email: piotr@infractionlabs.com

Version: 1.0 Author: Infraction Labs
License: See LICENSE file