Profiling

Profiling#

In slime, we can perform detailed performance analysis of the rollout process using the profiling interface provided by SGLang.

1. Sleeping the Rollout Process#

For more flexible stress testing and profiling, it is often useful to make the slime rollout process enter a waiting state after initialization, instead of starting generation immediately.

You can achieve this by replacing the rollout_function_path in your startup arguments without modifying the source code:

python train.py \
    --rollout-function-path slime.rollout.sleep_rollout.sleep \
    ... (other arguments)

This function will make the rollout process enter an infinite wait loop, allowing you to manually send requests or run stress testing tools.

2. Obtaining SGLang Engine List#

SGLang engines (workers) are registered with the router. You can retrieve the list of all active engines by accessing the /workers endpoint of the router.

The router address is typically printed in the startup logs:

Router launched at 127.0.0.1:3000

You can use curl to view the workers:

curl http://127.0.0.1:3000/workers

3. Using Automated Profiling Tool#

To simplify profiling across multiple engines simultaneously, we provide an automated script: tools/profile_rollout.py.

Starting Profiling#

By default, this tool starts profiling on all workers and will automatically stop after 3 steps:

python tools/profile_rollout.py --router-url http://127.0.0.1:3000 --action start --num-steps 3

Key Parameters:

  • --router-url: The URL of the Router.

  • --num-steps: Number of steps to record, defaults to 3.

  • --output-dir: Directory where trace files will be saved.

  • --activities: Activities to monitor, e.g., GPU CPU.

  • --profile-by-stage: Whether to profile by stage (prefill/decode).

Stopping Profiling Manually#

If you did not set num_steps or wish to stop early:

python tools/profile_rollout.py --router-url http://127.0.0.1:3000 --action stop

4. Running Stress Tests#

While the Rollout process is in a waiting state via sleep_rollout, you can:

  1. Start profiling using tools/profile_rollout.py.

  2. Use stress testing tools (such as SGLang’s built-in benchmark tools) to send requests to the router or directly to the engines.

  3. Wait for profiling to complete (if num_steps was set) or stop it manually.

  4. Collect the .json trace files from the output_dir and view them using chrome://tracing in Chrome or Perfetto.

Contents