Skip to main content
Parallelize your W&B Sweep agents on a multi-core or multi-GPU machine. Before you get started, ensure you have initialized your W&B Sweep. For more information on how to initialize a W&B Sweep, see Initialize sweeps.

​
Parallelize on a multi-CPU machine

Depending on your use case, explore the proceeding tabs to learn how to parallelize W&B Sweep agents using the CLI or within a Jupyter Notebook.
  • CLI
  • Jupyter Notebook
Use the wandb agent command to parallelize your sweep agent across multiple CPUs with the terminal. Provide the sweep ID that was returned when you initialized the sweep.
  1. Open more than one terminal window on your local machine.
  2. Copy and paste the code snippet below and replace sweep_id with your sweep ID:
wandb agent sweep_id

​
Parallelize on a multi-GPU machine

Follow the procedure outlined to parallelize your W&B Sweep agent across multiple GPUs with a terminal using CUDA Toolkit:
  1. Open more than one terminal window on your local machine.
  2. Specify the GPU instance to use with CUDA_VISIBLE_DEVICES when you start a W&B Sweep job (wandb agent). Assign CUDA_VISIBLE_DEVICES an integer value corresponding to the GPU instance to use.
For example, suppose you have two NVIDIA GPUs on your local machine. Open a terminal window and set CUDA_VISIBLE_DEVICES to 0 (CUDA_VISIBLE_DEVICES=0). Replace sweep_ID in the proceeding example with the W&B Sweep ID that is returned when you initialized a W&B Sweep: Terminal 1 
CUDA_VISIBLE_DEVICES=0 wandb agent sweep_ID
Open a second terminal window. Set CUDA_VISIBLE_DEVICES to 1 (CUDA_VISIBLE_DEVICES=1). Paste the same W&B Sweep ID for the sweep_ID mentioned in the proceeding code snippet: Terminal 2 
CUDA_VISIBLE_DEVICES=1 wandb agent sweep_ID