DDQN

DDQN

Synchronous n-step Double DQN.

DDQNTrainerConfig dataclass

DDQNTrainerConfig(train_mode: TrainMode = TrainMode.NSTEP, returns: Returns = Returns.NSTEP, total_steps: int = 50000000, nsteps: int = 5, gamma: float = 0.99, lambda_: float = 0.95, validate_freq: int = 1000000, num_val_episodes: int = 50, max_val_steps: int = 10000, log_dir: str = 'logs/', model_dir: str = 'models/', save_freq: int = 0, log_scalars: bool = True, update_target_freq: int = 0, render_freq: int = 0, epsilon_start: float = 1.0, epsilon_final: float = 0.01, epsilon_steps: float = 1000000.0, epsilon_test: float = 0.01)

Bases: TrainerConfig

Hyperparameters for :class:SyncDDQN.

SyncDDQN

SyncDDQN(envs, agent: DQN, target_agent: DQN, val_envs, action_size, config: DDQNTrainerConfig)

Bases: SyncMultiEnvTrainer

Synchronous Double-DQN trainer (n-step or one-step TD).

Source code in rlib/DDQN/trainer.py

def __init__(
    self,
    envs,
    agent: DQN,
    target_agent: DQN,
    val_envs,
    action_size,
    config: DDQNTrainerConfig,
):
    super().__init__(envs=envs, agent=agent, val_envs=val_envs, config=config)

    self.target_agent = self.TargetQ = target_agent
    self.Q = self.agent  # more readable alias
    self.epsilon = np.array([config.epsilon_start], dtype=np.float64)
    self.epsilon_final = config.epsilon_final
    self.epsilon_steps = config.epsilon_steps
    self.schedule = self.linear_schedule(
        self.epsilon, config.epsilon_final, int(config.epsilon_steps // self.num_envs)
    )
    self.epsilon_test = np.array(config.epsilon_test, dtype=np.float64)
    self.action_size = action_size