Seems like there is no easy way to insert EvalCallback to AdversarialTraining, no?
AdversarialTraining.train_gen() populates callback argument of sb3.BaseAlgorithm.learn() with self.gen_callback which is either None or reward_wrapper.RewardVecEnvWrapper().
As an additional note, learn_kwargs argument in AdversarialTraining.train_gen() is always None because AdversarialTraining.train() ignores that argument.
I think it is really crucial to monitor the progress of the imitation learning algorithm against true reward function. Using EvalCallback instead of specifying eval_freq and create_eval_env 1) is the recommended way in sb3 and 2) will be a "seamless plug" (because we just need to create eval environment inestad of doing fancy wrapping/unwrapping if we try to infer eval environment from the training environment).
Seems like there is no easy way to insert
EvalCallbacktoAdversarialTraining, no?AdversarialTraining.train_gen()populatescallbackargument ofsb3.BaseAlgorithm.learn()withself.gen_callbackwhich is eitherNoneorreward_wrapper.RewardVecEnvWrapper().As an additional note,
learn_kwargsargument inAdversarialTraining.train_gen()is alwaysNonebecauseAdversarialTraining.train()ignores that argument.I think it is really crucial to monitor the progress of the imitation learning algorithm against true reward function. Using
EvalCallbackinstead of specifyingeval_freqandcreate_eval_env1) is the recommended way in sb3 and 2) will be a "seamless plug" (because we just need to create eval environment inestad of doing fancy wrapping/unwrapping if we try to infer eval environment from the training environment).