the calculation of advantage values

[caokaifa]

Thank you for sharing such excellent work. I would like to ask about the calculation of advantage values. Initially, the data comes from the absolute_advantage in progress or stage 2. However, before stage 2 is trained, how is this data from progress calculated? Is it computed using Monte Carlo returns?

[Lirui-Zhao]

Thanks for your question!

The calculation of the progress-based advantage used before stage 2 training is explained in this comment:
#13 (comment)

Please refer to that discussion for details.

Lirui Zhao