code crashed after reaching maxstep=20k

[valiantljk]

Very interesting work! Thanks for sharing the code.

I run into an issue when setting the max_timesteps=20000,

To reproduce it:

python main.py --expID 002 --td --bu --morphologies walker_7_main --max_timesteps 20000

It looks the training is finished, but an error was produced at the end:

ExpID: 2, FPS: 5.03, TotalT: 19902, EpisodeNum: 157, SampleNum: 20059, ReplayBSize: 20059
walker_7_main === EpisodeT: 98, Reward: 232.93
*** training finished and model saved to ./results/EXP_0002/model.pyth ***
Process Process-1:
Traceback (most recent call last):
  File "/opt/anaconda3/envs/modular-rl/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/opt/anaconda3/envs/modular-rl/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/anaconda3/envs/modular-rl/lib/python3.6/site-packages/baselines/common/vec_env/subproc_vec_env.py", line 10, in worker
    cmd, data = remote.recv()
  File "/opt/anaconda3/envs/modular-rl/lib/python3.6/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/opt/anaconda3/envs/modular-rl/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes
    buf = self._recv(4)
  File "/opt/anaconda3/envs/modular-rl/lib/python3.6/multiprocessing/connection.py", line 383, in _recv
    raise EOFError
EOFError