PPO with policy and baseline based on model

[ahtsan]

PPO with GaussianMLPPolicyWithModel and GaussianMLPBaselineWithModel.

Interestingly, ppo with models achieve higher performance than ppo without models.

Red and blue: with models
Orange and cyan: without models

[codecov]

Codecov Report

Merging #616 into master will increase coverage by 0.07%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #616      +/-   ##
==========================================
+ Coverage   60.59%   60.67%   +0.07%     
==========================================
  Files         156      156              
  Lines        9045     9045              
  Branches     1237     1237              
==========================================
+ Hits         5481     5488       +7     
+ Misses       3257     3241      -16     
- Partials      307      316       +9

Impacted Files	Coverage Δ
garage/tf/policies/categorical_gru_policy.py	80% <0%> (ø)	⬆️
...arage/tf/samplers/off_policy_vectorized_sampler.py	75.34% <0%> (ø)	⬆️
garage/tf/policies/gaussian_gru_policy.py	78.67% <0%> (ø)	⬆️
garage/misc/krylov.py	17.94% <0%> (ø)	⬆️
garage/sampler/stateful_pool.py	38.63% <0%> (ø)	⬆️
garage/tf/policies/gaussian_lstm_policy.py	78.83% <0%> (ø)	⬆️
garage/tf/policies/categorical_lstm_policy.py	79.83% <0%> (ø)	⬆️
...rage/tf/optimizers/conjugate_gradient_optimizer.py	80.76% <0%> (+2.74%)	⬆️
garage/tf/optimizers/first_order_optimizer.py	68.05% <0%> (+2.77%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 15a4ed9...8af26f8. Read the comment docs.