Add a feature for changing some configs on the fly

[netanelC]

When developers test their code, they sometimes want to customize the config for a specific test.

We want to provide them with a convenient way to customize the config