When a task have an assigned user we can't remove him

**Use case who bug:**
1. Add a task with a user assigned
2. Visit the detail view of this task
3. Edit the task
4. Try to remove the assigned user