Refactor CI to improve source download and build reliability

Co-authored-by: weixuan <weixuan@giskard.ai>

Author: cursoragent