Some network problems encountered during installation

[hzy46]

After trying installing PAI in Azure China, I encounter some problems during installation. Most of them are network-related. The installation uses the quick start script. This issue is a record.

1. [during k8s deployment] Cannot download kubeadm and hyperkube

   Solution: Modify kubeadm_download_url and hyperkube_download_url in config

2. [during k8s deployment] Cannot pull images from gcr.io

   Solution: Modify gcr_image_repo and kube_image_repo to a mirror repo. I tried gcr.azk8s.cn and it worked. But, surprisingly, kubespray still tries to pull gcr.io after the modification. After some investigation, I found it is a problem with kubespray 2.11. In this file, some images are hard coded to use gcr.io and k8s.gcr.io (e.g. tiller_image_repo). I fixed this issue manually then the k8s deployment succeeded.

3. [during PAI service deployment] Downloading speed from docker.io is slow

   I exprienced very slow downloading speed from docker.io. I tried some mirrors, but they didn't work. To address this problem, I set up a local registry and used a machine outside China to push images to the local registry.

4. [during PAI service deployment] Cannot access raw.githubusercontent.com

   We use file from raw.githubusercontent.com in device-plugin. Unfortunately, this site is blocked for me. I added a dns record for githubusercontent.com to /etc/hosts, then it worked.

[ydye]

Issue 1: I think u should updated it in doc
Issue 2: Have you try to use this configuration to overwirte the default value https://github.com/microsoft/pai/blob/master/contrib/kubespray/example/config.yml#L56
Issue 3: Use local cloud provider's container registry service
Issue 4: @abuccts

[hzy46]

Issue 1: I think u should updated it in doc
Issue 2: Have you try to use this configuration to overwirte the default value https://github.com/microsoft/pai/blob/master/contrib/kubespray/example/config.yml#L56
Issue 3: Use local cloud provider's container registry service
Issue 4: @abuccts

I have included solution for issue 1 and issue 2 in maunal.

For issue 2, modifying https://github.com/microsoft/pai/blob/master/contrib/kubespray/example/config.yml#L56 will help but as I mentioned, some images in kubespray ignore this setting, such as https://github.com/kubernetes-sigs/kubespray/blob/release-2.11/roles/download/defaults/main.yml#L281 . I think it is a problem with kubespray 2.11.

[abuccts]

[during PAI service deployment] Cannot access raw.githubusercontent.com
We use file from raw.githubusercontent.com in device-plugin. Unfortunately, this site is blocked for me. I added a dns record for githubusercontent.com to /etc/hosts, then it worked.

It's dns pollution from your ISP, you can either

• add a dns record
• or change your dns server to 114.114.114.114

[VitoRazor]

@hzy46
Hello, I am currently blocked in the configuration of kubeadm_download_url and hyperkube_download_url. I don't know where to download these two tools in china. I hope you can help me. Thank you very much

[hzy46]

@hzy46
Hello, I am currently blocked in the configuration of kubeadm_download_url and hyperkube_download_url. I don't know where to download these two tools in china. I hope you can help me. Thank you very much

Could you follow
https://openpai.readthedocs.io/en/latest/manual/cluster-admin/installation-faqs-and-troubleshooting.html#cannot-download-kubeadm-or-hyperkube , and find a way to download the two files?

[VitoRazor]

@hzy46
Hello, I am currently blocked in the configuration of kubeadm_download_url and hyperkube_download_url. I don't know where to download these two tools in china. I hope you can help me. Thank you very much

Could you follow
https://openpai.readthedocs.io/en/latest/manual/cluster-admin/installation-faqs-and-troubleshooting.html#cannot-download-kubeadm-or-hyperkube , and find a way to download the two files?

Thank you very much. I benefited a lot from your answer

[hzy46]

Closed. For china users, refer to https://openpai.readthedocs.io/en/latest/manual/cluster-admin/configuration-for-china.html