[WINDOWS][AutoTVM] OSError: [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted and OSError: [WinError 10049] The requested address is not valid in its context

[Coderx7]

This issue is a followup to the #4819, After applying the #4820 changes, Under windows, trying to run the example : Auto-tuning a convolutional network for x86 CPU this error occurs.
the error occurs in the last cell :

def tune_and_evaluate(tuning_opt):
    # extract workloads from relay program
    print("Extract tasks...")
    mod, params, data_shape, out_shape = get_network(model_name, batch_size)
    tasks = autotvm.task.extract_from_program(mod["main"], target=target,
                                              params=params, ops=(relay.op.nn.conv2d,))

    # run tuning tasks
    print("Tuning...")
    tune_kernels(tasks, **tuning_opt)
    tune_graph(mod["main"], data_shape, log_file, graph_opt_sch_file)

    # compile kernels with graph-level best records
    with autotvm.apply_graph_best(graph_opt_sch_file):
        print("Compile...")
        with relay.build_config(opt_level=3):
            graph, lib, params = relay.build_module.build(
                mod, target=target, params=params)

        # upload parameters to device
        ctx = tvm.cpu()
        data_tvm = tvm.nd.array((np.random.uniform(size=data_shape)).astype(dtype))
        module = runtime.create(graph, lib, ctx)
        module.set_input(input_name, data_tvm)
        module.set_input(**params)

        # evaluate
        print("Evaluate inference time cost...")
        ftimer = module.module.time_evaluator("run", ctx, number=100, repeat=3)
        prof_res = np.array(ftimer().results) * 1000  # convert to millisecond
        print("Mean inference time (std dev): %.2f ms (%.2f ms)" %
              (np.mean(prof_res), np.std(prof_res)))

# We do not run the tuning in our webpage server since it takes too long.
# Uncomment the following line to run it by yourself.

# tune_and_evaluate(tuning_option)

Error :

Extract tasks...
ANTLR runtime and generated code versions disagree: 4.8!=4.7.2
ANTLR runtime and generated code versions disagree: 4.8!=4.7.2
Tuning...
[Task  1/12]  Current/Best:    0.00/   0.00 GFLOPS | Progress: (0/252) | 0.00 sTraceback (most recent call last):

  File "C:\Users\User\Anaconda3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)

  File "C:\Users\User\Anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)

  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\exec\rpc_server.py", line 138, in <module>
    main(args)

  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\exec\rpc_server.py", line 58, in main
    silent=args.silent)

  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\rpc\server.py", line 389, in __init__
    raise sock_err

  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\rpc\server.py", line 382, in __init__
    sock.bind((host, my_port))

OSError: [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted

Exception ignored in: <function Server.__del__ at 0x0000021F5BF3A8B8>
Traceback (most recent call last):
  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\rpc\server.py", line 419, in __del__
    self.terminate()
  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\rpc\server.py", line 414, in terminate
    if self.proc:
AttributeError: 'Server' object has no attribute 'proc'
Exception in thread Thread-3:
Traceback (most recent call last):
  File "C:\Users\User\Anaconda3\lib\threading.py", line 926, in _bootstrap_inner
    self.run()
  File "C:\Users\User\Anaconda3\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\autotvm\measure\measure_methods.py", line 572, in _check
    remote = request_remote(device_key, host, port, priority)
  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\autotvm\measure\measure_methods.py", line 539, in request_remote
    tracker = _rpc.connect_tracker(host, port)
  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\rpc\client.py", line 430, in connect_tracker
    return TrackerSession((url, port))
  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\rpc\client.py", line 221, in __init__
    self._connect()
  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\rpc\client.py", line 227, in _connect
    self._sock = base.connect_with_retry(self._addr)
  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\rpc\base.py", line 171, in connect_with_retry
    raise sock_err
  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\rpc\base.py", line 167, in connect_with_retry
    sock.connect(addr)
OSError: [WinError 10049] The requested address is not valid in its context

Traceback (most recent call last):

  File "tune_relay_x86.py", line 225, in <module>
    tune_and_evaluate(tuning_option)

  File "tune_relay_x86.py", line 198, in tune_and_evaluate
    tune_kernels(tasks, **tuning_opt)

  File "tune_relay_x86.py", line 170, in tune_kernels
    autotvm.callback.log_to_file(log_filename)])

  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\autotvm\tuner\tuner.py", line 125, in tune
    results = measure_batch(inputs)

  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\autotvm\measure\measure.py", line 260, in measure_batch
    build_results = builder.build(measure_inputs)

  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\autotvm\measure\measure_methods.py", line 105, in build
    **self.build_kwargs)

  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\autotvm\measure\local_executor.py", line 151, in submit
    process.start()

  File "C:\Users\User\Anaconda3\lib\multiprocessing\process.py", line 112, in start
    self._popen = self._Popen(self)

  File "C:\Users\User\Anaconda3\lib\multiprocessing\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)

  File "C:\Users\User\Anaconda3\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)

  File "C:\Users\User\Anaconda3\lib\multiprocessing\popen_spawn_win32.py", line 89, in __init__
    reduction.dump(process_obj, to_child)

  File "C:\Users\User\Anaconda3\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)

AttributeError: Can't pickle local object '_wrap_build_func.<locals>._wrapped'

 Done.
Exception ignored in: <function Server.__del__ at 0x000001EDA5A7BDC8>
Traceback (most recent call last):
  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\rpc\server.py", line 419, in __del__
    self.terminate()
  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\rpc\server.py", line 411, in terminate
    os.killpg(self.proc.pid, signal.SIGTERM)
AttributeError: module 'os' has no attribute 'killpg'

D:\Codes\tvm_testbed>Extract tasks...
ANTLR runtime and generated code versions disagree: 4.8!=4.7.2
ANTLR runtime and generated code versions disagree: 4.8!=4.7.2
Tuning...
[Task  1/12]  Current/Best:    0.00/   0.00 GFLOPS | Progress: (0/252) | 0.00 sTraceback (most recent call last):

  File "<string>", line 1, in <module>

  File "C:\Users\User\Anaconda3\lib\multiprocessing\spawn.py", line 105, in spawn_main
    exitcode = _main(fd)

  File "C:\Users\User\Anaconda3\lib\multiprocessing\spawn.py", line 114, in _main
    prepare(preparation_data)

  File "C:\Users\User\Anaconda3\lib\multiprocessing\spawn.py", line 225, in prepare
    _fixup_main_from_path(data['init_main_from_path'])

  File "C:\Users\User\Anaconda3\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
    run_name="__mp_main__")

  File "C:\Users\User\Anaconda3\lib\runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)

  File "C:\Users\User\Anaconda3\lib\runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)

  File "C:\Users\User\Anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)

  File "D:\Codes\tvm_testbed\tune_relay_x86.py", line 225, in <module>
    tune_and_evaluate(tuning_option)

  File "D:\Codes\tvm_testbed\tune_relay_x86.py", line 198, in tune_and_evaluate
    tune_kernels(tasks, **tuning_opt)

  File "D:\Codes\tvm_testbed\tune_relay_x86.py", line 170, in tune_kernels
    autotvm.callback.log_to_file(log_filename)])

  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\autotvm\tuner\tuner.py", line 108, in tune
    measure_batch = create_measure_batch(self.task, measure_option)

  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\autotvm\measure\measure.py", line 252, in create_measure_batch
    attach_objects = runner.set_task(task)

  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\autotvm\measure\measure_methods.py", line 332, in set_task
    tracker = Tracker('0.0.0.0', port=9000, port_end=10000, silent=True)

  File "C:\Users\User\Anaconda3\lib\site-packages\tvm-0.7.dev0-py3.7-win-amd64.egg\tvm\rpc\tracker.py", line 404, in __init__
    self.proc.start()

  File "C:\Users\User\Anaconda3\lib\multiprocessing\process.py", line 112, in start
    self._popen = self._Popen(self)

  File "C:\Users\User\Anaconda3\lib\multiprocessing\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)

  File "C:\Users\User\Anaconda3\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)

  File "C:\Users\User\Anaconda3\lib\multiprocessing\popen_spawn_win32.py", line 46, in __init__
    prep_data = spawn.get_preparation_data(process_obj._name)

  File "C:\Users\User\Anaconda3\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
    _check_not_importing_main()

  File "C:\Users\User\Anaconda3\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
    is not going to be frozen to produce an executable.''')

RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

 Done.

Compilmentary info:
OS: Windows 10 build 1803
CPU : Intel Core-i7-6770HQ @ 2.6Ghz
GPU: Intel(R) Iris(R) Pro Graphics 580
Python: Python 3.7.4 [MSC v.1915 64 bit (AMD64)] :: Anaconda, Inc. on win32
Compilers : Microsoft Visual C++14 (Visual Studio 2019 community Edition)
Build_Config:

• Set(LLVM ON)
• Set(OpenCL ON)
  LLVM ver: 9.0.1 - built from source code.
  OpenCL ver: 2.1 (Installed using Intel SDK for OpenCL Applications (intel_sdk_for_opencl_applications_2020.0.245.zip)

config.cmake

#---------------------------------------------
# Backend runtimes.
#---------------------------------------------

# Whether enable CUDA during compile,
#
# Possible values:
# - ON: enable CUDA with cmake's auto search
# - OFF: disable CUDA
# - /path/to/cuda: use specific path to cuda toolkit
set(USE_CUDA OFF)

# Whether enable ROCM runtime
#
# Possible values:
# - ON: enable ROCM with cmake's auto search
# - OFF: disable ROCM
# - /path/to/rocm: use specific path to rocm
set(USE_ROCM OFF)

# Whether enable SDAccel runtime
set(USE_SDACCEL OFF)

# Whether enable Intel FPGA SDK for OpenCL (AOCL) runtime
set(USE_AOCL OFF)

# Whether enable OpenCL runtime
set(USE_OPENCL ON)

# Whether enable Metal runtime
set(USE_METAL OFF)

# Whether enable Vulkan runtime
#
# Possible values:
# - ON: enable Vulkan with cmake's auto search
# - OFF: disable vulkan
# - /path/to/vulkan-sdk: use specific path to vulkan-sdk
set(USE_VULKAN OFF)

# Whether enable OpenGL runtime
set(USE_OPENGL OFF)

# Whether enable MicroTVM runtime
set(USE_MICRO OFF)

# Whether to enable SGX runtime
#
# Possible values for USE_SGX:
# - /path/to/sgxsdk: path to Intel SGX SDK
# - OFF: disable SGX
#
# SGX_MODE := HW|SIM
set(USE_SGX OFF)
set(SGX_MODE "SIM")
set(RUST_SGX_SDK "/path/to/rust-sgx-sdk")

# Whether enable RPC runtime
set(USE_RPC ON)

# Whether embed stackvm into the runtime
set(USE_STACKVM_RUNTIME OFF)

# Whether enable tiny embedded graph runtime.
set(USE_GRAPH_RUNTIME ON)

# Whether enable additional graph debug functions
set(USE_GRAPH_RUNTIME_DEBUG OFF)

# Whether enable additional vm profiler functions
set(USE_VM_PROFILER OFF)

# Whether enable uTVM standalone runtime
set(USE_MICRO_STANDALONE_RUNTIME OFF)

# Whether build with LLVM support
# Requires LLVM version >= 4.0
#
# Possible values:
# - ON: enable llvm with cmake's find search
# - OFF: disable llvm
# - /path/to/llvm-config: enable specific LLVM when multiple llvm-dev is available.
set(USE_LLVM ON)
set(LLVM_DIR D:/Codes/tvm_testbed/tools/llvm-9.0.1.src.tar/llvm-9.0.1.src/lib/cmake/llvm/)
#---------------------------------------------
# Contrib libraries
#---------------------------------------------
# Whether use BLAS, choices: openblas, mkl, atlas, apple
set(USE_BLAS none)

# /path/to/mkl: mkl root path when use mkl blas library
# set(USE_MKL_PATH /opt/intel/mkl) for UNIX
# set(USE_MKL_PATH ../IntelSWTools/compilers_and_libraries_2018/windows/mkl) for WIN32
# set(USE_MKL_PATH <path to venv or site-packages directory>) if using `pip install mkl`
set(USE_MKL_PATH none)

# Whether use MKLDNN library
set(USE_MKLDNN OFF)

# Whether use OpenMP thread pool, choices: gnu, intel
# Note: "gnu" uses gomp library, "intel" uses iomp5 library
set(USE_OPENMP none)

# Whether use contrib.random in runtime
set(USE_RANDOM OFF)

# Whether use NNPack
set(USE_NNPACK OFF)

# Possible values:
# - ON: enable tflite with cmake's find search
# - OFF: disable tflite
# - /path/to/libtensorflow-lite.a: use specific path to tensorflow lite library 
set(USE_TFLITE OFF)

# /path/to/tensorflow: tensorflow root path when use tflite library
set(USE_TENSORFLOW_PATH none)

# Possible values:
# - OFF: disable tflite support for edgetpu
# - /path/to/edgetpu: use specific path to edgetpu library
set(USE_EDGETPU OFF)

# Whether use CuDNN
set(USE_CUDNN OFF)

# Whether use cuBLAS
set(USE_CUBLAS OFF)

# Whether use MIOpen
set(USE_MIOPEN OFF)

# Whether use MPS
set(USE_MPS OFF)

# Whether use rocBlas
set(USE_ROCBLAS OFF)

# Whether use contrib sort
set(USE_SORT ON)

# Whether use MKL-DNN (DNNL) codegen
set(USE_DNNL_CODEGEN OFF)

# Build ANTLR parser for Relay text format
# Possible values:
# - ON: enable ANTLR by searching default locations (cmake find_program for antlr4 and /usr/local for jar)
# - OFF: disable ANTLR
# - /path/to/antlr-*-complete.jar: path to specific ANTLR jar file
set(USE_ANTLR OFF)

# Whether use Relay debug mode
set(USE_RELAY_DEBUG OFF)

# Whether to build fast VTA simulator driver
set(USE_VTA_FSIM ON)

# Whether to build cycle-accurate VTA simulator driver
set(USE_VTA_TSIM ON)

# Whether to build VTA FPGA driver (device side only)
set(USE_VTA_FPGA OFF)

# Whether to build the example external runtime module
set(USE_EXAMPLE_EXT_RUNTIME OFF)

[tqchen]

Given that we do not have a clearly actionable item atm, I would recommend to bring a trouble shooting thread on https://discuss.tvm.ai/ instead. Feel free to open new thread after we have something that is actionable

[Coderx7]

@tqchen: what do you mean by not having any actionable item? do you mean windows build is not the priority or you do not plan on focusing on fixing such issues?
In any case, imho, it would not help much to close this issue, when its not fixed as we may lose track of the issues that still exist.
I propsed a pr to fix one of the problems in this issue and I'll be fixing others (or at least try to) as my time permits. Having this open can at least show whats missing or needs to be fixed in this regard!

Cheers

[tqchen]

sure, we usually encourage everyone to open new thread on https://discuss.tvm.ai/ where the community work collectively to resolve the problems, once we have a clear plan to act on what can be fixed, then we start to quickly resolve the problem. Note that there is not really too much difference between the forum and the issue thread. The only thing is that we want to be able to keep all the issues managable(close within a period of time) without losing track.

In this case, we can also leave the issue open for a bit :)

[masahi]

AutoTVM doesn't work on Win at the moment, @jmorrill is working on it. See #4548