use installed C++ RMM in python build (rapidsai#588)

* build/python: don't pass --library-dir argument

For all I can tell it's not actually recognized by setuptools, nor actually
used, and in any case, there is no more actual library.

* build/python: add fallback to INSTALL_PREFIX

* build/python: use installed location of C++ RMM for python build

Various existing hardcoded paths were not actually used anymore,
this is generally cleaner, and importantly, if the C++ build pulled in
external dependencies that were not locally installed, this will ensure
that these are used in the python build.

* changelog

* autopep8

* reformat via black

* reformat again with black-19.10b0

* review feedback: don't require INSTALL_PREFIX to be set

If it is not set, fall back to using the uninstalled headers in
the source tree.

Author: germasch

CHANGELOG.md

@@ -46,6 +46,7 @@
 - PR #581 Improve logging documentation
 - PR #585 Update ci/local/README.md
 - PR #587 Replaced `move` with `std::move`
+- PR #588 Use installed C++ RMM in python build
 
 ## Bug Fixes
 

build.sh

@@ -45,9 +45,9 @@ PER_THREAD_DEFAULT_STREAM=OFF
 RAN_CMAKE=0
 
 # Set defaults for vars that may not have been defined externally
-#  FIXME: if INSTALL_PREFIX is not set, check PREFIX, then check
-#         CONDA_PREFIX, but there is no fallback from there!
-INSTALL_PREFIX=${INSTALL_PREFIX:=${PREFIX:=${CONDA_PREFIX}}}
+# If INSTALL_PREFIX is not set, check PREFIX, then check
+# CONDA_PREFIX, then fall back to install inside of $LIBRMM_BUILD_DIR
+INSTALL_PREFIX=${INSTALL_PREFIX:=${PREFIX:=${CONDA_PREFIX:=$LIBRMM_BUILD_DIR/install}}}
 PARALLEL_LEVEL=${PARALLEL_LEVEL:=""}
 
 function hasArg {
@@ -131,14 +131,15 @@ fi
 # Build and install the rmm Python package
 if (( NUMARGS == 0 )) || hasArg rmm; then
     cd "${REPODIR}/python"
+    export INSTALL_PREFIX
     if [[ ${INSTALL_TARGET} != "" ]]; then
         echo "building rmm..."
-        python setup.py build_ext --inplace --library-dir="${LIBRMM_BUILD_DIR}"
+        python setup.py build_ext --inplace
         echo "installing rmm..."
         python setup.py install --single-version-externally-managed --record=record.txt
     else
         echo "building rmm..."
-        python setup.py build_ext --inplace --library-dir="${LIBRMM_BUILD_DIR}"
+        python setup.py build_ext --inplace
     fi
 
 fi

python/setup.py

@@ -52,6 +52,13 @@ def get_cuda_version_from_header(cuda_include_dir):
 cuda_lib_dir = os.path.join(CUDA_HOME, "lib64")
 CUDA_VERSION = get_cuda_version_from_header(cuda_include_dir)
 
+INSTALL_PREFIX = os.environ.get("INSTALL_PREFIX", False)
+if os.path.isdir(INSTALL_PREFIX):
+    rmm_include_dir = os.path.join(INSTALL_PREFIX, "include")
+else:
+    # use uninstalled headers in source tree
+    rmm_include_dir = "../include"
+
 # Preprocessor step to specify correct pxd file with
 # valid symbols for specific version of CUDA.
 
@@ -78,9 +85,7 @@ def get_cuda_version_from_header(cuda_include_dir):
     nthreads = 0
 
 include_dirs = [
-    "../include/rmm",
-    "../include",
-    "../build/include",
+    rmm_include_dir,
     os.path.dirname(sysconfig.get_path("include")),
     cuda_include_dir,
 ]