With some hacking, we can start attempting to build hypre with CUDA
support (private build, or perhaps a separate hypre package in contrib).

Brief notes:

- The version of nvcc (nvidia-cuda-toolkit) in debian non-free is 8.0.61-3
nvcc does not yet support gcc 7, and nvcc 8 does not support gcc 6
To also use mpiCC (so mpi.h is known, etc), need OMPI_CXX
So need
  CC=nvcc CUDACXX=mpiCC dh_auto_configure -- $(extra_flags) --enable-shared

- nvcc does not recognise dpkg-buildflags flags (-W flags in
particular), so need to set an empty LDFLAGS and friends

- must hack src/config/configure.in where nvcc flags are defined,
replace -fPIC with --compiler-options '-fPIC'

- build with g++-5  fails at src/utilities/gpuMem.c:
$nvcc -O --compiler-options '-fPIC' -DUSE_NVTX -DHYPRE_USE_GPU -DHYPRE_USE_MANAGED  -ccbin=mpiCC -expt-extended-lambda -arch compute_35 --std=c++11 -Xcompiler -fopenmp -Xcompiler   -DHYPRE_USE_CUDA  -DUSE_NVTX -DHYPRE_USE_GPU  -DHYPRE_USE_MANAGED -I /usr/local/cuda/include  -DHAVE_CONFIG_H -I.. -I./.. -I.    -c gpuMem.c
  gpuMem.c: In function ‘getstreamOlde’:
  gpuMem.c:335:23: error: storage size of ‘s’ isn’t constant
     static cudaStream_t s[MAXSTREAMS];
                         ^
  gpuMem.c: In function ‘getdomain’:
  gpuMem.c:351:31: error: storage size of ‘h’ isn’t constant
       static nvtxDomainHandle_t h[MAXDOMAINS];
                                 ^
  gpuMem.c: In function ‘getevent’:
  gpuMem.c:364:22: error: storage size of ‘s’ isn’t constant
     static cudaEvent_t s[MAXEVENTS];


This gpuMem.c does compile cleanly under OMPI_CXX=clang++-3.8
Pass through using export OMPI_CXX=clang++-3.8 in debian/rules.

Finally then, using
  export OMPI_CXX=clang++-3.8
  CC=nvcc CUDACXX=mpiCC dh_auto_configure -- $(extra_flags) --with-cuda --enable-unified-memory --with-nvcc --enable-shared

the libHYPRE_utilities.so build proceeds but fails at link time
providing the soname,
  nvcc -shared -o libHYPRE_utilities-2.13.0.so amg_linklist.o binsearch.o exchange_data.o fortran_matrix.o hypre_ap.o hypre_complex.o hypre_memory.o hypre_printf.o hypre_qsort.o memory_dmalloc.o mpistubs.o qsplit.o random.o threading.o timer.o timing.o umalloc_local.o hypre_mpi_comm_f2c.o hypre_error.o F90_HYPRE_error.o hypre_prefix_sum.o hypre_merge_sort.o hypre_hopscotch_hash.o gpuErrorCheck.o gpuMem.o -lmpi -lm -Wl,-soname,libHYPRE_utilities-2.13.0.so -Wl,-z,defs
  nvcc fatal   : Unknown option 'Wl,-soname,libHYPRE_utilities-2.13.0.so'

Again, the problem is that -W is not recognised by nvcc.  Would have
to hack the soname patch to encapsulate it inside --compiler-options

Is it worth it?

Can get further using nvcc --linker-options to carry -Wl options.

But "CC=mpiCC CUDACXX=mpiCC dh_auto_configure" seems better than
"CC=nvcc", together with export OMPI_CXX=g++-5 to set mpiCC.

With other sundry hacks, gets to src/struct_mv and stalls at
assumed_part.c, apparently because of inconsistencies in C/C++
standards between nvcc and CC.Depending on whether g++-5 or clang++-3.8
is used, whether -std=c++11 or c++14 or c++98 is passed to gcc, get
either

 - conflict in multiple definitions of isnan, conflicting between
CUDA's /usr/include/math_functions.h and /usr/include/x86_64-linux-
gnu/bits/mathcalls.h

 - or similar multiple definitions in abs(float)

 - or  /usr/include/cuda_fp16.h:985:68: error: C does not support default arguments
__CUDA_FP16_DECL__ __half2 __shfl(__half2 var, int delta, int width=warpSize);

- or inconsistent C/C++ definitions
    ./_hypre_struct_mv.h:1464:8: error: expected identifier or '('
    extern "C++" {
           ^
    ./_hypre_struct_mv.h:1488:22: error: unexpected OpenMP directive '#pragma omp parallel for'
    #pragma omp parallel for schedule(static)