[global_batched_kokkos] Kokkos build failure

[PDoakORNL]

Using gcc 6.5.0 on x86

[33/61] Building CXX object kokkos/CMakeFiles/kokkos.dir/core/src/impl/Kokkos_HostSpace.cpp.o
FAILED: /lustre/or-hydra/cades-cnms/epd/dev/kokkos/bin/nvcc_wrapper -I/lustre/or-hydra/cades-cnms/epd/dev/kokkos/core/src -I/lustre/or-hydra/cades-cnms/epd/dev/kokkos/containers/src -I/lustre/or-hydra/cades-cnms/epd/dev/kokkos/algorithms/src -Ikokkos -I/software/dev_tools/swtree/cs400_centos7.2_pe2016-08/cuda/9.2/centos7.2_binary/include -Drestrict=restrict -D__forceinline=inline -O3 -DNDEBUG --std=c++11 -Xcudafe --diag_suppress=esa_on_defaulted_function_ignored -expt-extended-lambda -march=core-avx2 -mtune=core-avx2 -arch=sm_60 -MD -MT kokkos/CMakeFiles/kokkos.dir/core/src/impl/Kokkos_HostSpace.cpp.o -MF kokkos/CMakeFiles/kokkos.dir/core/src/impl/Kokkos_HostSpace.cpp.o.d -o kokkos/CMakeFiles/kokkos.dir/core/src/impl/Kokkos_HostSpace.cpp.o -c /lustre/or-hydra/cades-cnms/epd/dev/kokkos/core/src/impl/Kokkos_HostSpace.cpp
/software/user_tools/centos-7.2.1511/cades-cnms/spack/opt/spack/linux-centos7-x86_64/gcc-8.2.0/gcc-6.5.0-egooyqwfmyg6msi5xykwsvniotp774yx/lib/gcc/x86_64-pc-linux-gnu/6.5.0/include/avx512fintrin.h(12943): error: identifier "__builtin_ia32_kmov16" is undefined

[lshulen]

Are you building Kokkos inline? You should change the KOKKOS_ARCH to HSW in your cmake.

[PDoakORNL]

Yes building inline, lets table the AVX512 intrinsic renaming I still have issue with openmp=true and cuda=false

now I get:

[1/52] Building CXX object kokkos/CMakeFiles/kokkos.dir/core/src/impl/Kokkos_Spinwait.cpp.o
FAILED: /lustre/or-hydra/cades-cnms/epd/dev/kokkos/bin/nvcc_wrapper   -I/lustre/or-hydra/cades-cnms/epd/dev/kokkos/core/src -I/lustre/or-hydra/cades-cnms/epd/dev/kokkos/containers/src -I/lustre/or-hydra/cades-cnms/epd/dev/kokkos/algorithms/src -Ikokkos -Drestrict=__restrict__ -D__forceinline=inline -O3 -DNDEBUG   --std=c++11 -march=core-avx2 -mtune=core-avx2 -Xcompiler -fopenmp -MD -MT kokkos/CMakeFiles/kokkos.dir/core/src/impl/Kokkos_Spinwait.cpp.o -MF kokkos/CMakeFiles/kokkos.dir/core/src/impl/Kokkos_Spinwait.cpp.o.d -o kokkos/CMakeFiles/kokkos.dir/core/src/impl/Kokkos_Spinwait.cpp.o -c /lustre/or-hydra/cades-cnms/epd/dev/kokkos/core/src/impl/Kokkos_Spinwait.cpp
/lustre/or-hydra/cades-cnms/epd/dev/kokkos/core/src/impl/Kokkos_Atomic_Generic.hpp(360): warning: a __device__ function("Kokkos::atomic_fetch_and") redeclared without __device__, hence treated as a __host__ __device__ function

/lustre/or-hydra/cades-cnms/epd/dev/kokkos/core/src/impl/Kokkos_Atomic_Generic.hpp(360): error: function template "Kokkos::atomic_fetch_and" has already been defined

/lustre/or-hydra/cades-cnms/epd/dev/kokkos/core/src/impl/Kokkos_Atomic_Generic.hpp(366): warning: a __device__ function("Kokkos::atomic_fetch_or") redeclared without __device__, hence treated as a __host__ __device__ function

/lustre/or-hydra/cades-cnms/epd/dev/kokkos/core/src/impl/Kokkos_Atomic_Generic.hpp(366): error: function template "Kokkos::atomic_fetch_or" has already been defined

here's my actual cmake

rm -rf *; LDFLAGS="-L/software/user_tools/centos-7.2.1511/cades-cnms/spack/opt/spack/linux-centos7-x86_64/gcc-8.2.0/gcc-6.5.0-egooyqwfmyg6msi5xykwsvniotp774yx/lib6 -Wl,-rpath,/software/user_tools/centos-7.2.1511/cades-cnms/spack/opt/spack/linux-centos7-x86_64/gcc-8.2.0/gcc-6.5.0-egooyqwfmyg6msi5xykwsvniotp774yx/lib6"\
 cmake -DQMC_USE_KOKKOS=1 -DKOKKOS_PREFIX="/lustre/or-hydra/cades-cnms/epd/dev/kokkos" \
-DKOKKOS_ENABLE_CUDA=false -DKOKKOS_ENABLE_OPENMP=true -DKOKKOS_ARCH="HSW;Pascal60" \
-DKOKKOS_ENABLE_CUDA_UVM=true -DKOKKOS_ENABLE_CUDA_LAMBDA=true \
-DKOKKOS_ENABLE_EXPLICIT_INSTANTIATION=false -DCMAKE_CXX_COMPILER=/lustre/or-hydra/cades-cnms/epd/dev/kokkos/bin/nvcc_wrapper \
-DCMAKE_CXX_FLAGS="-Drestrict=__restrict__ -D__forceinline=inline " -GNinja ..

[ye-luo]

Could you change the title starting with [branch name]

[lshulen]

Problem looks to be that you are using nvcc_wrapper to build the code with KOKKOS_ENABLE_CUDA=false. Try building again with gcc directly.

[PDoakORNL]

tried that, get this:

FAILED: /software/user_tools/centos-7.2.1511/cades-cnms/spack/opt/spack/linux-centos7-x86_64/gcc-8.2.0/gcc-6.5.0-egooyqwfmyg6msi5xykwsvniotp774yx/bin/g++  -DADD_ -DH5_USE_16_API -DHAVE_CONFIG_H -DINLINE_ALL=inline -DUSE_REAL_STRUCT_FACTOR -I/lustre/or-hydra/cades-cnms/epd/dev/kokkos/core/src -I/lustre/or-hydra/cades-cnms/epd/dev/kokkos/containers/src -I/lustre/or-hydra/cades-cnms/epd/dev/kokkos/algorithms/src -Ikokkos -I../src -Isrc -Drestrict=__restrict__ -D__forceinline=inline -O3 -DNDEBUG   --std=c++11 -march=core-avx2 -mtune=core-avx2 -fopenmp -std=c++11 -MD -MT src/Drivers/CMakeFiles/miniqmc.dir/miniqmc.cpp.o -MF src/Drivers/CMakeFiles/miniqmc.dir/miniqmc.cpp.o.d -o src/Drivers/CMakeFiles/miniqmc.dir/miniqmc.cpp.o -c ../src/Drivers/miniqmc.cpp
../src/Drivers/miniqmc.cpp: In lambda function:
../src/Drivers/miniqmc.cpp:406:13: error: missing template arguments before 'thiswalker'
       Mover thiswalker(myPrimes[teamID], ions);
             ^~~~~~~~~~
../src/Drivers/miniqmc.cpp:408:7: error: 'thiswalker' was not declared in this scope
       thiswalker.spo = build_SPOSet_view(useRef, spo_main, team_size, teamID);
       ^~~~~~~~~~
[51/59] Building CXX object src/Drivers/CMakeFiles/miniqmc_sync_move.dir/miniqmc_sync_move.cpp.o
FAILED: /software/user_tools/centos-7.2.1511/cades-cnms/spack/opt/spack/linux-centos7-x86_64/gcc-8.2.0/gcc-6.5.0-egooyqwfmyg6msi5xykwsvniotp774yx/bin/g++  -DADD_ -DH5_USE_16_API -DHAVE_CONFIG_H -DINLINE_ALL=inline -DUSE_REAL_STRUCT_FACTOR -I/lustre/or-hydra/cades-cnms/epd/dev/kokkos/core/src -I/lustre/or-hydra/cades-cnms/epd/dev/kokkos/containers/src -I/lustre/or-hydra/cades-cnms/epd/dev/kokkos/algorithms/src -Ikokkos -I../src -Isrc -Drestrict=__restrict__ -D__forceinline=inline -O3 -DNDEBUG   --std=c++11 -march=core-avx2 -mtune=core-avx2 -fopenmp -std=c++11 -MD -MT src/Drivers/CMakeFiles/miniqmc_sync_move.dir/miniqmc_sync_move.cpp.o -MF src/Drivers/CMakeFiles/miniqmc_sync_move.dir/miniqmc_sync_move.cpp.o.d -o src/Drivers/CMakeFiles/miniqmc_sync_move.dir/miniqmc_sync_move.cpp.o -c ../src/Drivers/miniqmc_sync_move.cpp
../src/Drivers/miniqmc_sync_move.cpp: In function 'int main(int, char**)':
../src/Drivers/miniqmc_sync_move.cpp:586:77: error: no matching function for call to 'qmcplusplus::ParticleSet::multi_makeMoveAndCheckKokkos(std::vector<qmcplusplus::ParticleSet*>&, Kokkos::View<double* [3]>&, int&, std::vector<int>&)'
        anon_mover->els.multi_makeMoveAndCheckKokkos(P_list, dr, iel, isValid);
...

[lshulen]

Perfect. I need to update the CMakeLists.txt, The only executable you can build at the moment is miniqmc_sync_move_noref .