Nightly Trilinos failures, all backends: error: no template named 'SerialTrsvInternalUpper' in namespace 'KokkosBatched' #2456

ndellingwood · 2024-12-12T17:39:20Z

Description:

Nightly builds of Trilinos fail to compile some packages (ifpack2, intrepid2) with output:

Intrepid2:

17:47:05 /home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp:548:11: error: no template named 'SerialTrsvInternalUpper' in namespace 'KokkosBatched'; did you mean 'KokkosBatched::Impl::SerialTrsvInternalUpper'?
17:47:05           KokkosBatched::SerialTrsvInternalUpper<KokkosBatched::Algo::Trsv::Unblocked>::invoke(false,
17:47:05           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
17:47:05           KokkosBatched::Impl::SerialTrsvInternalUpper
17:47:05 /home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/kokkos-kernels/batched/dense/impl/KokkosBatched_Trsv_Serial_Internal.hpp:131:8: note: 'KokkosBatched::Impl::SerialTrsvInternalUpper' declared here
17:47:05 struct SerialTrsvInternalUpper {
17:47:05        ^
17:47:05 In file included from /home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Build/packages/intrepid2/unit-test/Projection/test_project_fields_Serial_DOUBLE.cpp:15:
17:47:05 In file included from /home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/unit-test/Projection/test_project_fields.hpp:34:
17:47:05 /home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp:589:11: error: no template named 'SerialTrsvInternalUpper' in namespace 'KokkosBatched'; did you mean 'KokkosBatched::Impl::SerialTrsvInternalUpper'?
17:47:05           KokkosBatched::SerialTrsvInternalUpper<KokkosBatched::Algo::Trsv::Unblocked>::invoke(false,
17:47:05           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
17:47:05           KokkosBatched::Impl::SerialTrsvInternalUpper
17:47:05 /home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/kokkos-kernels/batched/dense/impl/KokkosBatched_Trsv_Serial_Internal.hpp:131:8: note: 'KokkosBatched::Impl::SerialTrsvInternalUpper' declared here
17:47:05 struct SerialTrsvInternalUpper {
17:47:05        ^

Ifpack2:

19:42:03 /home/jenkins/weaver/workspace/KokkosEco_Trilinos_Weaver_CUDA112_Shared_Cplx/Trilinos/packages/ifpack2/src/Ifpack2_BlockTriDiContainer_impl.hpp(4131): error: namespace "KokkosBatched" has no member "SerialTrsvInternalLower"
19:42:03           detected during:
19:42:03             instantiation of "void Ifpack2::BlockTriDiContainerDetails::SolveTridiags<MatrixType>::operator()(const Ifpack2::BlockTriDiContainerDetails::SolveTridiags<MatrixType>::SingleVectorTag<B> &, const Ifpack2::BlockTriDiContainerDetails::SolveTridiags<MatrixType>::member_type &) const [with MatrixType=Tpetra::RowMatrix<double, int, longlong, Tpetra_KokkosCompat_KokkosSerialWrapperNode>, B=3]" 
19:42:03 /home/jenkins/weaver/workspace/KokkosEco_Trilinos_Weaver_CUDA112_Shared_Cplx/Trilinos/kokkos/core/src/Serial/Kokkos_Serial_Parallel_Team.hpp(238): here
19:42:03             instantiation of "std::enable_if_t<<expression>, void> Kokkos::Impl::ParallelFor<FunctorType, Kokkos::TeamPolicy<Properties...>, Kokkos::Serial>::exec<TagType>(Kokkos::Impl::HostThreadTeamData &) const [with FunctorType=Ifpack2::BlockTriDiContainerDetails::SolveTridiags<Tpetra::RowMatrix<double, int, longlong, Tpetra_KokkosCompat_KokkosSerialWrapperNode>>, Properties=<Kokkos::HostSpace::execution_space, Ifpack2::BlockTriDiContainerDetails::SolveTridiags<Tpetra::RowMatrix<double, int, longlong, Tpetra_KokkosCompat_KokkosSerialWrapperNode>>::SingleVectorTag<3>>, TagType=Ifpack2::BlockTriDiContainerDetails::SolveTridiags<Tpetra::RowMatrix<double, int, longlong, Tpetra_KokkosCompat_KokkosSerialWrapperNode>>::SingleVectorTag<3>]" 
19:42:03 /home/jenkins/weaver/workspace/KokkosEco_Trilinos_Weaver_CUDA112_Shared_Cplx/Trilinos/kokkos/core/src/Serial/Kokkos_Serial_Parallel_Team.hpp(266): here
19:42:03             instantiation of "void Kokkos::Impl::ParallelFor<FunctorType, Kokkos::TeamPolicy<Properties...>, Kokkos::Serial>::execute() const [with FunctorType=Ifpack2::BlockTriDiContainerDetails::SolveTridiags<Tpetra::RowMatrix<double, int, longlong, Tpetra_KokkosCompat_KokkosSerialWrapperNode>>, Properties=<Kokkos::HostSpace::execution_space, Ifpack2::BlockTriDiContainerDetails::SolveTridiags<Tpetra::RowMatrix<double, int, longlong, Tpetra_KokkosCompat_KokkosSerialWrapperNode>>::SingleVectorTag<3>>]" 
19:42:03 /home/jenkins/weaver/workspace/KokkosEco_Trilinos_Weaver_CUDA112_Shared_Cplx/Trilinos/kokkos/core/src/Kokkos_Parallel.hpp(146): here
19:42:03             instantiation of "void Kokkos::parallel_for(const std::__cxx11::string &, const ExecPolicy &, const FunctorType &) [with ExecPolicy=Kokkos::TeamPolicy<Kokkos::HostSpace::execution_space, Ifpack2::BlockTriDiContainerDetails::SolveTridiags<Tpetra::RowMatrix<double, int, longlong, Tpetra_KokkosCompat_KokkosSerialWrapperNode>>::SingleVectorTag<3>>, FunctorType=Ifpack2::BlockTriDiContainerDetails::SolveTridiags<Tpetra::RowMatrix<double, int, longlong, Tpetra_KokkosCompat_KokkosSerialWrapperNode>>, Enable=void]" 
19:42:03 (4816): here
19:42:03             instantiation of "void Ifpack2::BlockTriDiContainerDetails::SolveTridiags<MatrixType>::run(const Ifpack2::BlockTriDiContainerDetails::SolveTridiags<MatrixType>::impl_scalar_type_2d_view_tpetra &, const Ifpack2::BlockTriDiContainerDetails::SolveTridiags<MatrixType>::impl_scalar_type_1d_view &) [with MatrixType=Tpetra::RowMatrix<double, int, longlong, Tpetra_KokkosCompat_KokkosSerialWrapperNode>]" 
19:42:03 (5007): here
19:42:03             instantiation of "int Ifpack2::BlockTriDiContainerDetails::applyInverseJacobi(const Teuchos::RCP<const Ifpack2::BlockHelperDetails::ImplType<MatrixType>::tpetra_row_matrix_type> &, const Teuchos::RCP<const Ifpack2::BlockHelperDetails::ImplType<MatrixType>::tpetra_crs_graph_type> &, const Teuchos::RCP<const Ifpack2::BlockHelperDetails::ImplType<MatrixType>::tpetra_import_type> &, const Teuchos::RCP<Ifpack2::BlockTriDiContainerDetails::AsyncableImport<MatrixType>> &, __nv_bool, const Ifpack2::BlockHelperDetails::ImplType<MatrixType>::tpetra_multivector_type &, Ifpack2::BlockHelperDetails::ImplType<MatrixType>::tpetra_multivector_type &, Ifpack2::BlockHelperDetails::ImplType<MatrixType>::tpetra_multivector_type &, Ifpack2::BlockHelperDetails::ImplType<MatrixType>::impl_scalar_type_1d_view &, const Ifpack2::BlockHelperDetails::PartInterface<MatrixType> &, const Ifpack2::BlockTriDiContainerDetails::BlockTridiags<MatrixType> &, const Ifpack2::BlockHelperDetails::AmD<MatrixType> &, Ifpack2::BlockHelperDetails::ImplType<MatrixType>::vector_type_1d_view &, Ifpack2::BlockHelperDetails::NormManager<MatrixType> &, const Ifpack2::BlockHelperDetails::ImplType<MatrixType>::impl_scalar_type &, __nv_bool, int, Ifpack2::BlockHelperDetails::ImplType<MatrixType>::magnitude_type, int) [with MatrixType=Tpetra::RowMatrix<double, int, longlong, Tpetra_KokkosCompat_KokkosSerialWrapperNode>]" 
19:42:03 /home/jenkins/weaver/workspace/KokkosEco_Trilinos_Weaver_CUDA112_Shared_Cplx/Trilinos/packages/ifpack2/src/Ifpack2_BlockTriDiContainer_def.hpp(317): here
19:42:03             instantiation of "void Ifpack2::BlockTriDiContainer<MatrixType, Ifpack2::BlockTriDiContainerDetails::ImplSimdTag>::applyInverseJacobi(const Ifpack2::BlockTriDiContainer<MatrixType, Ifpack2::BlockTriDiContainerDetails::ImplSimdTag>::mv_type &, Ifpack2::BlockTriDiContainer<MatrixType, Ifpack2::BlockTriDiContainerDetails::ImplSimdTag>::mv_type &, Ifpack2::BlockTriDiContainer<MatrixType, Ifpack2::BlockTriDiContainerDetails::ImplSimdTag>::scalar_type, __nv_bool, int) const [with MatrixType=Tpetra::RowMatrix<double, int, longlong, Tpetra_KokkosCompat_KokkosSerialWrapperNode>]" 
19:42:03 /home/jenkins/weaver/workspace/KokkosEco_Trilinos_Weaver_CUDA112_Shared_Cplx/Build/packages/ifpack2/src/Ifpack2_BlockTriDiContainer_DOUBLE_INT_LONG_LONG_SERIAL.cpp(24): here
...

These errors began following merge of #2452 .

Trilinos builds enable deprecated code, I'm not sure if that indicates an inadvertent compatibility change that should be addressed, or if changes should be made in Trilinos itself (which will be necessary regardless, I suppose)

Reproducer (blake all queue):

# Repo prep
git clone -b develop https://github.com/trilinos/Trilinos.git
TRILINOS_DIR=$PWD/Trilinos
git clone -b develop https://github.com/kokkos/kokkos.git
KOKKOS_DIR=$PWD/kokkos
git clone -b develop https://github.com/kokkos/kokkos-kernels.git
KOKKOSKERNELS_DIR=$PWD/kokkos-kernels

cd $TRILINOS_DIR
ln -s ${KOKKOS_DIR}/kokkos kokkos
ln -s ${KOKKOSKERNELS_DIR}/kokkos-kernels kokkos-kernels
cd -

mkdir -p Build
cd Build

salloc -N 1 -p all

# Environment
module purge
module load cmake llvm openblas/0.3.23
module list

# Configure Trilinos
cmake \
  -D CMAKE_INSTALL_PREFIX="${TRILINOS_INSTALL_DIR}" \
  -D CMAKE_CXX_COMPILER="`which clang++`" \
  -D CMAKE_C_COMPILER="`which clang`" \
  -D CMAKE_Fortran_COMPILER="`which gfortran`" \
  -D CMAKE_CXX_FLAGS="-g -fPIC" \
  -D CMAKE_C_FLAGS="-g -fPIC" \
  -D CMAKE_Fortran_FLAGS="-g -fPIC" \
  -D CMAKE_BUILD_TYPE:STRING=DEBUG \
  -D BUILD_SHARED_LIBS:BOOL=OFF \
  -DTPL_ENABLE_MPI=OFF \
  -DTPL_ENABLE_BLAS:BOOL=ON \
   -DBLAS_LIBRARY_DIRS:FILEPATH=${OPENBLAS_ROOT}/lib \
   -DBLAS_LIBRARY_NAMES:STRING="openblas" \
  -DTPL_ENABLE_LAPACK:BOOL=ON \
   -DLAPACK_INCLUDE_DIRS:FILEPATH=${OPENBLAS_ROOT}/include \
   -DLAPACK_LIBRARY_DIRS:FILEPATH=${OPENBLAS_ROOT}/lib \
   -DLAPACK_LIBRARY_NAMES:STRING="openblas" \
  -DTrilinos_ENABLE_ALL_PACKAGES=OFF \
  -DTrilinos_ENABLE_ALL_OPTIONAL_PACKAGES=OFF \
  -DTrilinos_ENABLE_TESTS=ON \
  -DTrilinos_MUST_FIND_ALL_TPL_LIBS=TRUE \
  -DTrilinos_ENABLE_COMPLEX=ON \
  -DTrilinos_ENABLE_OpenMP=OFF \
  -DTrilinos_ENABLE_Kokkos=ON \
  -D Kokkos_ENABLE_SERIAL=ON \
   -D Kokkos_ENABLE_TESTS=ON \
  -D Kokkos_ARCH_SKX=ON \
   -D Kokkos_ENABLE_IMPL_MDSPAN=ON \
  -DTrilinos_ENABLE_KokkosKernels=ON \
   -D KokkosKernels_ENABLE_TESTS=ON \
  -DTrilinos_ENABLE_Tpetra=ON \
   -D Tpetra_ENABLE_TESTS=ON \
  -DTrilinos_ENABLE_Sacado=ON \
   -D Sacado_ENABLE_TESTS=ON \
  -DTrilinos_ENABLE_Stokhos=ON \
   -D Stokhos_ENABLE_TESTS=ON \
  -DTrilinos_ENABLE_Intrepid2=ON \
   -D Intrepid2_ENABLE_TESTS=ON \
  -DTrilinos_ENABLE_Adelus=ON \
   -D Adelus_ENABLE_TESTS:BOOL=ON \
  -DTrilinos_ENABLE_Compadre=ON \
   -D Compadre_ENABLE_TESTS=ON \
  -DTrilinos_ENABLE_Panzer=ON \
   -D Panzer_ENABLE_TESTS=ON \
  -DTrilinos_ENABLE_Amesos2=ON \
   -D Amesos2_ENABLE_TESTS=ON \
  -DTrilinos_ENABLE_Zoltan2=ON \
   -D Zoltan2_ENABLE_TESTS=ON \
  -DTrilinos_ENABLE_Teuchos=ON \
   -D Teuchos_ENABLE_TESTS=ON \
  -DTrilinos_ENABLE_Ifpack2=ON \
   -D Ifpack2_ENABLE_TESTS=ON \
\
  -DTPL_ENABLE_Matio=OFF \
\
-DKokkos_SOURCE_DIR_OVERRIDE:STRING=kokkos \
-DKokkosKernels_SOURCE_DIR_OVERRIDE:STRING=kokkos-kernels \
\
$TRILINOS_DIR

The text was updated successfully, but these errors were encountered:

lucbv · 2024-12-12T18:02:36Z

Yes, I think we moved some implementation details into the Impl namespace. Glad to see that Trilinos is using said details directly instead of the public API...
When I have time I will go and re-work the change so that the functions that moved will be marked as deprecated and call in the functions from the Impl namespace. Sorry about that!

yasahi-hpc · 2024-12-13T06:44:33Z

@ndellingwood @lucbv
Sorry, I made some mistakes.
I think for Intrepid2, the following fix in Trillinos side is preferable. For Ifpack2, I made a hot-fix PR to restore the aliases.

https://github.com/trilinos/Trilinos/blob/3e7486f75a306db72a7136e21fdc4f8302c29f16/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp#L548-L552

Current

KokkosBatched::SerialTrsvInternalUpper<KokkosBatched::Algo::Trsv::Unblocked>::invoke(false,
    A0.extent(0),
    1.0,
    A0.data(), A0.stride_0(), A0.stride_1(),
    b.data(),  b.stride_0());

Updated

KokkosBatched::SerialTrsv<KokkosBatched::Uplo::Upper, KokkosBatched::Trans::NoTranspose,
KokkosBatched::Diag::NonUnit, KokkosBatched::Algo::Trsv::Unblocked>::invoke(1.0, A0, b);

lucbv · 2024-12-13T15:15:17Z

@yasahi-hpc what you suggest above makes sense however I would also add the following at the end of KokkosBatched_Trsv_Serial_Internal.hpp:

namespace KokkosBatched {
namespace Impl {

...

}  // namespace Impl

template <typename AlgoType>
struct [[deprecated("Don't touch my impl stuff")]] SerialTrsvInternalLower {

  template <typename ScalarType, typename ValueType>
  KOKKOS_INLINE_FUNCTION static int invoke(const bool use_unit_diag, const bool do_conj, const int m,
                                           const ScalarType alpha, const ValueType *KOKKOS_RESTRICT A, const int as0,
                                           const int as1,
                                           /**/ ValueType *KOKKOS_RESTRICT b, const int bs0) {
    KokkosBatched::Impl::SerialTrsvInternalLower<AlgoType>::invoke<ScalarType, ValueType>(use_unit_diag, do_conj, m, alpha, A, as0, as1, b, bs0);
  }

};

}  // namespace KokkosBatched

That way users that may be relying on this will get some time to react to our move and should be able to fix things before we actually remove the non-Impl version completely.

Address issue kokkos/kokkos-kernels#2456 (comment) Co-authored-by: Yuuichi Asahi <[email protected]> Signed-off-by: Nathan Ellingwood <[email protected]>

ndellingwood · 2024-12-14T04:07:39Z

@yasahi-hpc thanks for the suggestion, I tried testing the intrepid2 change with Trilinos (created Draft PR here trilinos/Trilinos#13671) , but I am still seeing issues:

20:36:49 /home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp:549:26: error: no member named 'SerialTrsv' in namespace 'KokkosBatched'
20:36:49 In file included from /home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Build/packages/intrepid2/unit-test/Projection/test_interpolation_projection_HEX_Serial_DOUBLE.cpp:15:
20:36:49 In file included from /home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/unit-test/Projection/test_interpolation_projection_HEX.hpp:40:
20:36:49 /home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp:549:26:           KokkosBatched::SerialTrsv<KokkosBatched::Uplo::Upper, KokkosBatched::Trans::NoTranspose, KokkosBatched::Diag::NonUnit, KokkosBatched::Algo::Trsv::Unblocked>::invoke(1.0, A0, b);error: 
20:36:49           ~~~~~~~~~~~~~~~^no member named 'SerialTrsv' in namespace 'KokkosBatched'

The line 549 refers to the Trilinos PR trilinos/Trilinos#13671 , can you help advise?

yasahi-hpc · 2024-12-14T05:02:57Z

@ndellingwood Sorry for the inconvenience.
I think we need to change the header file as well.

Current

#ifdef HAVE_INTREPID2_KOKKOSKERNELS
#include "KokkosBatched_QR_Serial_Internal.hpp"
#include "KokkosBatched_ApplyQ_Serial_Internal.hpp"
#include "KokkosBatched_Trsv_Serial_Internal.hpp"
#include "KokkosBatched_Util.hpp"
#endif

Updated

#ifdef HAVE_INTREPID2_KOKKOSKERNELS
#include "KokkosBatched_QR_Serial_Internal.hpp"
#include "KokkosBatched_ApplyQ_Serial_Internal.hpp"
#if KOKKOS_VERSION >= 40599
#include "KokkosBatched_Trsv_Decl.hpp"
#else
#include "KokkosBatched_Trsv_Serial_Internal.hpp"
#endif
#include "KokkosBatched_Util.hpp"
#endif

yasahi-hpc · 2024-12-14T06:01:03Z

@lucbv I made a PR to add deprecated remarks. May I have your review please

Address issue kokkos/kokkos-kernels#2456 (comment) Co-authored-by: Yuuichi Asahi <[email protected]> Signed-off-by: Nathan Ellingwood <[email protected]>

ndellingwood · 2024-12-14T23:36:47Z

Thanks @yasahi-hpc for the fast response! I updated the PR and will let you know how the next run goes

ndellingwood · 2024-12-16T15:57:55Z

@yasahi-hpc unfortunately I am still seeing compilation issues with intrepid2 with these changes https://github.com/trilinos/Trilinos/pull/13671/files

17:43:20 /home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp/home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/unit-test/Projection/test_convergence_QUAD.hpp41::31In file included from :
17:43:20 :/home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpperror: :no template named 'SerialTrsvInternalUpper' in namespace 'KokkosBatched'; did you mean 'KokkosBatched::Impl::SerialTrsvInternalUpper'?11:
17:43:20 :/home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp::/home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp::
17:43:20 11:
17:43:20 /home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/unit-test/Projection/test_interpolation_projection_TET.hpp/home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp44:
17:43:20 :44:
17:43:20 :41:11597/home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp::::
17:43:20 597 :/home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp597:
17:43:20 597:::597 40597/home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpperror: 11::/home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp :error: 11no template named 'SerialTrsvInternalUpper' in namespace 'KokkosBatched'; did you mean 'KokkosBatched::Impl::SerialTrsvInternalUpper'?::11error: :
17:43:20 :::no template named 'SerialTrsvInternalUpper' in namespace 'KokkosBatched'; did you mean 'KokkosBatched::Impl::SerialTrsvInternalUpper'? 597error: :11:

yasahi-hpc · 2024-12-16T18:07:38Z

@yasahi-hpc unfortunately I am still seeing compilation issues with intrepid2 with these changes https://github.com/trilinos/Trilinos/pull/13671/files

17:43:20 /home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp/home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/unit-test/Projection/test_convergence_QUAD.hpp41::31In file included from :
17:43:20 :/home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpperror: :no template named 'SerialTrsvInternalUpper' in namespace 'KokkosBatched'; did you mean 'KokkosBatched::Impl::SerialTrsvInternalUpper'?11:
17:43:20 :/home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp::/home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp::
17:43:20 11:
17:43:20 /home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/unit-test/Projection/test_interpolation_projection_TET.hpp/home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp44:
17:43:20 :44:
17:43:20 :41:11597/home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp::::
17:43:20 597 :/home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp597:
17:43:20 597:::597 40597/home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpperror: 11::/home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp :error: 11no template named 'SerialTrsvInternalUpper' in namespace 'KokkosBatched'; did you mean 'KokkosBatched::Impl::SerialTrsvInternalUpper'?::11error: :
17:43:20 :::no template named 'SerialTrsvInternalUpper' in namespace 'KokkosBatched'; did you mean 'KokkosBatched::Impl::SerialTrsvInternalUpper'? 597error: :11:

@ndellingwood
Sorry, I just found that we need to make a similar change for lines 589-593

https://github.com/trilinos/Trilinos/blob/7f913e35edaa2f6b7e2d6b67e966574db5209a79/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp#L589-L593

Current

KokkosBatched::SerialTrsvInternalUpper<KokkosBatched::Algo::Trsv::Unblocked>::invoke(false,
              A.extent(0),
              1.0,
              A.data(), A.stride_0(), A.stride_1(),
              b.data(),  b.stride_0());

Updated

#if KOKKOS_VERSION >= 40599
KokkosBatched::SerialTrsv<KokkosBatched::Uplo::Upper, KokkosBatched::Trans::NoTranspose, KokkosBatched::Diag::NonUnit, KokkosBatched::Algo::Trsv::Unblocked>::invoke(1.0, A, b);
#else
KokkosBatched::SerialTrsvInternalUpper<KokkosBatched::Algo::Trsv::Unblocked>::invoke(false,
              A.extent(0),
              1.0,
              A.data(), A.stride_0(), A.stride_1(),
              b.data(),  b.stride_0());
#endif

Address issue kokkos/kokkos-kernels#2456 (comment) Co-authored-by: Yuuichi Asahi <[email protected]> Signed-off-by: Nathan Ellingwood <[email protected]>

ndellingwood · 2024-12-16T19:55:31Z

Thanks @yasahi-hpc , that helped make some progress. There is now a new bump:

12:41:03 In file included from /home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp:54:
12:41:03 In file included from /home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/kokkos-kernels/batched/dense/src/KokkosBatched_Trsv_Decl.hpp:92:
12:41:03 /home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/kokkos-kernels/batched/dense/impl/KokkosBatched_Trsv_Serial_Impl.hpp:30:3: error: static assertion failed due to requirement 'Kokkos::is_view_v<Kokkos::DynRankView<double, Kokkos::LayoutStride, Kokkos::Device<Kokkos::Serial, Kokkos::HostSpace>, Kokkos::MemoryTraits<0>>>': KokkosBatched::trsv: AViewType is not a Kokkos::View.
12:41:03   static_assert(Kokkos::is_view_v<AViewType>, "KokkosBatched::trsv: AViewType is not a Kokkos::View.");
12:41:03   ^             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
12:41:03 /home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/kokkos-kernels/batched/dense/impl/KokkosBatched_Trsv_Serial_Impl.hpp:288:38: note: in instantiation of function template specialization 'KokkosBatched::Impl::checkTrsvInput<Kokkos::DynRankView<double, Kokkos::LayoutStride, Kokkos::Device<Kokkos::Serial, Kokkos::HostSpace>, Kokkos::MemoryTraits<0>>, Kokkos::DynRankView<double, Kokkos::LayoutStride, Kokkos::Device<Kokkos::Serial, Kokkos::HostSpace>, Kokkos::MemoryTraits<0>>>' requested here
12:41:03     auto info = KokkosBatched::Impl::checkTrsvInput(A, b);
12:41:03                                      ^
12:41:03 /home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_LLVM1507_Serial_debug/Trilinos/packages/intrepid2/src/Projection/Intrepid2_ProjectionTools.hpp:553:169: note: in instantiation of function template specialization 'KokkosBatched::SerialTrsv<KokkosBatched::Uplo::Upper, KokkosBlas::Trans::NoTranspose, KokkosBatched::Diag::NonUnit, KokkosBlas::Algo::Level2::Unblocked>::invoke<double, Kokkos::DynRankView<double, Kokkos::LayoutStride, Kokkos::Device<Kokkos::Serial, Kokkos::HostSpace>, Kokkos::MemoryTraits<0>>, Kokkos::DynRankView<double, Kokkos::LayoutStride, Kokkos::Device<Kokkos::Serial, Kokkos::HostSpace>, Kokkos::MemoryTraits<0>>>' requested here
12:41:03           KokkosBatched::SerialTrsv<KokkosBatched::Uplo::Upper, KokkosBatched::Trans::NoTranspose, KokkosBatched::Diag::NonUnit, KokkosBatched::Algo::Trsv::Unblocked>::invoke(1.0, A0, b);
...

From the error message it looks like the updates constrained the algorithm to View, if so is it possible to relax the static_assert check to allow use of DynRankView as well? Unfortunately in this case, the rank check would need to be at runtime

lucbv · 2024-12-16T20:01:34Z

We might want to update our unit tests to cover these things a bit better, that would avoid all the back and force and improve the library robustness.

yasahi-hpc · 2024-12-17T06:51:59Z

Hi @ndellingwood @lucbv

I have relaxed the static_assert check in PR #2464. Could you please try this?

ndellingwood added bug bug develop-only labels Dec 12, 2024

yasahi-hpc mentioned this issue Dec 13, 2024

fix: alias for serial trsv #2458

Merged

lucbv closed this as completed in #2458 Dec 13, 2024

ndellingwood mentioned this issue Dec 14, 2024

intrepid2: compatibility update with kokkos-kernels trilinos/Trilinos#13671

Draft

ndellingwood reopened this Dec 14, 2024

yasahi-hpc mentioned this issue Dec 14, 2024

Deprecated remarks to trsv serial impl #2461

Merged

2 tasks

yasahi-hpc mentioned this issue Dec 17, 2024

Allow dyn-rank-view in serial trsv #2464

Open

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Nightly Trilinos failures, all backends: error: no template named 'SerialTrsvInternalUpper' in namespace 'KokkosBatched' #2456

Nightly Trilinos failures, all backends: error: no template named 'SerialTrsvInternalUpper' in namespace 'KokkosBatched' #2456

ndellingwood commented Dec 12, 2024

lucbv commented Dec 12, 2024

yasahi-hpc commented Dec 13, 2024

lucbv commented Dec 13, 2024

ndellingwood commented Dec 14, 2024

yasahi-hpc commented Dec 14, 2024

yasahi-hpc commented Dec 14, 2024

ndellingwood commented Dec 14, 2024

ndellingwood commented Dec 16, 2024

yasahi-hpc commented Dec 16, 2024 •

edited

Loading

ndellingwood commented Dec 16, 2024

lucbv commented Dec 16, 2024

yasahi-hpc commented Dec 17, 2024

Nightly Trilinos failures, all backends: error: no template named 'SerialTrsvInternalUpper' in namespace 'KokkosBatched' #2456

Nightly Trilinos failures, all backends: error: no template named 'SerialTrsvInternalUpper' in namespace 'KokkosBatched' #2456

Comments

ndellingwood commented Dec 12, 2024

lucbv commented Dec 12, 2024

yasahi-hpc commented Dec 13, 2024

lucbv commented Dec 13, 2024

ndellingwood commented Dec 14, 2024

yasahi-hpc commented Dec 14, 2024

yasahi-hpc commented Dec 14, 2024

ndellingwood commented Dec 14, 2024

ndellingwood commented Dec 16, 2024

yasahi-hpc commented Dec 16, 2024 • edited Loading

ndellingwood commented Dec 16, 2024

lucbv commented Dec 16, 2024

yasahi-hpc commented Dec 17, 2024

yasahi-hpc commented Dec 16, 2024 •

edited

Loading