Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build cpp test fails with GCC at v4.2.0 #6252

Open
natsukium opened this issue Dec 28, 2023 · 6 comments · May be fixed by #6315
Open

build cpp test fails with GCC at v4.2.0 #6252

natsukium opened this issue Dec 28, 2023 · 6 comments · May be fixed by #6315
Assignees
Labels

Comments

@natsukium
Copy link

natsukium commented Dec 28, 2023

Description

When I tried to build cpp_test with gcc, I got the following error.
error: explicit specialization in non-namespace scope ‘class ArrowChunkedArrayTest’
error: template-id ‘create_primitive_schema<float>’ in declaration of primary template

[ 97%] Built target _lightgbm
/LightGBM/tests/cpp_tests/test_arrow.cpp:95:13: error: explicit specialization in non-namespace scope ‘class ArrowChunkedArrayTest’
   95 |   template <>
      |             ^
/LightGBM/tests/cpp_tests/test_arrow.cpp:96:46: error: template-id ‘create_primitive_schema<float>’ in declaration of primary template
   96 |   ArrowSchema create_primitive_schema<float>() {
      |                                              ^
make[2]: *** [CMakeFiles/testlightgbm.dir/build.make:76: CMakeFiles/testlightgbm.dir/tests/cpp_tests/test_arrow.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:148: CMakeFiles/testlightgbm.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 98%] Linking CXX executable ../lightgbm
[ 98%] Built target lightgbm
make: *** [Makefile:130: all] Error 2

I'm not familiar with cpp and don't know how to handle it, but it seems that the following template needs to be defined outside of the class.

template <>
ArrowSchema create_primitive_schema<float>() {

Reproducible example

docker build . -f Dockerfile

with the following Dockerfile

FROM ubuntu:20.04

ENV \
    DEBIAN_FRONTEND=noninteractive \
    LANG=C.UTF-8 \
    LC_ALL=C.UTF-8

RUN apt-get update -y && \
    apt-get install -y --no-install-recommends \
        ca-certificates \
        cmake \
        build-essential \
        gcc \
        g++ \
        git \
        libomp-dev && \
    rm -rf /var/lib/apt/lists/*

RUN git clone \
        --recursive \
        --branch stable \
        --depth 1 \
        https://github.com/Microsoft/LightGBM && \
    mkdir LightGBM/build && \
    cd LightGBM/build && \
    cmake .. -DBUILD_CPP_TEST=ON && \
    make -j12 && \
    make install && \
    cd "${HOME}" && \
    rm -rf LightGBM

ENTRYPOINT ["lightgbm"]

Environment info

LightGBM version or commit hash: 3405ee8 or later

Command(s) you used to install LightGBM

cd LightGBM/build && cmake .. -DBUILD_CPP_TEST=ON && make -j

Additional Comments

@jameslamb jameslamb added the bug label Dec 28, 2023
@jameslamb
Copy link
Collaborator

jameslamb commented Dec 28, 2023

Thanks very much for the excellent report, and sorry for the inconvenience.

This is making me realize we don't actually run the c++ tests with gcc in continuous integration. Just:

clang-17 on Ubuntu 22.04:

LightGBM/.vsts-ci.yml

Lines 126 to 128 in 4588d64

cpp_tests:
TASK: cpp-tests
METHOD: with-sanitizers

AppleClang (unsure of the version) on macOS 11

LightGBM/.vsts-ci.yml

Lines 235 to 238 in 4588d64

cpp_tests:
TASK: cpp-tests
METHOD: with-sanitizers
SANITIZERS: "address;undefined"

MSVC 2019 on Windows

LightGBM/.vsts-ci.yml

Lines 275 to 276 in 4588d64

cpp_tests:
TASK: cpp-tests

@jameslamb
Copy link
Collaborator

@borchero is this something you could help with? I think the fix would be:

  1. add a cpp_test CI job to the Linux matrix at
    - job: Linux
  2. confirm that it reproduces the error reported here
  3. push additional commits fixing the issue

At the current low level of activity in this project, I think we can afford one more cpp_test CI job on Azure DevOps, in exchange for being able to catch these issues in the future.

@borchero
Copy link
Collaborator

I'll have a look at this!

@borchero
Copy link
Collaborator

I can't replicate the issue locally, unfortunately, let's see if #6315 helps.

@jameslamb
Copy link
Collaborator

I ran into this today investigating #4331.

On my M2 mac, latest master here.

brew install gcc-13

cmake \
    -B build \
    -S . \
    -DBUILD_CPP_TEST=ON \
    -DCMAKE_C_COMPILER='gcc-13' \
    -DCMAKE_CXX_COMPILER='g++-13'

cmake \
    --build build \
    --target testlightgbm \
    -j4

Configure logs.

-- The C compiler identification is GNU 13.2.0
-- The CXX compiler identification is GNU 13.2.0
-- Checking whether C compiler has -isysroot
-- Checking whether C compiler has -isysroot - yes
-- Checking whether C compiler supports OSX deployment target flag
-- Checking whether C compiler supports OSX deployment target flag - yes
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/homebrew/bin/gcc-13 - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Checking whether CXX compiler has -isysroot
-- Checking whether CXX compiler has -isysroot - yes
-- Checking whether CXX compiler supports OSX deployment target flag
-- Checking whether CXX compiler supports OSX deployment target flag - yes
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/homebrew/bin/g++-13 - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- Performing Test MM_PREFETCH
-- Performing Test MM_PREFETCH - Failed
-- Performing Test MM_MALLOC
-- Performing Test MM_MALLOC - Failed
-- Could NOT find GTest (missing: GTest_DIR)
-- Did not find Google Test in the system root. Fetching Google Test now...
-- Found Python3: /Users/jlamb/miniforge3/envs/pydistcheck-dev/bin/python3.12 (found version "3.12.3") found components: Interpreter 
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Configuring done (3.8s)
-- Generating done (0.0s)
-- Build files have been written to: /Users/jlamb/repos/LightGBM/build

End of the build logs.

[ 85%] Building CXX object CMakeFiles/testlightgbm.dir/tests/cpp_tests/test_chunked_array.cpp.o
/Users/jlamb/repos/LightGBM/tests/cpp_tests/test_arrow.cpp:159:13: error: explicit specialization in non-namespace scope 'class ArrowChunkedArrayTest'
  159 |   template <>
      |             ^
/Users/jlamb/repos/LightGBM/tests/cpp_tests/test_arrow.cpp:160:15: error: template-id 'create_primitive_schema<float>' in declaration of primary template
  160 |   ArrowSchema create_primitive_schema<float>() {
      |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/jlamb/repos/LightGBM/tests/cpp_tests/test_arrow.cpp:174:13: error: explicit specialization in non-namespace scope 'class ArrowChunkedArrayTest'
  174 |   template <>
      |             ^
/Users/jlamb/repos/LightGBM/tests/cpp_tests/test_arrow.cpp:175:15: error: template-id 'create_primitive_schema<bool>' in declaration of primary template
  175 |   ArrowSchema create_primitive_schema<bool>() {
...
/Users/jlamb/repos/LightGBM/tests/cpp_tests/test_arrow.cpp:175:15: error: 'ArrowSchema ArrowChunkedArrayTest::create_primitive_schema()' cannot be overloaded with 'ArrowSchema ArrowChunkedArrayTest::create_primitive_schema()'
/Users/jlamb/repos/LightGBM/tests/cpp_tests/test_arrow.cpp:160:15: note: previous declaration 'ArrowSchema ArrowChunkedArrayTest::create_primitive_schema()'
  160 |   ArrowSchema create_primitive_schema<float>() {
      |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1plus: note: unrecognized command-line option '-Wno-unused-private-field' may have been intended to silence earlier diagnostics
make[3]: *** [CMakeFiles/testlightgbm.dir/tests/cpp_tests/test_arrow.cpp.o] Error 1
make[3]: *** Waiting for unfinished jobs....
make[2]: *** [CMakeFiles/testlightgbm.dir/all] Error 2
make[1]: *** [CMakeFiles/testlightgbm.dir/rule] Error 2
make: *** [testlightgbm] Error 

A patch like this fixed it for me:

diff --git a/tests/cpp_tests/test_arrow.cpp b/tests/cpp_tests/test_arrow.cpp
index c2dbd6ce..29bff7dc 100644
--- a/tests/cpp_tests/test_arrow.cpp
+++ b/tests/cpp_tests/test_arrow.cpp
@@ -151,13 +151,7 @@ class ArrowChunkedArrayTest : public testing::Test {
 
   /* ------------------------------------- SCHEMA CREATION ------------------------------------- */
 
-  template <typename T>
-  ArrowSchema create_primitive_schema() {
-    std::logic_error("not implemented");
-  }
-
-  template <>
-  ArrowSchema create_primitive_schema<float>() {
+  ArrowSchema create_primitive_schema_float() {
     ArrowSchema schema;
     schema.format = "f";
     schema.name = nullptr;
@@ -171,8 +165,7 @@ class ArrowChunkedArrayTest : public testing::Test {
     return schema;
   }
 
-  template <>
-  ArrowSchema create_primitive_schema<bool>() {
+  ArrowSchema create_primitive_schema_bool() {
     ArrowSchema schema;
     schema.format = "b";
     schema.name = nullptr;
@@ -213,7 +206,7 @@ class ArrowChunkedArrayTest : public testing::Test {
 /* --------------------------------------------------------------------------------------------- */
 
 TEST_F(ArrowChunkedArrayTest, GetLength) {
-  auto schema = create_primitive_schema<float>();
+  auto schema = create_primitive_schema_float();
 
   std::vector<float> dat1 = {1, 2};
   auto arr1 = create_primitive_array(dat1);
@@ -241,8 +234,8 @@ TEST_F(ArrowChunkedArrayTest, GetColumns) {
   std::vector<ArrowArray*> arrs = {&arr1, &arr2};
   auto arr = created_nested_array(arrs);
 
-  auto schema1 = create_primitive_schema<float>();
-  auto schema2 = create_primitive_schema<float>();
+  auto schema1 = create_primitive_schema_float();
+  auto schema2 = create_primitive_schema_float();
   std::vector<ArrowSchema*> schemas = {&schema1, &schema2};
   auto schema = create_nested_schema(schemas);
 
@@ -266,7 +259,7 @@ TEST_F(ArrowChunkedArrayTest, IteratorArithmetic) {
   auto arr2 = create_primitive_array(dat2);
   std::vector<float> dat3 = {7};
   auto arr3 = create_primitive_array(dat3);
-  auto schema = create_primitive_schema<float>();
+  auto schema = create_primitive_schema_float();
 
   ArrowArray arrs[3] = {arr1, arr2, arr3};
   ArrowChunkedArray ca(3, arrs, &schema);
@@ -302,7 +295,7 @@ TEST_F(ArrowChunkedArrayTest, BooleanIterator) {
   auto arr1 = create_primitive_array(dat1, 0, {2});
   std::vector<bool> dat2 = {false, false, false, false, true, true, true, true, false, true};
   auto arr2 = create_primitive_array(dat2, 1);
-  auto schema = create_primitive_schema<bool>();
+  auto schema = create_primitive_schema_bool();
 
   ArrowArray arrs[2] = {arr1, arr2};
   ArrowChunkedArray ca(2, arrs, &schema);
@@ -328,7 +321,7 @@ TEST_F(ArrowChunkedArrayTest, BooleanIterator) {
 TEST_F(ArrowChunkedArrayTest, OffsetAndValidity) {
   std::vector<float> dat = {0, 1, 2, 3, 4, 5, 6};
   auto arr = create_primitive_array(dat, 2, {2, 3});
-  auto schema = create_primitive_schema<float>();
+  auto schema = create_primitive_schema_float();
   ArrowChunkedArray ca(1, &arr, &schema);
 
   auto it = ca.begin<double>();

@borchero could you try that on #6315?

@StrikerRUS
Copy link
Collaborator

StrikerRUS commented Sep 9, 2024

Just faced this issue with MinGW (gcc-14) on Windows.

Upstream bug in gcc: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=85282

Another possible workaround: https://cplusplus.com/forum/beginner/188847/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants