Skip to content

Conversation

@h-vetinari
Copy link
Member

Hopefully helps with the sentencepiece situation...

@conda-forge-linter
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@conda-forge-linter
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipe) and found some lint.

Here's what I've got...

For recipe:

  • Failed to even lint the recipe, probably because of a conda-smithy bug 😢. This likely indicates a problem in your meta.yaml, though. To get a traceback to help figure out what's going on, install conda-smithy and run conda smithy recipe-lint . from the recipe directory.

@conda-forge-linter
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@h-vetinari
Copy link
Member Author

h-vetinari commented Feb 21, 2022

@conda-forge/abseil-cpp
PTAL. The list of libs is maybe a bit overkill, but I did it as an exercise in completeness. It's based off of the logs, not upstream CMakeLists.txt, and so also contain _internal.so's. EDIT: on second thought, I removed the internal libs.

The windows packaging in particular seems to be slicing and dicing things differently, and most importantly is still also building static libs despite the BUILD_SHARED_LIBS. Not sure if we want to get rid of the static libs?

- cmake
- ninja

{% set absl_libs = [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bot will probably choke on this, I've seen several feedstock stop autoupdate because they had such a pattern. Could you put this into a test script?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I'd almost be tempted to rather fix the bot. I like this idiom for testing, keeps things nicely visible in meta.yaml.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bot will probably choke on this, [...]

So it's mostly about the version updates? I've looked here (and other places in that repo), and it sounds like it'd just need some extra handling for {% ... %} cases.

# absence of static libs
- test ! -f $PREFIX/lib/libabsl_{{ each_lib }}.a # [unix]
# TODO: do we want to get rid of the static windows libs?
# - if exist %LIBRARY_LIB%\\absl_{{ each_lib }}.lib exit 1 # [win]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Windows, you will always have .lib files. In the case of shared DLLs, these are small import libraries needed that only contain "glue" to lazy load the function definitions from the DLLs at runtime.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to learn already that .lib is both for static libs as well as the "import libraries" that wrap DLLs, but the logs here look pretty unambiguous to me:

[142/154] Linking CXX static library absl\flags\absl_flags_parse.lib

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems there's only one DLL (plus its import lib) being created on windows with BUILD_SHARED_LIBS

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's been some time since I last looked at the abseil build system, but my recollection was that on Windows only static builds were supported. Has this changed in the last year or so?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems so. abseil_dll.dll is pretty unambiguous as well IMO. ;-)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am probably being paranoid, and I understand that this would be additional work for you, but would it be possible to have a small tests that checks (on Windows) that the DLL actually works (i.e., compile and run a small executable which calls functions that have been compiled in the DLL)?

I am asking because I am a bit scared by some of the statements in the abseil docs regarding dynamics libraries:

https://abseil.io/about/compatibility

But in any case, thanks a lot for working on this! Dynamic libraries are definitely the way to go for a distribution like conda, it would be great to have this PR merged.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, AFAICT those warnings are for systems that cannot track ABI as comprehensively as conda(-forge) can.

I have an open PR that directly inspired this need. Not sure if a 5-line #include "absl/xxx" / int main() {} would help much to determine the proper operations, to be honest. What I could think of is publishing this PR to a separate channel (say absl_dev) first, and then I can test it in the sentencepiece PR (though, admittedly, a negative outcome can also just mean that the sentencepiece side isn't working yet).

How does that sound?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would be personally more in favour of pulling the trigger and merge this PR directly, and perhaps roll it back later in case it turns out it breaks existing packages.

But certainly @xhochy has a better feeling than I do about whether this course of action is appropriate for conda-forge as a whole, so I'll leave the decision to him.

@h-vetinari
Copy link
Member Author

Ping @xhochy, would appreciate your thoughts on this (e.g. if you'd agree with the libs as jinja variables if the bot were fixed (resp. which migrators are of concern...); an whether we should be including/excluding the windows static libs despite switching to shared builds)

@h-vetinari
Copy link
Member Author

Gentle ping @xhochy :)

@xhochy xhochy merged commit c42bb71 into conda-forge:master Feb 28, 2022
@h-vetinari
Copy link
Member Author

Thank you!

BTW, I'd be happy to try fixing the bot so it can update feedstocks with such a test idiom; if you can tell me which kind of migrator I should look at primarily from your POV, it'll probably be faster (but not strictly necessary).

@h-vetinari h-vetinari deleted the win_shared branch February 28, 2022 12:59
@lidavidm
Copy link

lidavidm commented Feb 28, 2022

I'm seeing several of Arrow's AppVeyor builds failing to link, and it seems the difference is that it uses the updated version of abseil. These builds all use abseil-cpp build h0e60522_1:

[285/349] Linking CXX shared library release\arrow_flight.dll
FAILED: release/arrow_flight.dll release/arrow_flight.lib 
cmd.exe /C "cd . && C:\Miniconda37-x64\envs\arrow\Library\bin\cmake.exe -E vs_link_dll --intdir=src\arrow\flight\CMakeFiles\arrow_flight_shared.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\mt.exe --manifests  -- C:\PROGRA~2\MIB055~1\2017\COMMUN~1\VC\Tools\MSVC\1416~1.270\bin\Hostx64\x64\link.exe /nologo src\arrow\flight\CMakeFiles\arrow_flight_shared.dir\Unity\unity_1_cxx.cxx.obj src\arrow\flight\CMakeFiles\arrow_flight_shared.dir\Unity\unity_0_cxx.cxx.obj  /out:release\arrow_flight.dll /implib:release\arrow_flight.lib /pdb:release\arrow_flight.pdb /dll /version:800.0 /machine:x64  /NODEFAULTLIB:LIBCMT /INCREMENTAL:NO  release\arrow.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\grpc++.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\libprotobuf.lib  ws2_32.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\brotlienc.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\brotlidec.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\brotlicommon.lib  C:\Miniconda37-x64\envs\arrow\Library\bin\aws-cpp-sdk-config.lib  C:\Miniconda37-x64\envs\arrow\Library\bin\aws-cpp-sdk-transfer.lib  C:\Miniconda37-x64\envs\arrow\Library\bin\aws-cpp-sdk-identity-management.lib  C:\Miniconda37-x64\envs\arrow\Library\bin\aws-cpp-sdk-cognito-identity.lib  C:\Miniconda37-x64\envs\arrow\Library\bin\aws-cpp-sdk-sts.lib  C:\Miniconda37-x64\envs\arrow\Library\bin\aws-cpp-sdk-s3.lib  C:\Miniconda37-x64\envs\arrow\Library\bin\aws-cpp-sdk-core.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\aws-c-event-stream.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\aws-c-io.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\aws-c-cal.lib  NCrypt.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\aws-checksums.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\aws-c-common.lib  BCrypt.lib  Kernel32.lib  Ws2_32.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\utf8proc.lib  mimalloc_ep\src\mimalloc_ep\lib\mimalloc-1.7\mimalloc-static.lib  ws2_32.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\grpc.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\libssl.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\libcrypto.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\re2.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\z.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\cares.lib  advapi32.lib  iphlpapi.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\address_sorting.lib  ws2_32.lib  crypt32.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\absl_statusor.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\gpr.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\upb.lib  C:\Miniconda37-x64\envs\arrow\Library\lib\abseil_dll.lib  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib  && cd ."
LINK: command "C:\PROGRA~2\MIB055~1\2017\COMMUN~1\VC\Tools\MSVC\1416~1.270\bin\Hostx64\x64\link.exe /nologo src\arrow\flight\CMakeFiles\arrow_flight_shared.dir\Unity\unity_1_cxx.cxx.obj src\arrow\flight\CMakeFiles\arrow_flight_shared.dir\Unity\unity_0_cxx.cxx.obj /out:release\arrow_flight.dll /implib:release\arrow_flight.lib /pdb:release\arrow_flight.pdb /dll /version:800.0 /machine:x64 /NODEFAULTLIB:LIBCMT /INCREMENTAL:NO release\arrow.lib C:\Miniconda37-x64\envs\arrow\Library\lib\grpc++.lib C:\Miniconda37-x64\envs\arrow\Library\lib\libprotobuf.lib ws2_32.lib C:\Miniconda37-x64\envs\arrow\Library\lib\brotlienc.lib C:\Miniconda37-x64\envs\arrow\Library\lib\brotlidec.lib C:\Miniconda37-x64\envs\arrow\Library\lib\brotlicommon.lib C:\Miniconda37-x64\envs\arrow\Library\bin\aws-cpp-sdk-config.lib C:\Miniconda37-x64\envs\arrow\Library\bin\aws-cpp-sdk-transfer.lib C:\Miniconda37-x64\envs\arrow\Library\bin\aws-cpp-sdk-identity-management.lib C:\Miniconda37-x64\envs\arrow\Library\bin\aws-cpp-sdk-cognito-identity.lib C:\Miniconda37-x64\envs\arrow\Library\bin\aws-cpp-sdk-sts.lib C:\Miniconda37-x64\envs\arrow\Library\bin\aws-cpp-sdk-s3.lib C:\Miniconda37-x64\envs\arrow\Library\bin\aws-cpp-sdk-core.lib C:\Miniconda37-x64\envs\arrow\Library\lib\aws-c-event-stream.lib C:\Miniconda37-x64\envs\arrow\Library\lib\aws-c-io.lib C:\Miniconda37-x64\envs\arrow\Library\lib\aws-c-cal.lib NCrypt.lib C:\Miniconda37-x64\envs\arrow\Library\lib\aws-checksums.lib C:\Miniconda37-x64\envs\arrow\Library\lib\aws-c-common.lib BCrypt.lib Kernel32.lib Ws2_32.lib C:\Miniconda37-x64\envs\arrow\Library\lib\utf8proc.lib mimalloc_ep\src\mimalloc_ep\lib\mimalloc-1.7\mimalloc-static.lib ws2_32.lib C:\Miniconda37-x64\envs\arrow\Library\lib\grpc.lib C:\Miniconda37-x64\envs\arrow\Library\lib\libssl.lib C:\Miniconda37-x64\envs\arrow\Library\lib\libcrypto.lib C:\Miniconda37-x64\envs\arrow\Library\lib\re2.lib C:\Miniconda37-x64\envs\arrow\Library\lib\z.lib C:\Miniconda37-x64\envs\arrow\Library\lib\cares.lib advapi32.lib iphlpapi.lib C:\Miniconda37-x64\envs\arrow\Library\lib\address_sorting.lib ws2_32.lib crypt32.lib C:\Miniconda37-x64\envs\arrow\Library\lib\absl_statusor.lib C:\Miniconda37-x64\envs\arrow\Library\lib\gpr.lib C:\Miniconda37-x64\envs\arrow\Library\lib\upb.lib C:\Miniconda37-x64\envs\arrow\Library\lib\abseil_dll.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST /MANIFESTFILE:release\arrow_flight.dll.manifest" failed (exit code 1120) with the following output:
   Creating library release\arrow_flight.lib and object release\arrow_flight.exp
grpc.lib(external_account_credentials.cc.obj) : error LNK2019: unresolved external symbol "char const * const absl::lts_20211102::RFC3339_full" (?RFC3339_full@lts_20211102@absl@@3QBDB) referenced in function "private: void __cdecl grpc_core::ExternalAccountCredentials::OnImpersenateServiceAccountInternal(struct grpc_error *)" (?OnImpersenateServiceAccountInternal@ExternalAccountCredentials@grpc_core@@AEAAXPEAUgrpc_error@@@Z)
grpc.lib(compression_internal.cc.obj) : error LNK2019: unresolved external symbol "unsigned char const * const absl::lts_20211102::ascii_internal::kPropertyBits" (?kPropertyBits@ascii_internal@lts_20211102@absl@@3QBEB) referenced in function "public: static class grpc_core::CompressionAlgorithmSet __cdecl grpc_core::CompressionAlgorithmSet::FromString(class absl::lts_20211102::string_view)" (?FromString@CompressionAlgorithmSet@grpc_core@@SA?AV12@Vstring_view@lts_20211102@absl@@@Z)
grpc.lib(uri_parser.cc.obj) : error LNK2001: unresolved external symbol "unsigned char const * const absl::lts_20211102::ascii_internal::kPropertyBits" (?kPropertyBits@ascii_internal@lts_20211102@absl@@3QBEB)
grpc.lib(rls.cc.obj) : error LNK2019: unresolved external symbol "private: static void const * const absl::lts_20211102::hash_internal::MixingHashState::kSeed" (?kSeed@MixingHashState@hash_internal@lts_20211102@absl@@0QEBXEB) referenced in function "public: static unsigned __int64 __cdecl absl::lts_20211102::hash_internal::MixingHashState::hash<struct grpc_core::`anonymous namespace'::RlsLb::RequestKey,0>(struct grpc_core::`anonymous namespace'::RlsLb::RequestKey const &)" (??$hash@URequestKey@RlsLb@?A0xd54f831c@grpc_core@@$0A@@MixingHashState@hash_internal@lts_20211102@absl@@SA_KAEBURequestKey@RlsLb@?A0xd54f831c@grpc_core@@@Z)
release\arrow_flight.dll : fatal error LNK1120: 3 unresolved externals
[286/349] Building CXX object src\arrow\python\CMakeFiles\arrow_python_flight_shared.dir\Unity\unity_0_cxx.cxx.obj
[287/349] Building CXX object src\arrow\flight\CMakeFiles\arrow_flight_testing_shared.dir\Unity\unity_0_cxx.cxx.obj
ninja: build stopped: subcommand failed.

Though, maybe we just need to get gRPC rebuilt against this abseil or something?

Example passing: https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/42731674

Example failing (unrelated PRs):

@h-vetinari
Copy link
Member Author

Apologies for the breakage @lidavidm @kszucs!

I'm going to mark the windows builds as broken, which should unblock you hopefully. I think we could test binaries of an eventual re-attempt by publishing them to an abseil_dev channel, which we could enable in a PR on apache/arrow and see what happens.

Though, maybe we just need to get gRPC rebuilt against this abseil or something?

I'm surprised that switching to shared libs causes an issue, because if anything, with only static libs before, those symbols should have been embedded in the consuming binaries. We can try rebuilding grpc too of course.

@lidavidm
Copy link

lidavidm commented Mar 1, 2022

No worries! Yeah, we can try to work things out on a separate channel. I haven't seriously dug into this yet, either.

@lidavidm
Copy link

lidavidm commented Mar 1, 2022

Just to confirm, the latest build indeed gets past the C++ build step now that it's back on the _0 build of the package: https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/42746214/job/8rk76j9aoluhgcwr?fullLog=true

@h-vetinari
Copy link
Member Author

Happy to hear it @lidavidm!

I opened an issue upstream and tried a first attempt at a fix in #27 which is pushing into the abseil_dev channel.

Would you be willing to open a PR to arrow that tries this out? You'd need to add (the equivalent of) -c conda-forge/label/abseil_dev to however you're calling conda to install the packges (i.e. add another channel, with higher priority than conda-forge)

@lidavidm
Copy link

lidavidm commented Mar 2, 2022

Thanks for putting this up! I kicked off a build here: (EDIT: it flaked, new build: https://ci.appveyor.com/project/lidavidm/arrow/builds/42761375)

@lidavidm
Copy link

lidavidm commented Mar 2, 2022

@h-vetinari
Copy link
Member Author

Thanks for testing! I saw that there are now many more missing symbols in your build. Could you confirm that you're compiling with C++17? Abseil's API/ABI depends on the standard that it's been compiled with, and we're at C++17 here (cf. #29).

A lot of the symbols on the sentencepiece side resolved themselves once I enforced the C++ standard, but some missing symbols remain - I'm still struggling to understand where these symbols go missing in the upstream CMake infra, though the patterns seems to be that they are all constants; in the case I'm hitting for sentencepiece, e.g. container_internal::kEmptyGroup, and same for hash_internal::MixingHashState::kSeed.

@lidavidm
Copy link

lidavidm commented Mar 4, 2022

Ah…no, Arrow is compiling targeting C++11 still: https://github.com/apache/arrow/blob/85f6f7c8fce2746cfde9be7f4dc870e2c48cff2a/cpp/cmake_modules/SetupCxxFlags.cmake#L122-L124

Hmm, this may be a problem then.

@h-vetinari
Copy link
Member Author

PS. How did you trigger the build? I don't see a corresponding PR in the arrow repo...

@lidavidm
Copy link

lidavidm commented Mar 4, 2022

That build is just because I have AppVeyor set up against my personal fork, but you can trigger a build by opening a PR as well, yes.

@h-vetinari
Copy link
Member Author

Could you try and see what happens if you set it to C++17?

@h-vetinari
Copy link
Member Author

Not saying that that's the solution, but just for testing.

Still, how does this work for other platforms? Because the conda-forge abseil builds already (implicitly) require C++17 since fef86bf. Feel free to chime in in #29.

@lidavidm
Copy link

lidavidm commented Mar 4, 2022

Uh, hmm. I'll have to check in the morning but I think we just didn't notice the bump, then. We do still want to target C++11 because there are supported platforms still on old compilers (I believe down to GCC 4.8 or 4.9), and we want to move to C++14 in the medium term…but jumping wholesale to C++17 seems difficult. Though, for conda, that may be a different question.

@h-vetinari
Copy link
Member Author

Well, as long as you're not getting compile time errors when using C++11 against abseil compiled with C++17, you should be fine 😅

But strictly speaking, I don't think it'll work necessarily (and on windows it certainly doesn't).

@lidavidm
Copy link

lidavidm commented Mar 4, 2022

Tried kicking off a new build: https://ci.appveyor.com/project/lidavidm/arrow/build/job/8sbrahsh07hx2wet

Yeah, it is curious that it works on the other platforms.

@lidavidm
Copy link

lidavidm commented Mar 4, 2022

Still no dice: https://ci.appveyor.com/project/lidavidm/arrow/builds/42790698/job/fwn13p399k6jmjsj#L1455

The missing symbols are *requested by gRPC. Maybe we do need to rebuild gRPC?

@h-vetinari
Copy link
Member Author

The missing symbols are *requested by gRPC. Maybe we do need to rebuild gRPC?

Ah yes, this certainly looks like gRPC hasn't been compiled with C++17 (which is where string_view changed from an internal implementation to the std-one).

@h-vetinari
Copy link
Member Author

A grpc compiled with C++17 against the shared builds here still runs into some missing symbols, but many less than in the arrow CI:

grpc.lib(compression_internal.cc.obj) : error LNK2019: unresolved external symbol "unsigned char const * const absl::lts_20211102::ascii_internal::kPropertyBits" (?kPropertyBits@ascii_internal@lts_20211102@absl@@3QBEB) referenced in function "bool __cdecl absl::lts_20211102::ascii_isspace(unsigned char)" (?ascii_isspace@lts_20211102@absl@@YA_NE@Z)
grpc.lib(uri_parser.cc.obj) : error LNK2001: unresolved external symbol "unsigned char const * const absl::lts_20211102::ascii_internal::kPropertyBits" (?kPropertyBits@ascii_internal@lts_20211102@absl@@3QBEB)
grpc.lib(rls.cc.obj) : error LNK2019: unresolved external symbol "private: static void const * const absl::lts_20211102::hash_internal::MixingHashState::kSeed" (?kSeed@MixingHashState@hash_internal@lts_20211102@absl@@0QEBXEB) referenced in function "public: static unsigned __int64 __cdecl absl::lts_20211102::hash_internal::MixingHashState::hash<struct grpc_core::`anonymous namespace'::RlsLb::RequestKey,0>(struct grpc_core::`anonymous namespace'::RlsLb::RequestKey const &)" (??$hash@URequestKey@RlsLb@?A0x0f9ab004@grpc_core@@$0A@@MixingHashState@hash_internal@lts_20211102@absl@@SA_KAEBURequestKey@RlsLb@?A0x0f9ab004@grpc_core@@@Z)
grpc.lib(external_account_credentials.cc.obj) : error LNK2019: unresolved external symbol "char const * const absl::lts_20211102::RFC3339_full" (?RFC3339_full@lts_20211102@absl@@3QBDB) referenced in function "private: void __cdecl grpc_core::ExternalAccountCredentials::OnImpersenateServiceAccountInternal(struct grpc_error *)" (?OnImpersenateServiceAccountInternal@ExternalAccountCredentials@grpc_core@@AEAAXPEAUgrpc_error@@@Z)

The MixingHashState::kSeed is one that also appears for sentencepiece, cf. abseil/abseil-cpp#1118. I guess the kPropertyBits could be fixed similarly to abseil/abseil-cpp#1119.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants