[Libwebsockets] Openssl is too old to support lws_tls_vhost_cert_info

Kyle Dias kyledias at vurbox.com
Wed Mar 2 05:36:34 CET 2022


Hi Andy,

After getting it to work for aarch64 I also needed to build a 32 bit arm
version. However this one when building no matter what I have tried, it
cannot find the openssl includes and cmake shows it is searching the
correct directories. As you suggested I did the proper "hygiene" although
even with specifying the include directories it cannot find it.

My cmake command is:
cmake .. -DLWS_WITHOUT_EXTENSIONS=1 -DLWS_WITH_SHARED=OFF
-DLWS_WITH_ZIP_FOPS=OFF -DLWS_WITH_ZLIB=OFF -DLWS_WITH_SSL=ON
-DLWS_OPENSSL_LIBRARIES="/media/diask5/NVMe/Cross_Builds/cross-root-generic32/usr/local/lib/libssl.a;/media/diask5/NVMe/Cross_Builds/cross-root-generic32/usr/local/lib/libcrypto.a"
\
-DLWS_OPENSSL_INCLUDE_DIRS='/media/diask5/NVMe/Cross_Builds/cross-root-generic32/usr/local/include"
 \
-DCMAKE_TOOLCHAIN_FILE=
'/media/diask5/NVMe/Test_Builds/Test_Scripts/WebSockets_Test_Scripts/libwebsockets/contrib/cross-arm-linux-gnueabihf.cmake'

Although it shows it is searching correctly in cmake output:
-- Found OpenSSL:
/media/diask5/NVMe/Cross_Builds/cross-root-generic32/usr/local/lib/libcrypto.a
(found version "1.1.1m")
OpenSSL include dir:
/media/diask5/NVMe/Cross_Builds/cross-root-generic32/usr/local/include
OpenSSL libraries:
/media/diask5/NVMe/Cross_Builds/cross-root-generic32/usr/local/lib/libssl.a;/media/diask5/NVMe/Cross_Builds/cross-root-generic32/usr/local/lib/libcrypto.a;dl;ssl;crypto

Then all openssl files show not found
-- Looking for openssl/ecdh.h
-- Looking for openssl/ecdh.h - not found
etc ...

Is there a different process or do I have to do something different for 32
bit arm? I assume I shouldn't but I could be wrong? I am using this
toolchain file:
https://libwebsockets.org/git/libwebsockets/tree/contrib/cross-arm-linux-gnueabihf.cmake

I have tried the debugging techniques you have kindly stated:

  - nuke any ./build dir removing the cmake cache and rebuild


This fixed issues when building for aarch64 but not fixed now for 32 bit
build dir. I created a new build directory from scratch.


>
>
  - study the first dozen lines of the cmake output, it should tell you
> how it feels about, eg, what compiler it is going to use.
>

 -- Check for working C compiler: /usr/bin/arm-linux-gnueabihf-gcc
-- Check for working C compiler: /usr/bin/arm-linux-gnueabihf-gcc -- works
-- Check for working CXX compiler: /usr/bin/arm-linux-gnueabihf-g++
-- Check for working CXX compiler: /usr/bin/arm-linux-gnueabihf-g++ -- works


>   - introduce a syntax error in the cross toolchain file temporarily and
> make sure that is actually getting seen


The error is caught.


>
>
  - sudo mv your cross-compiler out of the way temporarily and make sure
> that causes cross-builds to fail
>

Get cmake error saying it cannot build a test program.


>
>   - sudo vim a syntax error into the build machine's
> /usr/include/openssl/ssl.h temporarily and confirm if that is really
> being picked up
>

Get error: field ‘ctx’ has incomplete type
   85 |         HMAC_CTX ctx;
But this looks like it is just because it cannot find any openssl files.


>
>   - check with file that things you think you cross-built are the right
> arch
>
file on openssl:
/media/diask5/NVMe/Cross_Builds/cross-root-generic32/usr/local/bin/openssl:
ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked,
for GNU/Linux 3.2.0, with debug_info, not stripped


>   - build with make VERBOSE=1 and look at what the compiler commandline
> actually says compared to what you assume it says... if it just says
> "gcc", you're not cross compiling.
>

/usr/bin/arm-linux-gnueabihf-gcc being used.

Do you know of any other way I can try and debug why it is not building?

Thank you,
Kyle

On Mon, Feb 28, 2022 at 9:14 PM Andy Green <andy at warmcat.com> wrote:

>
>
> On March 1, 2022 12:06:41 AM GMT, Kyle Dias <kyledias at vurbox.com> wrote:
> >Hi Andy,
> >
> >After a lot of trial and error I got it to work :). But there are a few
> >things I'm confused about. I was hoping you might be able to clarify so I
> >can better understand for the future.
>
> We seem to have missed some episodes where you 'clarify' confusion from
> your last email, like the claim it's still looking in build machine
> includes.
>
> >In order for me to get it to work I had to make these changes
> >
> >   - Remove from CMAKE_C_FLAGS
> >
>  "-DOPTEE_DEV_KIT=../../../../out/arm-plat-hikey/export-ta_arm64/include"
>
> So what happened if you leave that in?  It looks like nothing should
> object if nothing looking for it specifically.
>
> >   - Had to directly specify directories as root path was not sufficient.
> >
>  -DLWS_OPENSSL_INCLUDE_DIRS=/media/diask5/NVMe/Cross_Builds/cross-root-aarch64/usr/local/include
> >
>  -DLWS_OPENSSL_LIBRARIES="/media/diask5/NVMe/Cross_Builds/cross-root-aarch64/usr/local/lib/libssl.a;/media/diask5/NVMe/Cross_Builds/cross-root-aarch64/usr/local/lib/libcrypto.a"
>
> CMAKE_FIND_ROOT_PATH and friends should be enough afaik, since cmake can
> find openssl normally.
>
> >   - Then in CMAKELISTS.txt I had to add
> >   - if (UNIX)
> >   list(APPEND LIB_LIST m)
> >   endif()
> >   if (UNIX)
> >   list(APPEND LIB_LIST dl)
> >   endif()
>
> This dependency is usually introduced by something else lws links to on
> your toolchain or platform.  Nothing in lws requires libm.  As such you
> should also be able to defer linking against that until you link your
> application, if you disable lws test app and example builds.
>
> >   - I saw there are change lists for this although will this be added to
> >   master?
> >
> https://libwebsockets.org/git/libwebsockets/commit/READMEs/README.lws_retry.md?id=2cb705f71fe176ac2746d15d7dfda21a0eaaa830
>
> That should be on main already.  But it doesn't do the above, it only
> changes libz linking order if you are using that.
>
> >   - My cmake command
> >   - cmake ..
> >
>  -DCMAKE_INSTALL_PREFIX:PATH=/media/diask5/NVMe/Cross_Builds/cross-root-aarch64/usr
> >
>  -DCMAKE_TOOLCHAIN_FILE=/media/diask5/NVMe/Test_Builds/Test_Scripts/WebSockets_Test_Scripts/libwebsockets/contrib/cross-aarch64.cmake
> >   -DLWS_WITHOUT_EXTENSIONS=1
> >
>  -DLWS_OPENSSL_INCLUDE_DIRS=/media/diask5/NVMe/Cross_Builds/cross-root-aarch64/usr/local/include
> >
>  -DLWS_OPENSSL_LIBRARIES="/media/diask5/NVMe/Cross_Builds/cross-root-aarch64/usr/local/lib/libssl.a;/media/diask5/NVMe/Cross_Builds/cross-root-aarch64/usr/local/lib/libcrypto.a"
> >   -DLWS_WITH_SHARED=OFF -DLWS_WITH_ZIP_FOPS=OFF -DLWS_WITH_ZLIB=OFF
> >
> >If I try to build with LWS_WITH_SHARED=ON my build fails at 27% with (I
> >have -fPIC set as a CMAKE_C_FLAG):
> >
>
> >/usr/lib/gcc-cross/aarch64-linux-gnu/9/../../../../aarch64-linux-gnu/bin/ld:
> >>
> /media/diask5/NVMe/Cross_Builds/cross-root-aarch64/usr/local/lib/libcrypto.a(sha1-armv8.o):
> >> relocation R_AARCH64_PREL64 against symbol `OPENSSL_armcap_P' which may
> >> bind externally can not be used when making a shared object; recompile
> with
> >> -fPIC
> >>
> /media/diask5/NVMe/Cross_Builds/cross-root-aarch64/usr/local/lib/libcrypto.a(sha1-armv8.o):
> >> in function `sha1_block_armv8':
> >> (.text+0x1240): dangerous relocation: unsupported relocation
> >>
> /usr/lib/gcc-cross/aarch64-linux-gnu/9/../../../../aarch64-linux-gnu/bin/ld:
> >>
> /media/diask5/NVMe/Cross_Builds/cross-root-aarch64/usr/local/lib/libcrypto.a(chacha-armv8.o):
> >> relocation R_AARCH64_PREL64 against symbol `OPENSSL_armcap_P' which may
> >> bind externally can not be used when making a shared object; recompile
> with
> >> -fPIC
> >
> >(.text+0x48): dangerous relocation: unsupported relocation
> >
> >relocation R_AARCH64_PREL64 against symbol `OPENSSL_armcap_P' which may
> >> bind externally can not be used when making a shared object; recompile
> with
> >> -fPIC
>
> These are nothing to do with lws, if you want shared libs, the loader
> relocates them at load-time and all the pieces must be built in a way that
> will allow that.
>
> -Andy
>
> >
> >
> >- Kyle
> >
> >On Mon, Feb 28, 2022 at 12:59 AM Andy Green <andy at warmcat.com> wrote:
> >
> >>
> >>
> >> On 2/28/22 00:05, Kyle Dias wrote:
> >> > Hi Andy,
> >> >
> >> > I have been trying that. I have a cross-root although it continues to
> >> > look in /usr/include/openssl/.
> >>
> >> Hm...
> >>
> >> > I have set my root path in the cmake toolchain file to:
> >> >
> >> >     # Where to look for the target environment. (More paths can be
> added
> >> >     here)
> >> >     set(CMAKE_FIND_ROOT_PATH
> >> >     "/media/diask5/NVMe/Cross_Builds/cross-root-aarch64")
> >> >
> >> >     # Adjust the default behavior of the FIND_XXX() commands:
> >> >     # search programs in the host environment only.
> >> >     set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM ONLY)
> >> >
> >> >     # Search headers and libraries in the target environment only.
> >> >     set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
> >> >     set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
> >>
> >> It looks reasonable, but it is not compatible with what you say just
> below.
> >>
> >> >     diask5 at diask5-X570-AORUS-ELITE-WIFI
> >> :/media/diask5/NVMe/Cross_Builds/cross-root-aarch64$
> >> >     ls
> >> >     usr
> >> >
> >> >
> >> > Although the main issue is it continues to look
> >> > in /usr/include/openssl/. Inside my cross-root openssl is installed in
> >>
> >> How do you know that is the case?  Usually that is something that's
> >> tricky to determine.
> >>
> >> > /usr/local/ I run this command for cmake:
> >>
> >> Inside, eg, /usr/local/lib/libssl.so or whatever you have in there, run
> >> file on it.
> >>
> >> >     cmake ..
> >> >
> >>
> -DCMAKE_INSTALL_PREFIX:PATH=/media/diask5/NVMe/Cross_Builds/cross-root-aarch64/usr
> >> >
> >>
> -DCMAKE_TOOLCHAIN_FILE=/media/diask5/NVMe/Test_Builds/Test_Scripts/WebSockets_Test_Scripts/libwebsockets/contrib/cross-aarch64.cmake
> >> >     -DLWS_WITHOUT_EXTENSIONS=1
> >> >
> >> >
> >> > I think I may also be generating the cross-root incorrectly. Is there
> >>
> >> No you don't have to do anything with it to "generate" it, except
> >> install cross-built things in there using DESTDIR or as you did point
> >> cmake to the right place as described before.
> >>
> >> It's just a directory on your build machine where you decide to put all
> >> your cross-built pieces for a specific arch.  If it's full of
> >> .../usr/xxx and if you use file utility on those it mentions aarch64
> >> rather than x86_64 or amd64 or whatever, you're doing well there.
> >>
> >> You just have to force other cross-built pieces to understand that is
> >> the only place they can look for dependencies, not build machine
> /usr/...
> >>
> >> >   As when I build openssl it creates the files and everything in my
> >> > cross root but how would I specify the compiler in the toolchain file,
> >> > would I also need to install the compiler in my cross-root?
> >>
> >> No the cross-root is not like a chroot.  You do not need to put build
> >> machine executables in there, the whole idea is that stuff that will run
> >> on your build machine in the official places in your build machine like
> >> actual /usr/...  Stuff that is *produced* cross-built on your build
> >> machine to run on a different machine is segregated into the
> >> arch-specific cross-root.  And builds or cross-builds look in one place
> >> or the other exclusively.
> >>
> >> Your cross-toolchain is an x86_64 executable, it has no place in your
> >> aarch64 cross-root.  This is why I think of this convention to separate
> >> things as "hygiene".
> >>
> >> It sounds like one of the things you think is happening, is not actually
> >> happening.  What I would do is go on a campaign of making things I think
> >> is happening, prove themselves:
> >>
> >>   - nuke any ./build dir removing the cmake cache and rebuild
> >>
> >>   - study the first dozen lines of the cmake output, it should tell you
> >> how it feels about, eg, what compiler it is going to use.
> >>
> >>   - introduce a syntax error in the cross toolchain file temporarily and
> >> make sure that is actually getting seen
> >>
> >>   - sudo mv your cross-compiler out of the way temporarily and make sure
> >> that causes cross-builds to fail
> >>
> >>   - sudo vim a syntax error into the build machine's
> >> /usr/include/openssl/ssl.h temporarily and confirm if that is really
> >> being picked up
> >>
> >>   - check with file that things you think you cross-built are the right
> >> arch
> >>
> >>   - build with make VERBOSE=1 and look at what the compiler commandline
> >> actually says compared to what you assume it says... if it just says
> >> "gcc", you're not cross compiling.
> >>
> >> -Andy
> >>
> >
> >
>


-- 
Kyle Dias
CEO & Founder
kyledias at vurbox.com



VurBox Inc.
Suite 201, 1300 Cornwall Rd. Oakville, ON | L6J 7W5
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://libwebsockets.org/pipermail/libwebsockets/attachments/20220301/d83eff0b/attachment-0001.htm>


More information about the Libwebsockets mailing list