How to avoid recursive dependencies on dynamic libraries when compiling executable files?

Assume that I ultimately want to output the string “Hello world”. In this scenario, the dynamic library libhello.so is used to output “hello”, and the dynamic library libworld.so is used to output “world”. The main function calls the interface hello() provided by libhello.so, which outputs the string “hello” and then calls the interface world() provided by libworld.so. The world() function outputs the string “world”. The dependency relationship is as follows: main —-> libhello.so —–> libworld.so, meaning that the main function does not directly depend on libworld.so.

Given that I do not want to place libworld.so in the default library search path, how can I avoid the warning message “warning: libworld.so, needed by libhello.so, not found (try using -rpath or -rpath-link)” when compiling the main function?

In other words, for a dependency relationship like main —-> libhello.so —–> libworld.so, how can I ensure that during compilation, the executable only checks for the existence of the direct dependency libhello.so and does not recursively check for the existence of the indirect dependency libworld.so?

In the CMakeLists.txt file used to compile libhello.so, the target_link_libraries command has already linked the libworld.so library. From my research online, I found that the link option –no-copy-dt-needed-entries could be used, but it hasn’t been effective.

Here is a part of CMakeLists.txt of libhello.so

cmake_minimum_required(VERSION 3.16.3)
cmake_policy(VERSION 3.16.3...3.20.3)

project(hello)

list(APPEND CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/cmake/)

find_package(World REQUIRED MODULE)

file(GLOB HELLO_HEAD ${CMAKE_CURRENT_SOURCE_DIR}/include)
file(GLOB HELLO_SRC ${CMAKE_CURRENT_SOURCE_DIR}/src/hello.cpp)

include_directories(${HELLO_HEAD} ${WORLD_INCLUDE_DIRS})
set(LIBRARY_OUTPUT_PATH ${PROJECT_SOURCE_DIR}/lib)

add_library(${PROJECT_NAME} SHARED ${HELLO_SRC} ${HELLO_HEAD}/hello.hpp)
SET(CMAKE_INSTALL_PREFIX ${PROJECT_SOURCE_DIR}/install)
link_directories(${WORLD_LIBRARY_DIRS})
target_link_libraries(${PROJECT_NAME}
    PRIVATE ${WORLD_LIBRARIES}
)
target_link_options(${PROJECT_NAME} PUBLIC "-Wl,-v,--no-copy-dt-needed-entries")
target_include_directories(${PROJECT_NAME} PUBLIC
    $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
    $<INSTALL_INTERFACE:include>)
set_target_properties(${PROJECT_NAME} PROPERTIES PUBLIC_HEADER "include/hello.hpp")

Here is apart of CMakeLists.txt of main

cmake_minimum_required(VERSION 3.10)
project(Te)

# 设置find_package的查找路径

# list(APPEND CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/../helloTest/install/lib/cmake/mylib/)
# set(CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/../helloTest/install/lib/cmake/mylib/)
set(hello_DIR ${CMAKE_CURRENT_SOURCE_DIR}/../helloTest/install/lib/cmake/mylib/)
find_package(hello REQUIRED)
if(hello_FOUND)
    message(STATUS "hello_FOUND = ${hello_FOUND}")
    message(STATUS "hello_INCLUDE_DIRS = ${hello_INCLUDE_DIRS}")
    message(STATUS "hello_LIBRARIES = ${hello_LIBRARIES}")
    message(STATUS "hello_LIBRARY_DIRS = ${hello_LIBRARY_DIRS}")
endif()


add_executable(ppp ${CMAKE_CURRENT_SOURCE_DIR}/main.cpp)

target_link_directories(ppp PUBLIC ${hello_LIBRARY_DIRS})
target_link_libraries(ppp hello)
target_include_directories(ppp PUBLIC ${hello_INCLUDE_DIRS})
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,--no-copy-dt-needed-entries")

From my research online, I found that the link option –no-copy-dt-needed-entries could be used, but it hasn’t been effective.

2

If you don’t plan to have libworld as a shared object, thus not visible, shared or usable for other applications, then you can make it a static library and link it statically into the shared object libhello.so

This behaves similar as if you would implement the function world() directly in libhello.so and don’t export its symbols.
Other applications only need to link libhello.so and don’t know about libworld.

Unfortunately I cannot help you how to do that with CMAKE.

You need to follow the advice suggested to you by the linker diagnostic:

warning: libworld.so, needed by libhello.so, not found (try using -rpath or -rpath-link)

when you build libhello.so: specifically, using -rpath to instruct the linker to inscribe
in libhello.so the runpath ( = directory) where its dynamic dependency libworld.so can be found
when libhello.so is linked with a program, such as your ppp, (or with another library).

When a library libfoo is required for the linkage of a program, whether as a direct or indirect dependency,
the linker must be told:

  • that libfoo is to be input to the linkage.
  • a search directory in which it can find (the right) libfoo, unless libfoo resides in one of the default
    search directories hard-coded into the linker.

The linker can get to know what libraries are needed and any non-default directories in which to search for them from
two sources:

  1. The commandline passed to the linker.
    Specifically, -l lib and -L dir options in that commandline. There are in turn two sub-sources from which it can get such options:

    • The programmer who explicitly writes them for the linker.
    • The language frontend (gcc, g++,gfortran…) that the programmer normally uses to delegate and simplify the invocation of
      the linker. The simplifying work of the frontend consists in augmenting the linkage options written by the programmer
      with many boilerplate options that are invariant for linkages performed via that language frontend on the host system it was built for.
      The boilerplate linkage options generated by a frontend (which are passed silently to the linker unless it is invoked in verbose mode)
      include boilerplate -l lib and -L dir options. For simplicity I will lump together all the search directories that are
      hard-coded in the linker and all the boilerplate search directories that are passed to it by a frontend and call them the default search
      directories
      for that frontend. These default search directories are always ones which the system dynamic linker
      is configured to search at runtime to load the shared libraries that a program needs.
  2. The dynamic section in any required shared library libfoo that the linker has already discovered.
    This ELF section can specify (among other things):

    • Further shared libraries that are dependencies of libfoo, by way of NEEDED libname entries.
    • Further (non-default) search directories for such dependencies, by way of RUNPATH dirname entries.
      A NEEDED libname entry is inscribed in the dynamic section of a program or shared library when it is created by the linker for each shared
      library specified by -l name in the linkage options. A RUNPATH dirname entry is inscribed for each -rpath=dirname in the linkage options.
      Any -rpath=dirname linkage option will be one that is explicitly written by the programmer. A language frontend does not need to to
      emit -rpath options for the linker because any libraries that it passes to the linker are bound to reside in default search
      directories.

The dynamic section of an ELF binary is interrogated by both the static linker the and dynamic linker to discover (among other things) dynamic dependency information:
what shared libraries are NEEDED and what RUNPATHs to search for them.

So if:

  • your program ppp depends on libhello.so and libhello.so depends in libworld.so, and
  • for some reason you don’t want libworld.so to reside in a default search directory, and
  • for some reason you also don’t want to mention the libworld.so dependency and its location in the linkage command of ppp

then your only choice is to supply this information to the linker via the dynamic section of libhello.so.

Here’s how you do that from first principles.

Source files:

$ cat main.cpp
#include <cstdlib>

extern void hello();

int main()
{
    hello();
    exit(EXIT_SUCCESS);
}

$ cat hello.cpp
#include <iostream>

extern void world();

void hello()
{
    std::cout << "Hello ";
    world();
}

$ cat world.cpp
#include <iostream>

void world()
{
    std::cout << "Worldn";
}

Compile the sources:

$ g++ -c main.cpp
$ g++ -c -fPIC hello.cpp world.cpp # Position Independent Code for shared libraries.

Build libworld.so:

$ g++ -shared -o libworld.so world.o

Build libhello.so:

$ g++ -shared -o libhello.so hello.o -L . -lworld -Wl,-rpath=$(pwd)

Here, we’ve told the linker that libhello needs libworld, to search for it in the current directory (-L .)
and that the absolute name of the current directory is to be inscribed
as a RUNPATH in the dynamic section of the output file (-Wl,-rpath=$(pwd)). In this example I’m just
using the scrap directory I’m working in as a specimen non-default directory. Let’s look at the top of that dynamic section:

$ readelf --dynamic libhello.so

Dynamic section at offset 0x2de0 contains 26 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libworld.so]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x000000000000001d (RUNPATH)            Library runpath: [/home/imk/develop/so/scrap]
 ...[cut]...
 

Note that besides libworld.so a dynamic dependency has also been inserted on libstdc++, because the linker has found
symbols that are resolved by that library, which g++ passes to the linker by default. That library will be found in a default search directory: no extra RUNPATH needed.

Now let’s move libhello.so into one of the linker’s default search directories and leave libworld.so in its non-default location.

$ sudo mv libhello.so /usr/local/lib/

Not forgetting to refresh the linker cache to catch that update:

$ sudo ldconfig

Finally, link a program prog against libhello without mentioning libworld or its location:

$ g++ -o prog main.o -lhello

Here’s the top of the dynamic section of prog:

$ readelf --dynamic prog

Dynamic section at offset 0x2db0 contains 28 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libhello.so]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
...[cut]... 

This linkage succeeds because:

  • -l hello is sufficient for the static linker to find libhello.so in the default directory
    /usr/local/lib and resolve the reference to hello therein.
  • The dynamic section of libhello.so informs the static linker that libworld.so is needed and
    can be searched for in the non-default directory /home/imk/develop/so/scrap, where indeed it is found. The
    static linker resolves the reference to world therein.

Success means that the static linker (besides succeeding with all the static symbol resolution) has determined that the dynamic linker will be able to resolve
all dynamic symbols recursively required by the program when the dynamic linker loads it: The NEEDED and RUNPATH entries
in the dynamic sections of prog and libhello.so provide the library and search directory information
that the dynamic linker will need to do that – as we can see:

$ ./prog
Hello World

But note that success does not show you an answer to this question:

for a dependency relationship like main —-> libhello.so —–> libworld.so, how can I ensure that
during compilation, the executable [you mean the linker] only checks for the existence of the direct
dependency libhello.so and does not recursively check for the existence of the indirect dependency libworld.so?

That shortcut is not taken by the linkage above and and cannot be taken in a linkage that determines the dynamic
linker will be able to resolve all symbols at runtime. To determine that, the static linker must recursively search
for and find libraries that resolve all symbol references and make sure that the dynamic linker will be
able to the same at runtime (via NEEDED and RUNPATH entries). If you want the static linker to link a program without doing that, so that you can really omit
indirect dynamic dependency information the linkage, then you must use unusual options to coerce the linker to tolerate undefined symbol references.
And unless you then manually pre-load shared libraries at runtime that resolve all the symbols the program will seg-fault.

How you do specify an -rpath with CMake.

There’s more than one way. The most obvious is to specify target_link_options for libhello.so in its
CMakeLists.txt:

...
target_link_directories(${PROJECT_NAME} PRIVATE <non-default-dir-of-libworld>
...
target_link_options(${PROJECT_NAME} PRIVATE "-Wl,-rpath=<non-default-dir-of-libworld")

A more CMake-ish way is to set the BUILD_RPATH property for the libhello target:

...
set_property(TARGET ${PROJECT_NAME} APPEND PROPERTY BUILD_RPATH <non-default-dir-of-libworld>)
...
target_link_directories(${PROJECT_NAME} PRIVATE <non-default-dir-of-libworld>)

But do you really want to do that? (Optional reading).

When you say:

In the CMakeLists.txt file used to compile libhello.so, the target_link_libraries command has
already linked the libworld.so library.

it suggests you were thinking: My libhello build already links libworld. So that
dependency info is already baked into
libhello, so why should I have to repeat it to the linker
again when I link a program against
libhello?

You’ve now learned, however, that an essential component of that dependency info – namely, the non-default search-directory
for libworld – is not baked into libhello, unless you specify it with -rpath. The part that is baked into libhello.sois the dynamic section entry NEEDED libworld.so, which is not enough for successfully linking a program against libhello, unless libworld.so
is in a default search directory. And it is baked in because you linked libhello.so against libworld.so – which you have no need to do in order to build libhello.
It is a shared library and, unlike a program, external references do not need to be resolved in the successful
linkage of a shared library. External references need only be resolved for successful linkage of a program.

When we say, e.g. that libhello.so depends on libworld.so, what we strictly mean is that linking a program against libhello imposes upon the program
an additional dependency on libworld. Not that the linkage of libhello.so requires libworld.so.

When building a shared library we do not as matter of course link it against it other shared libraries it refers to: in the absence of technical reasons for linking against those
others, we just let the undefined symbols in the library remain unresolved, as they would be in static library. Equally, when we build a program our default expectation is that we need to link it against all the shared libraries that it actually requires, not just its direct dependencies. These conventions make the -l lib options for a linkage invariant whether we choose to link static or shared versions of the libraries; they defer commitments about exactly what library binaries to link with a program and where to find them to the point where they have to be made – that is, the program linkage – and put them under the control of the person responsible for getting a program to link and run as desired in a specific environment. That’s how program builders generally prefer it. There is no virtue in just attempting to “hide” indirect dynamic dependencies in a program linkage and they can’t be hidden from anyone who knows how to do:

$ ldd prog
    linux-vdso.so.1 (0x00007ffc7e914000)
    libhello.so => /usr/local/lib/libhello.so (0x000079d4a57c8000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x000079d4a5400000)
    libworld.so => /home/imk/develop/so/scrap/libworld.so (0x000079d4a57c3000)
    libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x000079d4a5000000)
    /lib64/ld-linux-x86-64.so.2 (0x000079d4a57e7000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x000079d4a56da000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x000079d4a56ab000)

Nothing in your post suggests itself as a technical motive for linking libhello against libworld except possibly:

I do not want to place libworld.so in the default library search path.

The -rpath option answers that requirement because it enables linking shared libraries that are not in the default search directories. But you
can equally use -rpath for that purpose in the linkage of a program as well as the linkage of a shared library. Without linking libhello against
libworld, a program can load libworld.so from a non-default search directory with linkages like the following:

$ g++ -shared -o libworld.so world.o
$ g++ -shared -o libhello.so hello.o
# Then move `libhello.so` to a default search directory.
$ g++ -o prog -lhello -L . -lworld -Wl,-rpath=$(pwd)

This would be the ordinary approach. The dynamic dependency info in libhello.so is the same as in libworld.so:

$ readelf --dynamic /usr/local/lib/libhello.so 

Dynamic section at offset 0x2e00 contains 24 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 ...[cut]...
 

And in prog it looks like:

$ readelf --dynamic prog

Dynamic section at offset 0x2d90 contains 30 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libhello.so]
 0x0000000000000001 (NEEDED)             Shared library: [libworld.so]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000001d (RUNPATH)            Library runpath: [/home/imk/develop/so/scrap]
 ...[cut]...

In CMake, you would implement an -rpath for the ppp target in the same way as you’d do it for libhello.

Trang chủ Giới thiệu Sinh nhật bé trai Sinh nhật bé gái Tổ chức sự kiện Biểu diễn giải trí Dịch vụ khác Trang trí tiệc cưới Tổ chức khai trương Tư vấn dịch vụ Thư viện ảnh Tin tức - sự kiện Liên hệ Chú hề sinh nhật Trang trí YEAR END PARTY công ty Trang trí tất niên cuối năm Trang trí tất niên xu hướng mới nhất Trang trí sinh nhật bé trai Hải Đăng Trang trí sinh nhật bé Khánh Vân Trang trí sinh nhật Bích Ngân Trang trí sinh nhật bé Thanh Trang Thuê ông già Noel phát quà Biểu diễn xiếc khỉ Xiếc quay đĩa Dịch vụ tổ chức sự kiện 5 sao Thông tin về chúng tôi Dịch vụ sinh nhật bé trai Dịch vụ sinh nhật bé gái Sự kiện trọn gói Các tiết mục giải trí Dịch vụ bổ trợ Tiệc cưới sang trọng Dịch vụ khai trương Tư vấn tổ chức sự kiện Hình ảnh sự kiện Cập nhật tin tức Liên hệ ngay Thuê chú hề chuyên nghiệp Tiệc tất niên cho công ty Trang trí tiệc cuối năm Tiệc tất niên độc đáo Sinh nhật bé Hải Đăng Sinh nhật đáng yêu bé Khánh Vân Sinh nhật sang trọng Bích Ngân Tiệc sinh nhật bé Thanh Trang Dịch vụ ông già Noel Xiếc thú vui nhộn Biểu diễn xiếc quay đĩa Dịch vụ tổ chức tiệc uy tín Khám phá dịch vụ của chúng tôi Tiệc sinh nhật cho bé trai Trang trí tiệc cho bé gái Gói sự kiện chuyên nghiệp Chương trình giải trí hấp dẫn Dịch vụ hỗ trợ sự kiện Trang trí tiệc cưới đẹp Khởi đầu thành công với khai trương Chuyên gia tư vấn sự kiện Xem ảnh các sự kiện đẹp Tin mới về sự kiện Kết nối với đội ngũ chuyên gia Chú hề vui nhộn cho tiệc sinh nhật Ý tưởng tiệc cuối năm Tất niên độc đáo Trang trí tiệc hiện đại Tổ chức sinh nhật cho Hải Đăng Sinh nhật độc quyền Khánh Vân Phong cách tiệc Bích Ngân Trang trí tiệc bé Thanh Trang Thuê dịch vụ ông già Noel chuyên nghiệp Xem xiếc khỉ đặc sắc Xiếc quay đĩa thú vị
Trang chủ Giới thiệu Sinh nhật bé trai Sinh nhật bé gái Tổ chức sự kiện Biểu diễn giải trí Dịch vụ khác Trang trí tiệc cưới Tổ chức khai trương Tư vấn dịch vụ Thư viện ảnh Tin tức - sự kiện Liên hệ Chú hề sinh nhật Trang trí YEAR END PARTY công ty Trang trí tất niên cuối năm Trang trí tất niên xu hướng mới nhất Trang trí sinh nhật bé trai Hải Đăng Trang trí sinh nhật bé Khánh Vân Trang trí sinh nhật Bích Ngân Trang trí sinh nhật bé Thanh Trang Thuê ông già Noel phát quà Biểu diễn xiếc khỉ Xiếc quay đĩa
Thiết kế website Thiết kế website Thiết kế website Cách kháng tài khoản quảng cáo Mua bán Fanpage Facebook Dịch vụ SEO Tổ chức sinh nhật