Assume that I ultimately want to output the string “Hello world”. In this scenario, the dynamic library libhello.so is used to output “hello”, and the dynamic library libworld.so is used to output “world”. The main function calls the interface hello() provided by libhello.so, which outputs the string “hello” and then calls the interface world() provided by libworld.so. The world() function outputs the string “world”. The dependency relationship is as follows: main —-> libhello.so —–> libworld.so, meaning that the main function does not directly depend on libworld.so.
Given that I do not want to place libworld.so in the default library search path, how can I avoid the warning message “warning: libworld.so, needed by libhello.so, not found (try using -rpath or -rpath-link)” when compiling the main function?
In other words, for a dependency relationship like main —-> libhello.so —–> libworld.so, how can I ensure that during compilation, the executable only checks for the existence of the direct dependency libhello.so and does not recursively check for the existence of the indirect dependency libworld.so?
In the CMakeLists.txt file used to compile libhello.so, the target_link_libraries command has already linked the libworld.so library. From my research online, I found that the link option –no-copy-dt-needed-entries could be used, but it hasn’t been effective.
Here is a part of CMakeLists.txt of libhello.so
cmake_minimum_required(VERSION 3.16.3)
cmake_policy(VERSION 3.16.3...3.20.3)
project(hello)
list(APPEND CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/cmake/)
find_package(World REQUIRED MODULE)
file(GLOB HELLO_HEAD ${CMAKE_CURRENT_SOURCE_DIR}/include)
file(GLOB HELLO_SRC ${CMAKE_CURRENT_SOURCE_DIR}/src/hello.cpp)
include_directories(${HELLO_HEAD} ${WORLD_INCLUDE_DIRS})
set(LIBRARY_OUTPUT_PATH ${PROJECT_SOURCE_DIR}/lib)
add_library(${PROJECT_NAME} SHARED ${HELLO_SRC} ${HELLO_HEAD}/hello.hpp)
SET(CMAKE_INSTALL_PREFIX ${PROJECT_SOURCE_DIR}/install)
link_directories(${WORLD_LIBRARY_DIRS})
target_link_libraries(${PROJECT_NAME}
PRIVATE ${WORLD_LIBRARIES}
)
target_link_options(${PROJECT_NAME} PUBLIC "-Wl,-v,--no-copy-dt-needed-entries")
target_include_directories(${PROJECT_NAME} PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
$<INSTALL_INTERFACE:include>)
set_target_properties(${PROJECT_NAME} PROPERTIES PUBLIC_HEADER "include/hello.hpp")
Here is apart of CMakeLists.txt of main
cmake_minimum_required(VERSION 3.10)
project(Te)
# 设置find_package的查找路径
# list(APPEND CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/../helloTest/install/lib/cmake/mylib/)
# set(CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/../helloTest/install/lib/cmake/mylib/)
set(hello_DIR ${CMAKE_CURRENT_SOURCE_DIR}/../helloTest/install/lib/cmake/mylib/)
find_package(hello REQUIRED)
if(hello_FOUND)
message(STATUS "hello_FOUND = ${hello_FOUND}")
message(STATUS "hello_INCLUDE_DIRS = ${hello_INCLUDE_DIRS}")
message(STATUS "hello_LIBRARIES = ${hello_LIBRARIES}")
message(STATUS "hello_LIBRARY_DIRS = ${hello_LIBRARY_DIRS}")
endif()
add_executable(ppp ${CMAKE_CURRENT_SOURCE_DIR}/main.cpp)
target_link_directories(ppp PUBLIC ${hello_LIBRARY_DIRS})
target_link_libraries(ppp hello)
target_include_directories(ppp PUBLIC ${hello_INCLUDE_DIRS})
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,--no-copy-dt-needed-entries")
From my research online, I found that the link option –no-copy-dt-needed-entries could be used, but it hasn’t been effective.
2
If you don’t plan to have libworld as a shared object, thus not visible, shared or usable for other applications, then you can make it a static library and link it statically into the shared object libhello.so
This behaves similar as if you would implement the function world() directly in libhello.so and don’t export its symbols.
Other applications only need to link libhello.so and don’t know about libworld.
Unfortunately I cannot help you how to do that with CMAKE.
You need to follow the advice suggested to you by the linker diagnostic:
warning: libworld.so, needed by libhello.so, not found (try using -rpath or -rpath-link)
when you build libhello.so
: specifically, using -rpath
to instruct the linker to inscribe
in libhello.so
the runpath ( = directory) where its dynamic dependency libworld.so
can be found
when libhello.so
is linked with a program, such as your ppp
, (or with another library).
When a library libfoo
is required for the linkage of a program, whether as a direct or indirect dependency,
the linker must be told:
- that
libfoo
is to be input to the linkage. - a search directory in which it can find (the right)
libfoo
, unlesslibfoo
resides in one of the default
search directories hard-coded into the linker.
The linker can get to know what libraries are needed and any non-default directories in which to search for them from
two sources:
- The commandline passed to the linker.
Specifically,-l lib
and-L dir
options in that commandline. There are in turn two sub-sources from which it can get such options:- The programmer who explicitly writes them for the linker.
- The language frontend (
gcc
,g++
,gfortran
…) that the programmer normally uses to delegate and simplify the invocation of
the linker. The simplifying work of the frontend consists in augmenting the linkage options written by the programmer
with many boilerplate options that are invariant for linkages performed via that language frontend on the host system it was built for.
The boilerplate linkage options generated by a frontend (which are passed silently to the linker unless it is invoked in verbose mode)
include boilerplate-l lib
and-L dir
options. For simplicity I will lump together all the search directories that are
hard-coded in the linker and all the boilerplate search directories that are passed to it by a frontend and call them the default search
directories for that frontend. These default search directories are always ones which the system dynamic linker
is configured to search at runtime to load the shared libraries that a program needs.
- The dynamic section in any required shared library
libfoo
that the linker has already discovered.
This ELF section can specify (among other things):- Further shared libraries that are dependencies of
libfoo
, by way ofNEEDED libname
entries. - Further (non-default) search directories for such dependencies, by way of
RUNPATH dirname
entries.
ANEEDED libname
entry is inscribed in the dynamic section of a program or shared library when it is created by the linker for each shared
library specified by-l name
in the linkage options.A RUNPATH dirname
entry is inscribed for each-rpath=dirname
in the linkage options.
Any-rpath=dirname
linkage option will be one that is explicitly written by the programmer. A language frontend does not need to to
emit-rpath
options for the linker because any libraries that it passes to the linker are bound to reside in default search
directories.
- Further shared libraries that are dependencies of
The dynamic section of an ELF binary is interrogated by both the static linker the and dynamic linker to discover (among other things) dynamic dependency information:
what shared libraries are NEEDED
and what RUNPATHs
to search for them.
So if:
- your program
ppp
depends onlibhello.so
andlibhello.so
depends inlibworld.so
, and - for some reason you don’t want
libworld.so
to reside in a default search directory, and - for some reason you also don’t want to mention the
libworld.so
dependency and its location in the linkage command ofppp
then your only choice is to supply this information to the linker via the dynamic section of libhello.so
.
Here’s how you do that from first principles.
Source files:
$ cat main.cpp
#include <cstdlib>
extern void hello();
int main()
{
hello();
exit(EXIT_SUCCESS);
}
$ cat hello.cpp
#include <iostream>
extern void world();
void hello()
{
std::cout << "Hello ";
world();
}
$ cat world.cpp
#include <iostream>
void world()
{
std::cout << "Worldn";
}
Compile the sources:
$ g++ -c main.cpp
$ g++ -c -fPIC hello.cpp world.cpp # Position Independent Code for shared libraries.
Build libworld.so
:
$ g++ -shared -o libworld.so world.o
Build libhello.so
:
$ g++ -shared -o libhello.so hello.o -L . -lworld -Wl,-rpath=$(pwd)
Here, we’ve told the linker that libhello
needs libworld
, to search for it in the current directory (-L .
)
and that the absolute name of the current directory is to be inscribed
as a RUNPATH
in the dynamic section of the output file (-Wl,-rpath=$(pwd)
). In this example I’m just
using the scrap directory I’m working in as a specimen non-default directory. Let’s look at the top of that dynamic section:
$ readelf --dynamic libhello.so
Dynamic section at offset 0x2de0 contains 26 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libworld.so]
0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6]
0x000000000000001d (RUNPATH) Library runpath: [/home/imk/develop/so/scrap]
...[cut]...
Note that besides libworld.so
a dynamic dependency has also been inserted on libstdc++
, because the linker has found
symbols that are resolved by that library, which g++
passes to the linker by default. That library will be found in a default search directory: no extra RUNPATH
needed.
Now let’s move libhello.so
into one of the linker’s default search directories and leave libworld.so
in its non-default location.
$ sudo mv libhello.so /usr/local/lib/
Not forgetting to refresh the linker cache to catch that update:
$ sudo ldconfig
Finally, link a program prog
against libhello
without mentioning libworld
or its location:
$ g++ -o prog main.o -lhello
Here’s the top of the dynamic section of prog
:
$ readelf --dynamic prog
Dynamic section at offset 0x2db0 contains 28 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libhello.so]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
...[cut]...
This linkage succeeds because:
-l hello
is sufficient for the static linker to findlibhello.so
in the default directory
/usr/local/lib
and resolve the reference tohello
therein.- The dynamic section of
libhello.so
informs the static linker thatlibworld.so
is needed and
can be searched for in the non-default directory/home/imk/develop/so/scrap
, where indeed it is found. The
static linker resolves the reference toworld
therein.
Success means that the static linker (besides succeeding with all the static symbol resolution) has determined that the dynamic linker will be able to resolve
all dynamic symbols recursively required by the program when the dynamic linker loads it: The NEEDED
and RUNPATH
entries
in the dynamic sections of prog
and libhello.so
provide the library and search directory information
that the dynamic linker will need to do that – as we can see:
$ ./prog
Hello World
But note that success does not show you an answer to this question:
for a dependency relationship like main —-> libhello.so —–> libworld.so, how can I ensure that
during compilation, the executable [you mean the linker] only checks for the existence of the direct
dependency libhello.so and does not recursively check for the existence of the indirect dependency libworld.so?
That shortcut is not taken by the linkage above and and cannot be taken in a linkage that determines the dynamic
linker will be able to resolve all symbols at runtime. To determine that, the static linker must recursively search
for and find libraries that resolve all symbol references and make sure that the dynamic linker will be
able to the same at runtime (via NEEDED
and RUNPATH
entries). If you want the static linker to link a program without doing that, so that you can really omit
indirect dynamic dependency information the linkage, then you must use unusual options to coerce the linker to tolerate undefined symbol references.
And unless you then manually pre-load shared libraries at runtime that resolve all the symbols the program will seg-fault.
How you do specify an -rpath
with CMake.
There’s more than one way. The most obvious is to specify target_link_options
for libhello.so
in its
CMakeLists.txt
:
...
target_link_directories(${PROJECT_NAME} PRIVATE <non-default-dir-of-libworld>
...
target_link_options(${PROJECT_NAME} PRIVATE "-Wl,-rpath=<non-default-dir-of-libworld")
A more CMake-ish way is to set the BUILD_RPATH
property for the libhello
target:
...
set_property(TARGET ${PROJECT_NAME} APPEND PROPERTY BUILD_RPATH <non-default-dir-of-libworld>)
...
target_link_directories(${PROJECT_NAME} PRIVATE <non-default-dir-of-libworld>)
But do you really want to do that? (Optional reading).
When you say:
In the CMakeLists.txt file used to compile libhello.so, the target_link_libraries command has
already linked the libworld.so library.
it suggests you were thinking: My libhello
build already links libworld
. So that
dependency info is already baked into libhello
, so why should I have to repeat it to the linker
again when I link a program against libhello
?
You’ve now learned, however, that an essential component of that dependency info – namely, the non-default search-directory
for libworld
– is not baked into libhello
, unless you specify it with -rpath
. The part that is baked into libhello.so
is the dynamic section entry NEEDED libworld.so
, which is not enough for successfully linking a program against libhello
, unless libworld.so
is in a default search directory. And it is baked in because you linked libhello.so
against libworld.so
– which you have no need to do in order to build libhello
.
It is a shared library and, unlike a program, external references do not need to be resolved in the successful
linkage of a shared library. External references need only be resolved for successful linkage of a program.
When we say, e.g. that libhello.so
depends on libworld.so
, what we strictly mean is that linking a program against libhello
imposes upon the program
an additional dependency on libworld
. Not that the linkage of libhello.so
requires libworld.so
.
When building a shared library we do not as matter of course link it against it other shared libraries it refers to: in the absence of technical reasons for linking against those
others, we just let the undefined symbols in the library remain unresolved, as they would be in static library. Equally, when we build a program our default expectation is that we need to link it against all the shared libraries that it actually requires, not just its direct dependencies. These conventions make the -l lib
options for a linkage invariant whether we choose to link static or shared versions of the libraries; they defer commitments about exactly what library binaries to link with a program and where to find them to the point where they have to be made – that is, the program linkage – and put them under the control of the person responsible for getting a program to link and run as desired in a specific environment. That’s how program builders generally prefer it. There is no virtue in just attempting to “hide” indirect dynamic dependencies in a program linkage and they can’t be hidden from anyone who knows how to do:
$ ldd prog
linux-vdso.so.1 (0x00007ffc7e914000)
libhello.so => /usr/local/lib/libhello.so (0x000079d4a57c8000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x000079d4a5400000)
libworld.so => /home/imk/develop/so/scrap/libworld.so (0x000079d4a57c3000)
libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x000079d4a5000000)
/lib64/ld-linux-x86-64.so.2 (0x000079d4a57e7000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x000079d4a56da000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x000079d4a56ab000)
Nothing in your post suggests itself as a technical motive for linking libhello
against libworld
except possibly:
I do not want to place libworld.so in the default library search path.
The -rpath
option answers that requirement because it enables linking shared libraries that are not in the default search directories. But you
can equally use -rpath
for that purpose in the linkage of a program as well as the linkage of a shared library. Without linking libhello
against
libworld
, a program can load libworld.so
from a non-default search directory with linkages like the following:
$ g++ -shared -o libworld.so world.o
$ g++ -shared -o libhello.so hello.o
# Then move `libhello.so` to a default search directory.
$ g++ -o prog -lhello -L . -lworld -Wl,-rpath=$(pwd)
This would be the ordinary approach. The dynamic dependency info in libhello.so
is the same as in libworld.so
:
$ readelf --dynamic /usr/local/lib/libhello.so
Dynamic section at offset 0x2e00 contains 24 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6]
...[cut]...
And in prog
it looks like:
$ readelf --dynamic prog
Dynamic section at offset 0x2d90 contains 30 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libhello.so]
0x0000000000000001 (NEEDED) Shared library: [libworld.so]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000001d (RUNPATH) Library runpath: [/home/imk/develop/so/scrap]
...[cut]...
In CMake, you would implement an -rpath
for the ppp
target in the same way as you’d do it for libhello
.