Resolving the Initialization Order Problem

Part VI of Libraries, Linking, Initialization, and C++ Series
By Darryl Gove and Stephen Clamage, July 2011

Part V - Libraries in C++

Using Link Order to Resolve Library Dependencies

One of the quickest approaches for resolving the problem of the linker picking a function from the wrong library is to change the link order for the libraries on the command line. This is both the quickest approach and the least satisfactory approach. It is unsatisfactory because it does not resolve the underlying problem of multiple symbol definitions; instead, it encourages the linker to pick the appropriate symbol.

Therefore, this approach is a workaround rather than a true solution, and it is still possible for the problem to be exposed if, for example, libraries are dynamically loaded rather than explicitly linked into the application, or, as is often the case, the libraries are loaded lazily. In addition, linkers on different operating systems, or on different versions of the same OS, might resolve symbols differently.

As an example, in Oracle Solaris, we can resolve the library ordering problem by linking with the libstat.so before we link with libdata.so, as shown in Listing 1.

Listing 1: Using Link Ordering to Help the Run-Time Linker Pick the Appropriate Symbol

$ CC -o main main.cpp -L. -R'$ORIGIN' -lstat -ldata

$ ./main
data

The Oracle Solaris run-time linker resolves symbols in the order specified on the command line, so symbols will be searched for in libstat.so before they are searched for in libdata.so. Hence libstat.so ends up binding to the Cstat constructor in itself rather than the one in libdata.so.

Unfortunately, this approach can sometimes fail. If the Oracle Solaris run-time linker identifies a circular dependence in the init sections for the libraries, it calls the init sections from the last loaded library to the first loaded library, which may cause a problem to occur. Linkers on other systems might behave differently.

This means that reordering the libraries on the link line can be a quick fix, but it might not fix every possible situation, and it might not be a robust fix, since some other symbol might cause the problem to recur.

Avoiding Multiple Template Definitions Using -instlib

There are two ways the compiler generates functions in a library. The first way is by generating out-of-line (that is, non-inlined) versions of routines. The second way is by generating routines from template code. If an executable or library contains template code, then correct functionality depends on it also having definitions for any routines that the templates depend upon. The safest way to ensure that this is the case is for the module to also contain definitions of those functions. This leads to the situation where definitions for the same symbol exist in multiple load objects.

The C++ compiler flag -instlib=<library> takes the name of a library and causes the compiler not to emit duplicate definitions for any template instances found in the specified library.

The C++ option -instlib=<lib> causes the compiler to check <lib> for any template instance it would otherwise generate. You can have multiple -instlib options on one command line. Assuming all the libraries in our examples might generate template instances, we build the libraries from the bottom up, and add -instlib options for each library lower in the hierarchy. Our build process for the four libraries would then look like Listing 2.

Listing 2: Compiling a Hierarchy of Libraries

$ CC -G -o libD.so D.cpp ...
$ CC -G -o libC.so -instlib=libD.so C.cpp ...
$ CC -G -o libB.so -instlib=libC.so -instlib=libD.so B.cpp ...
$ CC -G -o libA.so -instlib=libB.so -instlib=libC.so -instlib=libD.so A.cpp ...

Any template generated in libD.so will not be generated in any of the other libraries and so on up the hierarchy. The only template instances generated in libA.so will be those not already present in libB.so, libC.so, or libD.so.

You would also use -instlib options for any of the libraries you explicitly link on the command line that build the main program. Assume, for example, that libA.so and libB.so are the only libraries the main program knows about explicitly, and that the others are, in effect, implementation details of libA.so and libB.so. The command to build the application would look like Listing 3.

Listing 3: Using -instlib to Avoid Duplicate Template Instantiations

$ CC -o myprog -instlib=libA.so -instlib=libB.so myprog.cpp ... -lA -lB

Inline functions that are generated out of line are not affected by the -instlib option. This is because the compiler cannot know whether you meant to interpose on a library function with a function that happened to be declared inline in the application, and it can't tell whether a function in a library was originally declared inline.

For example, consider a template that is declared in a header file, as shown in Listing 4. For a template to be declared in a header, all the definitions of the template member functions should be included in the header file. The compiler must have compile-time access to all the template member functions for it to be able to produce the specialized versions required when the template is applied to a particular data type.

Listing 4: Template Declared in a Header File

$ more template.h
template <class CType> class Counter
{
  CType value;
public:
  Counter(CType initial) { value = initial; }
  CType add(CType increment) { value += increment; return value; }
  CType get() {return value; }
};

Suppose the template is instantiated for the type int in two libraries. The source code for the two libraries is shown in Listing 5.

Listing 5: Two Libraries that Instantiate the Counter Template on the int Data Type

$ more lib1.cpp
#include <stdio.h>
#include "template.h"

void usetemplate1()
{
  Counter<int> object(0);
  object.add(10);
  printf("Library 1 Value = %i\n",object.get());
}

$ more lib2.cpp
#include <stdio.h>
#include "template.h"

void usetemplate2()
{
  Counter<int> object(0);
  object.add(17);
  printf("Library 2 Value = %i\n",object.get());
}

Listing 6 shows the results of compiling the two libraries and inspecting them for the definition of the symbol get().

Listing 6: Compiling Two Libraries and Examining Them for Template Code

$ CC -g -G -Kpic -o lib1.so lib1.cpp
$ CC -g -G -Kpic -o lib2.so lib2.cpp
$ nm lib1.so|grep Counter | c++filt |grep get
[60]    |      1512|      36|FUNC |GLOB |0    |8      |int Counter<int>::get()
$ nm lib2.so|grep Counter | c++filt |grep get
[60]    |      1512|      36|FUNC |GLOB |0    |8      |int Counter<int>::get()

In this example, the compiler generates out-of-line versions of the template routines when debug information is requested without optimization. Consequently, both libraries end up providing definitions for the same routines. As we discussed earlier, this can lead to run-time problems.

Using the -instlib flag, we can request that the compiler does not duplicate template definitions from existing libraries in the library code currently being compiled. This can be used to ensure that lib1.so contains the implementation of the template code and that lib2.so uses these definitions. This method is shown in Listing 7.

Listing 7: Using -instlib to Ensure that Only One Library Provides Definitions for Template Code

$ CC -g -G -Kpic -o lib1.so lib1.c
$ CC -g -G -Kpic -o lib2.so -instlib=./lib1.so lib2.c
$ nm lib1.so|grep Counter | c++filt |grep get
[60]    |      1512|      36|FUNC |GLOB |0    |8      |int Counter<int>::get()
$ nm lib2.so|grep Counter | c++filt |grep get
[60]    |         0|       0|FUNC |GLOB |0    |UNDEF  |int Counter<int>::get()

Using the flag -instlib, the definition of the template code resides in lib1.so, and lib2.so contains unresolved symbols that will use the definitions provided by lib1.so at run time.

Using Direct Binding to Record Dependencies at Compile Time

The linker flag -Bdirect causes the compile-time linker to record dependency information in the library at compile time rather than having the run-time linker figure it out at run time. In some instances, this might reduce the potential for the run-time linker to pick the wrong symbol.

Using the flag might also result in a startup time improvement because the run-time linker has to perform less work to locate the correct symbol and library.

However, there are potential downsides to using direct binding. When there are multiple definitions of a symbol, direct binding increases the chances of having more than one active definition of a single symbol in the program. If the program depends on all references to the symbol referring to the same physical address, the program could fail.

As an example of direct binding, consider the situation where two libraries declare a function of the same name, but with different implementations. This is demonstrated in the two libraries in Listing 8. Both libraries declare a function display(), and each library provides a different implementation.

Listing 8: Two Libraries Declaring the Same Function

$ more lib1.c
#include <stdio.h>

void display()
{
  printf("In lib1\n");
}

$ more lib2.c
#include <stdio.h>

void display()
{
  printf("In lib2\n");
}

Now, suppose we have two further libraries that each use the display function, but depend on the two different implementations. These two libraries are shown in Listing 9, together with an application that calls the second pair of libraries.

Listing 9: Libraries that Use the Same Symbol but Expect Different Implementations

$ more libu1.c
#include <stdio.h>

void display();

void use1()
{
  printf("In libu1\n");
  display();
}

$ more libu2.c
#include <stdio.h>

void display();

void use2()
{
  printf("In libu2\n");
  display();
}

$ more main.c
#include <stdio.h>

void use1();
void use2();

int main()
{
  use1();
  use2();
}

The result of compiling and running these libraries is shown in Listing 10.

Listing 10: Compiling and Running Code that Contains Duplicate Symbols

$ cc -G -Kpic -o lib1.so lib1.c
$ cc -G -Kpic -o lib2.so lib2.c
$ cc -G -Kpic -o libu1.so libu1.c -L. -R'$ORIGIN' -l1
$ cc -G -Kpic -o libu2.so libu2.c -L. -R'$ORIGIN' -l2
$ cc -o main main.c -L. -R'$ORIGIN' -lu1 -lu2
$ ./main
In libu1
In lib1
In libu2
In lib1

As might be expected, the run-time linker resolves both the calls to the function display() to the same symbol exported by lib1.so. However, that is not what was desired. Direct binding provides a way of having the two libraries bind to different instances of the same symbol, as shown in Listing 11.

Listing 11: Using Direct Binding to Link to Multiple Instances of the Same Symbol

$ cc -G -o lib1.so lib1.c
$ cc -G -o lib2.so lib2.c
$ cc -G -o libu1.so libu1.c -L . -R'$ORIGIN' -Bdirect -l1
$ cc -G -o libu2.so libu2.c -L . -R'$ORIGIN' -Bdirect -l2
$ cc -o main main.c -L. -R'$ORIGIN' -lu1 -lu2
$ ./main
In libu1
In lib1
In libu2
In lib2

When the libraries are linked using direct binding, the two libraries record the location of the symbols that satisfied their unresolved symbol definitions, and then they use these definitions at run time rather than relying upon the run-time linker to locate a definition for the undefined symbols.

Direct binding can also limit the ability to interpose on (replace) functions in the library.

Summary of Recommendations

You can work around some symbol binding problems by carefully ordering the libraries on the link line. Unfortunately, this might not be a permanent solution because changes in the library load order might still cause problems to surface.
Use the compile time flag -instlib=<library> to avoid implementations of template code appearing in multiple load objects.
The linker supports direct binding, which ensures that the exact library dependencies are recorded at compile time. The downside of direct binding is that multiple instances of the same data object or function can be defined.

Revision 1, 07/11/2011