PIC and PIE executables

For years most operating systems have run each process in its own virtual address space. Not only does this mean that no process can `accidentally' read or modify data of another, but it also means that each process has full access to a nice, empty, address space.

So a process can rely on being loaded at the same address every time it is executed, and all of its local variables and functions also have guaranteed addresses. This makes accessing them quick and easy.

Shared libraries are different. They must be relocatable, because when they are built they do not know which other shared libraries an executable will use them with. This cannot even be fixed at compile time, for one point of shared libraries is that the version used at runtime may differ from that used at compile time. So their code must be position independent, and there is the need for a small amount of indirection when accessing their data and functions, for it is necessary to add an offset representing the address at which the library actually got loaded into memory to any references to addresses within it. This results in a slight size and performance penalty.

Given that shared libraries are moveable through necessity, modern practice is to make the addresses at which they are loaded deliberately random and unpredictable, for this makes exploiting certain forms of buffer overflow harder. But this still leaves the main executable loading at a very predictable address, generally 0x00400000 on Linux on x86_64, for instance.

Very modern practice is to make the whole executable position independent, and to randomise its start address too each time it is used.

Linking issues

Compilers need to be told to produce position independent code. Currently there are three options, don't, generate position independent code suitable for shared libraries (PIC), and generate position independent code suitable for position independent executables (PIE). The compatibility matrix is something like:

neither: code cannot be used in a shared library, nor be included in a PIE executable.

PIE: code cannot be used in a shared library, but can be included in both a PIE and non-PIE executable.

PIC: code can be used anywhere.

However the overheads of PIC are greater than those of PIE which are greater than neither.

In Ubuntu LTS gcc defaulted to neither, up to and including Ubuntu 16.04 LTS. With Ubuntu 18.04 LTS gcc defaults to PIE (-fpie), which means that one cannot include objects which were compiled under 16.04 with its default settings. But objects compiled under 16.04 with -fpie can be used under both 16.04 and 18.04, or one can use -no-pie under 18.04 to revert to 16.04's behaviour. Different distributions may set different defaults, as it depends on the configuration of gcc. Ubuntu switched at 17.04.

The link error is typically something like

/usr/bin/ld: libfoo.a(bar.o): relocation R_X86_64_32 against `baz' can not be used when making a PIE object; recompile with -fPIC

Shared libraries are not an issue, as they are always built as PIC. It is static libraries (.a) and object files (.o) which might cause trouble.

Compilers' defaults


Up to Ubuntu 16.04 inclusive: neither
From Ubuntu 18.04: PIE (one can still link against old non-PIE static libraries by compiling with -no-pie)

NAG Fortran

As GCC, but its own run-time libraries appear to be PIE.

Intel Fortran/C/C++

Neither, even on Ubuntu 18.04. So static libraries built with icc will not work with gcc unless one over-rides the defaults.

Sun/Oracle Fortran/C/C++

Neither, even on Ubuntu 18.04.

What is what?

Executable files


$ file /bin/ls
/bin/ls: ELF 64-bit LSB executable, x86-64, [...]


$ file /bin/ls
/bin/ls: ELF 64-bit LSB shared object, x86-64, [...]

Object files

$ file hello.o
hello.o: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped

in all three cases, which is unhelpful.

$ readelf -a hello.o | grep  R_X86_64
000000000005  00050000000a R_X86_64_32       0000000000000000 .rodata + 0
00000000000a  000a00000002 R_X86_64_PC32     0000000000000000 puts - 4
000000000020  000200000002 R_X86_64_PC32     0000000000000000 .text + 0

for unrelocatable code, but

$ readelf -a hello.o | grep  R_X86_64
000000000007  000500000002 R_X86_64_PC32     0000000000000000 .rodata - 4
00000000000c  000b00000004 R_X86_64_PLT32    0000000000000000 puts - 4
000000000020  000200000002 R_X86_64_PC32     0000000000000000 .text + 0

for relocatable (PIC or PIE): note the appearance of R_X86_64_PLT32.