Daily bit(e) of C++ | Hardened mode of standard library implementations
Daily bit(e) of C++ #384, Improve your development experience using hardened mode of the standard library implementations.
One of the tricky parts of programming is dealing with assumptions. With C++, assumptions can be deadly, as a violated assumption will likely lead to undefined behaviour, making the program invalid.
We can mitigate this problem to a large degree with proper engineering practices. Notably, good unit test coverage combined with sanitizers can catch most bugs.
Sanitizers can detect language-level undefined behaviour, even if that behaviour doesn’t manifest as a test failure. However, this approach has an issue. Sanitizers do not detect library-level UB, and while this is generally fine, it can make tracing issues back to their root cause tricky.
This is where the hardened mode of stdlibc++ and libc++ comes into play.
-fhardened
We will start with the -fhardened flag provided by GCC. Clang doesn’t support this umbrella flag; however, it does support each of the sub-features.
The TL;DR is that the -fhardened flag enables safety features that do not affect ABI. It covers a set of safety flags that address specific attacks and library-level checks (for both glibc and stdlibc++).
-D_FORTIFY_SOURCE=3
-D_GLIBCXX_ASSERTIONS
-ftrivial-auto-var-init=zero
-fPIE -pie -Wl,-z,relro,-z,now
-fstack-protector-strong
-fstack-clash-protection
-fcf-protection=full (x86 GNU/Linux only)Three protections for specific attacks
Three of the sub-features are designed to prevent specific attacks. I won’t go into the details, but you can easily find details for each attack by searching for the flag (high-quality sources for these topics are GCC, LLVM, RedHat and LWN).
-fstack-protector-strong
-fstack-clash-protection
-fcf-protection=full
Trivial initialization (-ftrivial-auto-var-init=pattern)
Default initialization in C++ can be tricky, as it can leave variables uninitialized.
This flag ensures that default initialization initializes variables with a pattern of values. GCC uses 0xFE and Clang 0xAA.
The downside of this feature is that if you require an uninitialized variable, you have to annotate it with the __attribute__ ((uninitialized)).
Generated assembly from GCC.
funa():
push rbp
mov rbp, rsp
sub rsp, 4194304
lea rax, [rbp-4194304]
mov edx, 4194304
mov esi, 254
mov rdi, rax
call memset
nop
leave
ret
funb():
push rbp
mov rbp, rsp
sub rsp, 4194184
nop
leave
retPosition-independent executable (-fPIE -pie)
The main point of position-independent executables is that they can operate with address space layout randomization (ASLR), randomizing the positions code, which makes exploits more difficult.
The related feature is Relocation Read-Only (-Wl,-z,relro,-z,now), which prevents modification of the Global Offset Table (locations of functions from dynamically linked libraries) after the program startup.
The downside is that this prevents tools like ltrace from working.
Binary with lazily loaded libraries.
$ g++ safe.cc -g -fno-PIE -no-pie -Wl,-z,lazy
$ ltrace -C ./a.out 1>/dev/null
std::ios_base::Init::Init()(0x404191, 0xffff, 0x7ffd5eeebb98, 0x403e00) = 0x7f9f058fc4b8
__cxa_atexit(0x4010a0, 0x404191, 0x404040, 0x7f9f05b2dda0) = 0
std::basic_ostream<char, std::char_traits<char> >& std::operator<< <std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*)(0x404080, 0x402004, 0x7ffd5eeebb98, 0x403e08) = 0x404080
std::ios_base::Init::~Init()(0x404191, 0, 1, 4) = 0x7f9f05b2be80
+++ exited (status 0) +++Position independent executable with Relocation Read-Only.
$ g++ safe.cc -g -fPIE -pie -Wl,-z,relro,-z,now
$ ltrace -C ./a.out 1>/dev/null
+++ exited (status 0) +++C standard library (-D_FORTIFY_SOURCE=3)
The first set of library-level checks is the fortification of the C standard library, specifically when dealing with buffers with known size.
Because of this, this flag only works when optimizations are also enabled.
Detected buffer overflow.
*** buffer overflow detected ***: terminated
Program terminated with signal: SIGSEGVC++ Standard library
The last part of hardening is where GCC and Clang diverge. This is also where things get interesting.
stdlibc++ offers two levels:
_GLIBCXX_ASSERTIONS (non-ABI breaking)
_GLIBCXX_DEBUG (ABI breaking)
libc++ offers three levels, all of which are non-ABI breaking:
_LIBCPP_HARDENING_MODE_FAST
_LIBCPP_HARDENING_MODE_EXTENSIVE
(roughly equivalent to _GLIBCXX_ASSERTIONS)_LIBCPP_HARDENING_MODE_DEBUG
Note that for libc++, you don’t define these macros. Instead, these are values for the _LIBCPP_HARDENING_MODE macro, e.g., -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_DEBUG.
While most of the other features enabled by -fhardened are either performance neutral or have only negligible impact on performance, _GLIBCXX_ASSERTIONS can be fairly intrusive. However, because this is also the most impactful feature for C++ development, you should use this flag, at least for your local development loop.
And to justify the previous statement, let’s explore some of the features you get for your trouble.
_GLIBCXX_ASSERTIONS and _LIBCPP_HARDENING_MODE_EXTENSIVE
If you are familiar with the standard library, you might know that most operations have narrow contracts. For example, using the element access operator on a vector is only defined for values [0, size). This is precisely where assertions/hardening comes in.
Assertion '__n < this->size()' failed.And we get the same level of enforcement for the other methods.
Assertion '!this->empty()' failed.Another good example is std::views::iota. You might be tempted to express a decreasing sequence; however, iota works explicitly by incrementing the initial value, and a decreasing sequence cannot be expressed.
Assertion 'bool(__value <= __bound)' failed.While stdlibc++ and libc++ are mostly at parity, there are some differences.
Only caught by stdlibc++:
Assertion '__p == nullptr || __p != _M_ptr' failed.Debug mode
Finally, we need to talk about the debug mode. While the previously mentioned checks are beneficial, the debug mode is where you will find the heavy-duty checks that find subtle and hard-to-find bugs.
However, naturally, most of the checks are very expensive. This is not an exaggeration; checks in debug can change the big-O complexity of operations.
Currently, libc++ is in the process of implementing these checks (you can follow the progress here), so I will only demonstrate using GCC.
Side note on ABI
The stdlibc++ implementation took the ABI-breaking route by replacing containers and iterators with instrumented versions that are not ABI-compatible. This means that a mixed binary will generally not link.
If you have precompiled functions that return containers, the resulting binary will link (since return types are not encoded in the mangled name), but the result will be a corrupted binary that contains undefined behaviour.
$ g++ -c lib.cc
$ g++ main.cc -D_GLIBCXX_DEBUG lib.o -o main
$ ./main
free(): invalid pointer
AbortedThis is also why stdlibc++ doesn’t provide a debug version of std::string. This container is pervasively used throughout the standard library as a result type. If you want to use the debug mode with GCC, ensure you have built your entire source base with this flag.
On the other hand, libc++ is taking the non-ABI-breaking stance. This means that it will enforce whatever checks can be enforced using the ABI provided by the platform. If you want to get checks requiring container and iterator instrumentation, you will have to build your libc++ with that support.
Debug features
The main functionality offered by the debug mode in stdlibc++ is built around container-aware iterators. In the simplest form, this includes checks for dangling iterators.
In function:
constexpr gnu_debug::_Safe_iterator<_Iterator, _Sequence,
_Category>::reference gnu_debug::_Safe_iterator<_Iterator, _Sequence,
_Category>::operator*() const [with _Iterator = gnu_cxx::
normal_iterator<int*, std::vector<int, std::allocator<int> > >;
_Sequence = std::debug::vector<int>; _Category =
std::forward_iterator_tag; reference = int&]
Error: attempt to dereference a singular iterator.
Objects involved in the operation:
iterator "this" @ 0x7ffea687e7d0 {
type = gnu_cxx::normal_iterator<int*, std::vector<int, std::allocator<int> > > (mutable iterator);
state = singular;
references sequence with type 'std::debug::vector<int, std::allocator<int> >' @ 0x7ffea687e800
}
Program terminated with signal: SIGSEGVIterators passed to algorithms are checked to determine whether they represent a valid range.
In function:
constexpr _Tp std::accumulate(_InputIterator, _InputIterator, _Tp) [with
_InputIterator = gnu_debug::_Safe_iterator<gnu_cxx::
normal_iterator<int*, vector<int, allocator<int> > >,
debug::vector<int>, random_access_iterator_tag>; _Tp = int]
Error: function requires a valid iterator range [first, last).
Objects involved in the operation:
iterator "first" @ 0x7ffd67954750 {
type = gnu_cxx::normal_iterator<int*, std::vector<int, std::allocator<int> > > (mutable iterator);
state = past-the-end;
references sequence with type 'std::debug::vector<int, std::allocator<int> >' @ 0x7ffd679547b0
}
iterator "last" @ 0x7ffd67954780 {
type = gnu_cxx::normal_iterator<int*, std::vector<int, std::allocator<int> > > (mutable iterator);
state = dereferenceable (start-of-sequence);
references sequence with type 'std::debug::vector<int, std::allocator<int> >' @ 0x7ffd679547b0
}
Program terminated with signal: SIGSEGVCounted algorithms are also checked.
In function:
constexpr _OIter std::copy_n(_IIter, _Size, _OIter) [with _IIter =
gnu_debug::_Safe_iterator<gnu_cxx::normal_iterator<int*, vector<int,
allocator<int> > >, debug::vector<int>, random_access_iterator_tag>;
_Size = int; _OIter = gnu_debug::_Safe_iterator<gnu_cxx::
normal_iterator<int*, vector<int, allocator<int> > >,
debug::vector<int>, random_access_iterator_tag>]
Error: attempt to subscript a dereferenceable (start-of-sequence) iterator 6
step from its current position, which falls outside its dereferenceable
range.
Objects involved in the operation:
iterator "first" @ 0x7ffd49e98670 {
type = gnu_cxx::normal_iterator<int*, std::vector<int, std::allocator<int> > > (mutable iterator);
state = dereferenceable (start-of-sequence);
references sequence with type 'std::debug::vector<int, std::allocator<int> >' @ 0x7ffd49e98740
}
Program terminated with signal: SIGSEGVThe debug containers provide the checks from _GLIBCXX_ASSERTIONS with the verbose debug output.
In function:
constexpr std::debug::vector<_Tp, _Allocator>::reference std::
debug::vector<_Tp, _Allocator>::operator[](size_type) [with _Tp = int;
_Allocator = std::allocator<int>; reference = int&; size_type = long
unsigned int]
Error: attempt to subscript container with out-of-bounds index 5, but
container only holds 5 elements.
Objects involved in the operation:
sequence "this" @ 0x7ffc817482e0 {
type = std::debug::vector<int, std::allocator<int> >;
}
Program terminated with signal: SIGSEGVDebug mode also covers expensive checks, such as validating that the comparator for std::sort satisfies the strict-weak-ordering requirement.
In function:
constexpr void std::sort(_RAIter, _RAIter, _Compare) [with _RAIter =
gnu_debug::_Safe_iterator<gnu_cxx::normal_iterator<int*, vector<int,
allocator<int> > >, debug::vector<int>, random_access_iterator_tag>;
_Compare = ranges::detail::make_comp_proj<main()::<lambda(int, int)>,
std::identity>(main()::<lambda(int, int)>&,
std::identity&)::<lambda(auto:6&&, auto:7&&)>]
Error: comparison doesn't meet irreflexive requirements, assert(!(a < a)).
Objects involved in the operation:
instance "functor" @ 0x7ffee62fcc20 {
type = std::ranges::detail::make_comp_proj<main::{lambda(int, int)#1}, std::identity>(main::{lambda(int, int)#1}&, std::identity&)::{lambda(auto:1&&, auto:2&&)#1};
}
iterator::value_type "ordered type" {
type = int;
}
Program terminated with signal: SIGSEGVConclusion
The debug mode provides the best checks for the development loop. However, the GCC stdlibc++ implementation is problematic due to the chosen ABI approach.
I hope the LLVM libc++ debug mode will offer a more systemic approach. Notably, if we can get the ability to compile a special version of libc++ for local development, it would be a very clean solution that would provide the benefits of debug mode without affecting the rest of the Continuous Integration.