1. Background

Subnormal numbers provide the guarantee that addition and subtraction of floating-point numbers never underflows; two nearby floating-point numbers always have a representable non-zero difference. Without gradual underflow, the subtraction a − b can underflow and produce zero even though the values are not equal. This can, in turn, lead to division by zero errors that cannot occur when gradual underflow is used.^[1]

Subnormal numbers were implemented in the Intel 8087 while the IEEE 754 standard was being written. They were by far the most controversial feature in the K-C-S format proposal that was eventually adopted,^[2] but this implementation demonstrated that subnormal numbers could be supported in a practical implementation. Some implementations of floating-point units do not directly support subnormal numbers in hardware, but rather trap to some kind of software support. While this may be transparent to the user, it can result in calculations that produce or consume subnormal numbers being much slower than similar calculations on normal numbers.

2. Performance Issues

Some systems handle subnormal values in hardware, in the same way as normal values. Others leave the handling of subnormal values to system software ("assist"), only handling normal values and zero in hardware. Handling subnormal values in software always leads to a significant decrease in performance. When subnormal values are entirely computed in hardware, implementation techniques exist to allow their processing at speeds comparable to normal numbers.^[3] However, the speed of computation remains significantly reduced on many modern x86 processors; in extreme cases, instructions involving subnormal operands may take as many as 100 additional clock cycles, causing the fastest instructions to run as much as six times slower.^[4][5]

This speed difference can be a security risk. Researchers showed that it provides a timing side channel that allows a malicious web site to extract page content from another site inside a web browser.^[6]

Some applications need to contain code to avoid subnormal numbers, either to maintain accuracy, or in order to avoid the performance penalty in some processors. For instance, in audio processing applications, subnormal values usually represent a signal so quiet that it is out of the human hearing range. Because of this, a common measure to avoid subnormals on processors where there would be a performance penalty is to cut the signal to zero once it reaches subnormal levels or mix in an extremely quiet noise signal.^[7] Other methods of preventing subnormal numbers include adding a DC offset, quantizing numbers, adding a Nyquist signal, etc.^[8] Since the SSE2 processor extension, Intel has provided such a functionality in CPU hardware, which rounds subnormal numbers to zero.^[9]

3. Disabling Subnormal Floats at the Code Level

3.1. Intel SSE

Intel's C and Fortran compilers enable the DAZ (denormals-are-zero) and FTZ (flush-to-zero) flags for SSE by default for optimization levels higher than -O0.^[10] The effect of DAZ is to treat subnormal input arguments to floating-point operations as zero, and the effect of FTZ is to return zero instead of a subnormal float for operations that would result in a subnormal float, even if the input arguments are not themselves subnormal. clang and gcc have varying default states depending on platform and optimization level.

A non-C99-compliant method of enabling the DAZ and FTZ flags on targets supporting SSE is given below, but is not widely supported. It is known to work on Mac OS X since at least 2006.^[11]

#include <fenv.h> #pragma STDC FENV_ACCESS ON // Sets DAZ and FTZ, clobbering other CSR settings. // See https://opensource.apple.com/source/Libm/Libm-287.1/Source/Intel/, fenv.c and fenv.h. fesetenv(FE_DFL_DISABLE_SSE_DENORMS_ENV); // fesetenv(FE_DFL_ENV) // Disable both, clobbering other CSR settings.

For other x86-SSE platforms where the C library has not yet implemented this flag, the following may work:^[12]

#include <xmmintrin.h> _mm_setcsr(_mm_getcsr() | 0x0040); // DAZ _mm_setcsr(_mm_getcsr() | 0x8000); // FTZ _mm_setcsr(_mm_getcsr() | 0x8040); // Both _mm_setcsr(_mm_getcsr() & ~0x8040); // Disable both

The _MM_SET_DENORMALS_ZERO_MODE and _MM_SET_FLUSH_ZERO_MODE macros wrap a more readable interface for the code above.^[13]

// To enable DAZ #include <pmmintrin.h> _MM_SET_DENORMALS_ZERO_MODE(_MM_DENORMALS_ZERO_ON); // To enable FTZ #include <xmmintrin.h> _MM_SET_FLUSH_ZERO_MODE(_MM_FLUSH_ZERO_ON);

Most compilers will already provide the previous macro by default, otherwise the following code snippet can be used (the definition for FTZ is analogous):

#define _MM_DENORMALS_ZERO_MASK 0x0040 #define _MM_DENORMALS_ZERO_ON 0x0040 #define _MM_DENORMALS_ZERO_OFF 0x0000 #define _MM_SET_DENORMALS_ZERO_MODE(mode) _mm_setcsr((_mm_getcsr() & ~_MM_DENORMALS_ZERO_MASK) | (mode)) #define _MM_GET_DENORMALS_ZERO_MODE() (_mm_getcsr() & _MM_DENORMALS_ZERO_MASK)

The default denormalization behavior is mandated by the ABI, and therefore well-behaved software should save and restore the denormalization mode before returning to the caller or calling code in other libraries.

3.2. ARM

AArch32 NEON (SIMD) FPU always uses a flush-to-zero mode, which is the same as FTZ + DAZ. For the scalar FPU and in the AArch64 SIMD, the flush-to-zero behavior is optional and controlled by the FZ bit of the control register – FPSCR in Arm32 and FPCR in AArch64.

Some ARM processors have hardware handling of subnormals.