Fixed-point number support?

[DaZombieKiller]

I discovered Zig yesterday and have spent pretty much all my time since then delving into it and reading various materials on it. Loving it so far.

One feature that I feel is missing currently is built-in support for fixed-point numbers. As the language doesn't allow operator overloading, this isn't something I could implement in user code and use like any other numeric type. For reference, the Embedded C standard specifies support for such numbers (refer to Annex A), although I'm not aware of any C compiler that supports them for a desktop target.

I found it interesting that Zig offers support for variable-width integers through i# and u#. I wonder if this could be applied to fixed-point numbers by supporting some form of Q-notation, perhaps?

[andrewrk]

LLVM has

• @llvm.smul.fix.*
• @llvm.umul.fix.*

https://llvm.org/docs/LangRef.html#fixed-point-arithmetic-intrinsics

But that's it. Interesting. I'll have to learn why these intrinsics are provided. It may be that some architectures provide some fixed point multiplication operations.

The null hypothesis here is that you would have to implement fixed-point in userland, and yes operations would be function calls rather than overloaded operators.

[ryanavella]

Several architectures provide fixed-point multiplication operations. Atmel's AVR UC3 provides fixed-point DSP arithmetic, but has no support for floating point representations. This means that e.g. FFT's are more performant than in comparable microcontrollers with a FPU. This is not unusual for a microcontroller specialized for DSP applications.

[Diltsman]

Most processors that I use at work don't even have a hardware multiply. Generally we work in ADC counts, but sometimes we will work in fixed-point encoded in 8 or 16 bit integers.

It seems to me that, if LLVM IR has minimal support for fixed-point numbers (only data type, no operations), that it wouldn't be terribly difficult to map fixed-point operations to sequences of bit-shifts and integer operations.

Fixedpoint Numbers

A smooth upgrade for integers which approximates real numbers and maximizes zen.

Why do we want this?

They are useful for
a) financial systems
b) software that benefits from both cross platform determinism and fractional precision, such as video games using lockstep networking
c) software running on hardware with no floating point support
d) software designed to maximize performance of small real numbers using simd operations, such as machine learning.

What is this and how do I use it?

Fixedpoint numbers are an alternative approximation of real numbers, compared to the floats you're probably familiar with. They're integer hardware under the hood, but with a specific number of bits allocated to represent the integer part, and a specific number to represent the fractional part.

In concept, they look like this

// 2.8125, with 4 bits allocated for the integer part, and 4 bits allocated for the fractional part
0010.1101
  ^  ^^ ^
   \  \\ \
    \  \\ \
     \  \\ '0.0625' (1/16)
      \  \'0.25' (1/4)
       \  \
        \  '0.5' (1/2)
         \
          '2'

In code, they look like this:

// -1.5, in signed fixedpoint, with 4 bits allocated for the integer, 
// and 4 for the fractional
const negative_one_point_five: i4_4 = -1.5;

// 0.25, in unsigned fixedpoint, with zero bits allocated for the integer, 
// and 2 for the fractional
const one_quarter: u0_2 = 0.25

// Vector of 8 signed fixedpoints, with 20 bits allocated for the integer part, 
// and 12 for the fractional part, with each lane initialized to the closest 
// possible value to 1337.1337
const lots_of_numbers: @Vector(8, u20_12) = @splat(@truncateLow(u20_12, 1337.1337));

u#_# and i#_# represent unsigned and signed fixedpoint numbers, respectively, with the first number representing the integer bit-width, and the second number (after the _) representing the fractional bit-width. These bit-widths define its radix point, and are what we call the fixedpoint's format.

Zig supports arbitrary bit-width fixedpoint. The maximum allowed total bit-width of a fixedpoint type (i.e., the sum of the integer and fractional bit-widths) is 65535.

If an assigned comptime value does not fit perfectly into the format, it is a compile error.

const impossible: i0_2 = 0.125; // Compile error: literal cannot fit into fixedpoint format

If the format has a fractional bit-width of 0, it is called an integer. In Zig, fixedpoint numbers and integers are actually one and the same!

const one_point_two_five: i8_8 = 1.25;
const golly_gee: i8 = @fixCast(one_point_two_five); // 1.0

Some operations require 'integers' as input. If you try to use a non-integer fixedpoint in them, you'll get a compile error.

const a: i2_4 = 3.25;
const shift_amount: i5_3 = 2.5;
const oops = a << shift_amount; // Compile error: shift_amount must be an integer

Fixedpoint numbers have the following operations defined:

• Addition (Wrapping) +%

• Addition (Non-Wrapping) +

• Addition (Overflowing) @addWithOverflow

• Addition (Saturating) #1284

• Subtraction (Wrapping) -%

• Subtraction (Non-Wrapping) -

• Subtraction (Overflowing) @subWithOverflow

• Subtraction (Saturating) #1284

• Multiplication (Wrapping) *%

• Multiplication (Non-Wrapping) *

• Multiplication (Overflowing) @mulWithOverflow

• Multiplication (Saturating) #1284

• Division (Unsigned, Const Signed) /

• Division (Signed) @divExact, @divRoundTowardZero, @divRoundTowardNeg

• Remainder @rem

• Modulo %, @mod

• Left Shift (Truncating) <<, <<=

• Left Shift (Non-Truncating) @shlExact

• Left Shift (Overflowing) @shlWithOverflow

• Sticky Left Shift (Truncating) <<<, <<<= Proposal: Explicit Shift Operators #5220

• Sticky Left Shift (Non-Truncating) @shlStickyExact Proposal: Explicit Shift Operators #5220

• Sticky Left Shift (Overflowing) @shlStickyWithOverflow Proposal: Explicit Shift Operators #5220

• Right Shift (Truncating) >>, >>=

• Right Shift (Non-Truncating) @shrExact

• Right Shift (Overflowing) @shrWithOverflow

• Sticky Right Shift (Truncating) >>>, >>>= Proposal: Explicit Shift Operators #5220

• Sticky Right Shift (Non-Truncating) @shrStickyExact Proposal: Explicit Shift Operators #5220

• Sticky Right Shift (Overflowing) @shrStickyWithOverflow Proposal: Explicit Shift Operators #5220

• Negation (Wrapping) -

• Negation (Non-Wrapping) @negExact

• Negation (Saturating) #1284

• Negation (Overflowing) @negWithOverflow

• Bitwise And &, &=

• Bitwise Or |, |=

• Bitwise Xor ^, ^=

• Implicit widening coercion (as long the individual integer bit-width and fractional bit-width will fit into the destination type, it will be implicitly shifted to fit)

• Explicit Truncating Floatingpoint to Fixedpoint conversion @floatToFix(comptime T: type, float: var) T
  (Converts a floatingpoint to a fixedpoint, truncating as necessary.)

• Explicit Exact Floatingpoint to Fixedpoint converstion @floatToFixExact(comptime T: type, float: var) T
  (Converts a floatingpoint to a fixedpoint. If the floatingpoint number cannot fit in the destination type, it invokes safety-checked Undefined Behavior.)

• Explicit Truncating Fixedpoint to Floatingpoint conversion @fixToFloat(comptime T: type, float: var) T
  (Converts a fixedpoint to a floatingpoint, truncating as necessary.)

• Explicit Exact Fixedpoint to Floatingpoint conversion @fixToFloatExact(comptime T: type, float: var) T
  (Converts a fixedpoint to a floatingpoint. If the fixedpoint number cannot fit in the destination type, it invokes safety-checked Undefined Behavior.)

• Explicit Truncating Fixedpoint to Fixedpoint Cast @fixCast(comptime T: type, value: var) T
  (Converts a fixedpoint type to another fixedpoint, truncating the value as necessary.)

• Explicit Exact Fixedpoint to Fixedpoint Cast@fixCastExact(comptime T: type, value: var) T
  (Converts a fixedpoint to another fixedpoint while keeping the same numerical value. Attempting to convert a number which is out of range of the destination type results in safety-protected Undefined Behavior.)

• Truncate High Bits @truncateHigh(comptime T: type, value: var) T
  (Truncates high bits from a fixedpoint type, resulting in a smaller or same-sized fixedpoint type composed of the low bits, which may no longer store a sensible fixedpoint value. Useful for unsafe bit-surgery that @fixCast doesn't allow. e.g. const a: i4_4 = ...; const b = @truncateHigh(i2, a);

• Truncate Low Bits @truncateLow(comptime T: type, value: var) T
  (Truncates low bits from a fixedpoint type, resulting in a smaller or same-sized fixedpoint type composed of the high bits, which may no longer store a sensible fixedpoint value. Useful for unsafe bit-surgery that @fixCast doesn't allow. e.g. const a: u2_6 = ...; const b = @truncateLow(u3, a);

• Floor @roundTowardNeg

• Ceil @roundTowardPositive

• Trunc @roundTowardZero

• Round @roundTowardInfinity

• Bit Cast @bitCast

• Bit Reverse @bitReverse

• Byte Swap @byteSwap

• Pop Count @popCount

• Count Leading Zeros @clz

• Count Trailing Zeros @ctz

• Square Root @sqrt (See 'Alternative Implementations' below)

• Sin @sin (See 'Alternative Implementations' below)

• Cos @cos (See 'Alternative Implementations' below)

• The rest of the trig functions (...which are in in std.math? How do @sin and @cos relate? I feel I'm missing something here.)

• Pretty much everything in std.math, really.

• Pow (Wrapping) @pow

• Pow (Non-Wrapping) @powExact

• Pow (Overflowing) @powWithOverflow

• Pow (Saturating) #1284

• Exp (Wrapping) @exp

• Exp (Non-Wrapping) @expExact

• Exp (Overflowing) @expWithOverflow

• Exp (Saturating) #1284

• Exp2 (Wrapping) @exp2

• Exp2 (Non-Wrapping) exp2Exact

• Exp2 (Overflowing) @exp2WithOverflow

• Exp2 (Saturating) #1284

• Log (Wrapping) @log

• Log (Non-Wrapping) @logExact

• Log (Overflowing) @logWithOverflow

• Log (Saturating) #1284

• Log2 (Wrapping) @log2

• Log2 (Non-Wrapping) @log2Exact

• Log2 (Overflowing) @log2WithOverflow

• Log2 (Saturating) #1284

• Log10 (Wrapping) @log10

• Log10 (Non-Wrapping) @log10Exact

• Log10 (Overflowing) @log10WithOverflow

• Log10 (Saturating) #1284

• Abs (Wrapping) @abs

• Abs (Non-Wrapping) @absExact

• Abs (Saturating) #1284

• Abs (Overflowing) @absWithOverflow

• Minimum Value @minNum

• Maximum Value @maxNum

• Minimum @minimum

• Maximum @maximum

• Copy Sign @copySign

These operations are also implemented on @Vectors with fixedpoint element types.

Our Greasy Forearms

Fixedpoint numbers add an additional approximation of real numbers. The Zig compiler currently makes some assumptions that all real number representations must be floats.

The following breaking changes will be prudent:

1. comptime_int -> comptime_fix(i, f)

Furthermore, a number of operations that assume floats should now be made generic,

1. @fabs -> @abs
2. @fshl -> @shl
3. @fshr -> @shr
4. @floor
5. @ceil
6. @trunc
   ...and many others

@truncate and @trunc are now less clear than they already were, and should be revisited. @truncateHigh and @truncateLow are suggested as they follow common use cases, while @roundTowardPositive, @roundTowardNegative, @roundTowardZero, and @roundTowardInfinity are suggested to communicate intent precisely. The exact names of these functions are in discussion.

Fixedpoint numbers also extend the current integer codebase, causing integers (e.g. i8) to be a shorthand for fixedpoints with no fractional part (e.g. i8_0). This adds valuable functionality without increasing the language's complexity or api surface area.

LLVM's builtins for fixedpoint math should be used whenever possible, otherwise we implement it within Zig atop the existing LLVM integer code. Thankfully, basic operations are quite trivial to implement:

// I know this definition is, with mild imagination, recursive. 
// Please mentally substitude llvm integer intrinsics.
const i8_8 = struct {
    bits: i16,
    
    const integer_bit_width: usize = 8;
    const fractional_bit_width: usize = 8;
    
    // Add is the same as integers
    const add = fn (self: Self, other: Self) Self {
        return self.bits + other.bits;
    };

    // Sub is the same as integers
    const sub = fn (self: Self, other: Self) Self {
        return self.bits - other.bits;
    };

    // Multiplication requires the factors be widened and
    // the product be left-shifted by the fractional bitcount,
    // to offset scaling
    const mul = fn (self: Self, other: Self) Self {
        return Self {
            bits: fixCast(i16, (fixCast(i32, self) * fixCast(i32, other) >> Self.fractional_bit_width)
        };
    }

    // Division requires the dividend be widened and 
    // left-shifted by the fractional bitcount to offset scaling
    const div = fn (self: Self, other: Self) Self {
        return Self {
            bits: fixCast(i16, (fixCast(i32, self) << Self.fractional_bit_width) / fixCast(i32, other))
        };
    }
}

However, stdlib trigonometric functions and square root must, for the purpose of the users sanity, return the mathematically correct answer for the number of bits. It is suggested that whatever methods are used (taylor series, slippery polynomial approximations, newtons method), the expansion and/or iterations be comptime tailored to the fixedpoint format. These implementations are not for those uninclined toward numerical analysis.

Drawbacks

• Will break existing code, although that's par for the course nowadays
• More stuff tossed into the compiler just so we can have infix ops and native-looking @roundTowardZero, etc.
• sin, cos, tan, acos, asin, atan, sqrt and friends nontrivial to implement. Numerical analysis is addicting and may eat your developers alive.

Rationale and alternatives

Alternative Syntax

The proposed i#_#/u#_# syntax matches zig's i# / u# syntax for integers, with a _# added on as a status-quo-parser friendly delineation between the integer and fractional.

const foo: i12_20 = 32.4; // not unexpected, really.

Alternatively, . instead of _ would not only match the standard Q format syntax, but turn the format names into uniquely reserved words not available in userspace, not unlike the current infix operations. This solves the issue of squatting the whole i#_# / u#_# namespace.

const foo: i12.20 = 32.4; // the platonic ideal!

If special-casing the . syntax is too egregious, f could be used in its place, matching the i/u moniker, but creating a sort of alphabet soup.

const foo: i12f20 = 32.4; // is this an integer12/float20 lovechild?

Variations of the Q syntax are also possible, such as qi12_20 / qu12_20, qi12.20 / qu12.20, iq12_20 / uq12_20, iq12.20 / uq12.20, though they add typing for no real (geddit?) advantage.

Finally, we could simply keep integer syntax at status quo and require any fractionals to use std.meta.FixedPoint(i, f), though it throws comeliness out with the bikeshed.

const foo: std.meta.FixedPoint(12, 20) = 32.4; // where's the romance?

Alternative Implementations

The trignometric functions and sqrt are likely to fall into an awkward position where users expect them to be 100% accurate to the bit-width, but in practice 100% accuracy is often second-place to performance for most numerics applications. It may be practical to provide functions for users to build custom approximations suited to their use-case.

// Returns the value of sin(x), exact to the bit-width.
// If hardware support is unavailable, uses @sinPoly with an 
// internally specified order, then iterates from an initial guess using 
// newton's method.
const @sinExact = fn(value: var) var { ... };

// Returns the approximate sin of value, via a polynomial of the 
// specified order. 
// See Prior Art #5 and #6 for example implementations
const @sinPoly = fn(comptime order: comptime_int, value: var) var { ... };

This pattern could be expanded to other expensive functions, like:

// Returns the quotient of dividend and divisor. 
// Caller guarantees denominator != 0 and 
// @divTrunc(numerator, denominator) * denominator == numerator.
// If hardware support is unavailable, computes via @divPoly with an 
// internally specified order, then iterates from an initial guess using 
// newton's method.
// See Prior Art #7 for a hardware-accelerated example implementation
const @divExact = fn(dividend: var, divisor: var) var { ... };

// Returns the approximate quotient of dividend and divisor, via 
// polynomials of the specified order.
// See Prior Art #7 for a hardware-accelerated example implementation
const @divPoly = fn(comptime order: comptime_int, dividend: var, divisor: var) var { ... };

While mirroring the reality of nontrivial fixedpoint numerics, and immensely practical for many situations, this increases api surface area and may feel a bit too 'batteries included' for Zig.

Alternative Formats

Fixedpoint numbers are the most performance-efficient alternative approximation of real numbers beside hardware floats. They are not the only alternative, however.

• Unum3's Posits and Valids are a convincing alternative to floats when it comes to space efficiency and accuracy, but are slower and more complicated to implement than fixedpoint.
• Interval Arithmetic, implemented via integer arrays, could be used to represent values between a range of [a..b]. A very convenient option for mathematics involving unknowns, but sidesteps the problem of reals altogether by worsening the accuracy, and makes most mathematics impractical even when aggressively averaging back to a single value.
• Floats claim determinism when the floating point unit is set to 'strictly' follow the IEE754 standard, but not only does this lower average performance, it is also questionably deterministic in practice.
• 16 bit floats are implemented in some hardware, but only compete on size, and have all the problems of larger floats.

If we don't implement fixedpoint into the language, userspace libraries will rise to the challenge, but be hindered by Zig's lack of ǒ̷̹̙̠̲p̶̞͗̐̐͑ḙ̴̡͘r̴̞̂͆ạ̵̠͚̬̅́̾͒ť̶̛̗̈́o̸̡͕̼͊͘r̶̟̝̗̐̽̽̚ ̷̞̤̻͂̀͆͠ó̴͓̼̺̺̒͐̉v̷̛̥e̶̢͔̲̪̔̓ŕ̴̞̆̚͜ļ̸̝̏̿ỏ̵̼̦͋̋́a̷̼̟͍̚̚͘d̴̩̮͎̅̄̋i̵̭͔͉̰̕n̵̝̩͂́g̸̤̈.

As the elementary math operations for fixedpoint numbers use trivially more operations than hardware integer mathematics (an occasional shift), it is unlikely that any alternative formats would warrant language implementation, as they are too complex to be practical without hardware support. However, if hardware support is implemented for other alternative formats, such as Posits and Valids, implementing fixedpoint puts us well over the edge of the slipperly slope to implementing those as well.

A reasonable alternative that lets us avoid the need to futureproof the compiler and our technical debt against any practical future number format, is to implement o̸̢̳̝̘͓̰̲̗̫͈͉͓̱͆̄̑̽̓͜p̷̛͔̹͚̅̈́͂̀̓̌͜ę̸̠͗͆̌̋̋̏̈́̋̏̈́̃́̚͠r̷͕̗͓͎̗͑̾͋͜a̶̹͇̯̒t̷̺̔̆̾͗͑͗̑͌͌͜͠ö̶̧̢̤̯̞̹̲̗̳͉͓̪͈͚̈̍͐̅̿̐̀̇̕̚͘ŕ̷̡̧̯̻̞̜̲̭͉͉̮͉̤̟̾̈́͂̏͋͌̔̆̇̐͆͘͘͝͠ͅ ̴͎̺̙͎̹̯̘͇̘̊̿̍̿́͑̉̾̒̾͐͊̓̐͘o̵̧̟͓̜̩͎̦͙̮̺͔̦̎͑̃̃̈̀̉͐́̈́̽̕v̵̬͍̙̰̘̟͚̦̱̞̠͈͔̈́e̴̡̦͓͙̤̜͇̮͉̪̲͖̘̐̀̅ͅr̶̜̳͖͇̮̟̖̱͇̼̀̒͂̏͑̕l̴̡̪̞̮̖̟̥̯̰̰̙͙̯̻̺̐̄́̓̀̎̇͆̚͝͝ͅo̴̩̜̟͍̽à̷̙̱̻͕͍̼̥̳͇͈̗͜d̸̹̜̯̭̙̠͋̈́́̊ï̷̢̲͈͈̱̠͔͔̪͋̽̏͐̈͂̎̾̈́̄͘͘͠n̷̢͉̣̺̝̺̥̲̩̈́͐̌́͝ͅǵ̴͚̹.

Alternative Social Constructs

Aside from syntax and overall grooviness, nothing we're doing here is really specific to Zig. Our software implementation could be upstreamed into LLVM for other languages to benefit from.

Prior art

1. Fixed-point arithmetic, Wikipedia article
2. Fixed Point Arithmetic and Tricks
3. The Tonc Tutorial on Fixed-Point Numbers and LUTs
4. Doing it Fast: Fixed Point Arithmetic
5. Another fast fixed-point sine approximation
6. Nvidia's sin approximation used in CG
7. Fast fixed-point divider based on Newton-Raphson method and piecewise polynomial approximation
8. ZigGBA's Fixedpoint Type
9. libfixmath, an implementation the author does not particularly care for.
10. capsize, a rather brutal macro-based implementation in Rust. Includes many approximation functions.
11. fixed, a less featureful, generics-based Rust implementation.

• ...and many more, including implementations.

Unresolved questions

None. Absolutely None. If you've read this far, how can you doubt?

Future possibilities

Fixedpoint numbers' small size, cross-platform support, enforced determinism, and computational simplicity put them in an ideal niche that floats don't touch.

The main barrier to wider adoption has been lack of ease of use - even if it makes more sense to use fixedpoint, finding a good fixedpoint library, or writing one's own, will make users "just use floats", unless they absolutely have to.

Making these numerics first-class citizens in Zig with the power of comptime can open up a whole class of application development to less-specialized programmers. As Zig takes over the world, this will encourage hardware vendors to implement more fixedpoint functions in silicon, bringing us to a bright and happy future of @Vector(64, i12_4), free of the scourge of -0 and NaN.

[Diltsman]

Also, basic integer types could be thought of as special cases of fixed point numbers. i32 is equivalent to i32_0 in the proposed syntax.

Also, basic integer types could be thought of as special cases of fixed point numbers. i32 is equivalent to i32_0 in the proposed syntax.

Interesting point. Operations like @intToFloat could then be dropped in support of @fixToFloat, keeping the fn sig footprint down to two types, except in cases where an integer (i.e. no fractional bits) is explicitly required.

Edit: Updated the RFC accordingly.