Create cosmwasm-math

[webmaster128]

This is a new package with the following goals:

• Versioning independent of cosmwasm-std, such that CosmWasm 0.16 and 1.0 developers can use the same library
• Add features required by defi users and implement them in a correct and heavility tested way
• Ensure a small footprint in terms of code size and performance
• Ensure we don't pull in float operations

TODOs:

• Implement Sum for all types consistently (Use the same Sum implementation for all number types #1187)
• Create checked_multiply_ratio returning an Option allowing to recover from overflow
• Implement floor/ceil for all decimals
• Copy over Uint64, Uint128, Uint256, Uint512, Decimal and Decimal256
• Let all the checked_* functions returns Option instead of StdResult for better stdard library complience and avoiding performance hit (see Mark more Uint128, Uint64 and Decimal operations as const #1157).
• Consider using macros to ensure API and implementation consistency across types
• Signed integers
• Signed decimals?
• constify all the things
• Find a consistent way to convert Uint64/Uin128 to the primitives u64 and u128. Are we happy with the current API?
• Deprecate cosmwasm_std:{Decimal, Decimal256, Uint128, Uint256, Uint512, Uint64} and point the users to the math library

Feature	Uint64	Uint128	Uint256	Uint512	Decimal	Decimal256
const zero	✅	✅	✅	✅	✅	✅
const one	✅ (#1333)	✅ (#1333)	✅ (#1333)	✅ (#1333)	✅	✅
Sum	✅	✅	✅	✅	✅	✅
Add	✅	✅	✅	✅	✅	✅
AddAssign	✅	✅	✅	✅	✅ (#1253)	✅ (#1253)
Sub	✅ (#1242)	✅	✅	✅	✅	✅
SubAssign	✅ (#1242)	✅	✅	✅	✅ (#1253)	✅ (#1253)
Mul	✅ (#1245)	✅	✅	✅	✅	✅
MulAssign	✅ (#1245)	✅	✅	✅	✅ (#1255)	✅ (#1255)
Div	✅	✅	✅	✅	✅ (#1279)	✅ (#1279)
DivAssign	✅	✅	✅	✅	✅ (#1279)	✅ (#1279)
Rem	✅	✅	✅	✅	✅ (#1340 )	✅ (#1340)
RemAssign	✅ (#1250)	✅ (#1250)	✅ (#1250)	✅ (#1250)	✅ (#1340 )	✅ (#1340)
floor/ceil	N/A	N/A	N/A	N/A	✅ (#1347 )	✅ (#1347)
pow	✅ (#1251)	✅ (#1251)	✅	✅ (#1251)	✅ (#1342 )	✅ (#1342)
checked_add	✅	✅	✅	✅	✅ (#1341 )	✅ (#1341)
checked_sub	✅	✅	✅	✅	✅ (#1341 )	✅ (#1341)
checked_rem	✅	✅	✅	✅	✅ (#1341 )	✅ (#1341)
checked_mul	✅	✅	✅	✅	✅	✅
checked_div	✅	✅	✅	✅	✅ (#1341 )	✅ (#1341)
checked_div_euclid	✅	✅	(#1376)	(#1376)	N/A	N/A
checked_pow	✅	✅	✅	✅	✅	✅
saturating_add	✅	✅	✅	✅	✅ (#1343 )	✅ (#1343)
saturating_sub	✅	✅	✅	✅	✅ (#1343 )	✅ (#1343)
saturating_mul	✅	✅	✅	✅	✅ (#1343 )	✅ (#1343)
saturating_pow	✅	✅	✅ (#1343 )	✅ (#1343)	✅ (#1342 )	✅ (#1342)
wrapping_add	✅	✅	✅ (#1459)	✅ (#1459)
wrapping_sub	✅	✅	✅ (#1459)	✅ (#1459)
wrapping_mul	✅	✅	✅ (#1459)	✅ (#1459)
wrapping_pow	✅	✅	✅ (#1459)	✅ (#1459)
checked_from_ratio	N/A	N/A	N/A	N/A	✅ (#1281)	✅ (#1281)
checked_multiply_ratio	✅ (#1280)	✅ (#1280)	✅ (#1280)	N/A	N/A	N/A
A const constructor	✅	✅	✅	✅ (#1256)	✅ (#1264)	✅ (#1264)
const is_zero	✅ (#1256)	✅ (#1256)	✅ (#1256)	✅ (#1256)	✅ (#1256)	✅ (#1256)
const fn atomics	N/A	N/A	N/A	N/A	✅ (#1256)	✅ (#1256)
const fn decimal_places	N/A	N/A	N/A	N/A	✅ (#1256)	✅ (#1256)
pub const MAX: Self	✅	✅	✅	✅	✅	✅
pub const MIN: Self	✅ (#1366)	✅ (#1366)	✅ (#1366)	✅ (#1366)	✅ (#1366)	✅ (#1366)

[hashedone]

Consider using macros to ensure API and implementation consistency across types

I have another proposal. Use num-traits crate for that, so Uint64 and Uint128 can be just represented using generics (it would give Uint8 up to Uint128 for free, and their I equivalents).

Also consider having all operations defined for native equivalent [u/i]8..[u/i]128 so it would be easier to use literals on calculations with those numbers.

Also I have another idea which might be part of this. Something I called the "safemath". Basically create the type, let call it Safe<T>(T) which behaves as type T, except that all operations performed by it are checked. How would it work? Basically every operation on Safe<T> return the type of SafeProxy(Option<T>) propagating the none upwards (so behavior is exactly as you would use checked_ for everything). At the end the result could be converted back to Option<T> with function like fn native(self) -> Option<T>. Additionally both Safe<T> and SafeProxy would have operators defined with their underlying types (so SafeProxy<T> can be added to T. Also SafeProxy would have operations defined for U where T: num_traits::AsPrimitive<T> so it can be used transparently with literals. The cherry on the top would be extension trait on anything implementing the num_traits::num which would provide checked function wrapping the stuff with Safe type.

Why such thing? Consider:

let reg = a.checked_mul(b)?.checked_add(c.checked_mull(d)?)?;

Kind of difficult to read, and this is only the linear regression. Now with "safemath":

let reg = (c.checked() * b + c.checked() * d).native()?;

Shorter, but what is more important - more natural (basically some checked() and single native() calls even for bigger contexts. The need of this is very much dependent on the fact if we actually want to care about the edge cases overflow. Basically - do we actually need to use U128/256/512, or do we do it as over-protection so we never face the case when it is relevant. It may be also important, that all those checked operations are not for free - they generate gas cost obviously. But guarantee not to overflow. And they are 100% opt-in (you use it only if you need this behavior).

Another step further may be having similar wrappers defaulting to wrapping_ operations (which are basically native types not panicking in debug mode), and saturating_ ones - but I didn't see much usecases for them (I think I've seen literally single use of wrapping_ in contracts and no uses of saturating_).

[dakom]

Signed decimals?

Yes, definitely! I'd suggest something like:

#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Number {
    value: Decimal256, // or Uint256
    sign: NumberSign,
}

#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum NumberSign {
  Positive,
  Negative
}

I implemented something like this with all the ops, cmp implementations you'd expect, as well as helpers like .invert_sign()... very happy to ditch it in favor of a PR here when the building blocks are in place :)

[webmaster128]

I have another proposal. Use num-traits crate for that, so Uint64 and Uint128 can be just represented using generics (it would give Uint8 up to Uint128 for free, and their I equivalents).

I made bad experience with the num-traits crate in the past becasue it pulls in float operations.

Using generics, hmm, maybe. We only need Uint64 and Uint128. Then for Uint256 and Uint512 we need a separate implementation anyways. How would documentation look like? Everything should be rolled out completely for the specific types. The users should not see the extra abstraction.

[archseer]

I made bad experience with the num-traits crate in the past becasue it pulls in float operations.

I had some success with num-bigint if no_std was enabled, after using rust-num/num-bigint#225, but I'd prefer not using these crates if possible.

[hashedone]

I just had a talk with friend and it turns out decimals are not so trivial, and common rust crates actually makes it wrong (it has overhead of computational error). There is very strong and heavily documented C library doing it properly - https://github.com/dnotq/decNumber. Probably we can use it as example how to do it properly.

[dakom]

common rust crates actually makes it wrong

That's a pretty heavy statement... can you be more specific with an example of an arithmetic error?

[dakom]

@hashedone - if you want to discuss something, please quote and reply. Do not edit the history of other people's comments!!!

[dakom]

To address what you said in regard to #1186 (comment):

Why not to use u2 for proper arithmetic? The u1 (what you efectively proposed) is problematic on everything starting from addition, not mentioning more difficult operations...

I am not sure what you mean by u2 vs. u1... but assuming you mean to pack it in the available 256 bits instead of storing the sign flag separately:

1. Keeping the sign flag separate allows for deferring to the underlying value for fixed-point arithmetic, which keeps the complexity isolated in one place.
2. Keeping the sign flag separate allows TryFrom/TryInto in both directions and From/Into from the underlying value type (always positive).
3. I haven't run benchmarks, but it's almost definitely negligible in terms of performance cost (we're already going through jumps and extracting parts of byte arrays... a bool won't make much of an impact, and if we're really concerned about cache misses or something it could be packed with repr, but I wouldn't go down that road personally)
4. I've implemented all the ops and cmp things. It's very easy, you just pattern match on the sign and then treat positive and negative separately. Like I said, I'm happy to make a PR once this crate has the basic Decimal256/Uint256 stuff available

[webmaster128]

I just had a talk with friend and it turns out decimals are not so trivial, and common rust crates actually makes it wrong (it has overhead of computational error). There is very strong and heavily documented C library doing it properly - https://github.com/dnotq/decNumber. Probably we can use it as example how to do it properly.

This talks about "Decimal Floating Point", which is hard to get right for sure. But we don't need floating point types. We can simply use fixed point decimal types, which makes all the things very easy because those are just integers with a fixed decimal point. Most operations are implemented using integer arithmetic.

[dakom]

@webmaster128 - it would be really nice to also have npm wasm-packed lib that exported a typescript interface. Ideally a full class wrapper around the structs with all the methods, but for starters even pure functions could work.

That way frontend/cli code could work with the full Decimal256 and do math on the frontend, without having to worry about floating point errors or compatibility issues from some other third-party JS fixed-point lib

Of course even if this idea is accepted, it's low-priority and depends on the main crate existing first

[webmaster128]

That way frontend/cli code could work with the full Decimal256 and do math on the frontend, without having to worry about floating point errors or compatibility issues from some other third-party JS fixed-point lib

You can use import { Decimal } from "@cosmjs/math" which does that. You have to pass 18 as fractional digits to it to make it compatible. It can store decimals of arbitrary length, so it works with both cosmwasm_std::Decimal and cosmwasm_std::Decimal256.

[dakom]

I see.. and that is great, thanks!

That's even better, suggestion could just be a drop-in replacement to that package, maybe even without breaking anything much downstream, i.e. to use the underlying wasm and just pass strings around directly, instead of going through bn.js.

Would also make it easier to implement more methods like abs(), pow(), etc.

[webmaster128]

I think what you are bringing up there (compiling cosmwasm-math to a client lib) is a valid and valuable aproach. It will just not happy anytime soon due to reasource limits and priorities. Compiling Wasm for JS environments is not as simple as it sounds. Embedding them in a way that works for Node, bundlers and the web requires you to inline the Wasm as text in .js files. When it comes to bundling Wasm as part of JS libraries it gets very tricky. In many places we work towards getting rid of Wasm dependencies because they create big bundles and you cannot optimize for bundle size using tree shaking when you have Wasm.