Are INLINE pragmas worth the compilation time?

[phadej]

Without (after removal of pragmas)

INLINE pragmas in FromJSON.hs and ToJSON.hs

Second line ais nother clean build with aeson, criterion, statistics and aeson-benchmarks:
It seems that INLINE pragmas slowdown the downstream package compilation as well?

stack build aeson  69.23s user 2.65s system 99% cpu 1:12.35 total
stack build       313.90s user 12.86s system 100% cpu 5:26.45 total

benchmarking generic/direct/twitter100
time                 405.4 μs   (402.9 μs .. 407.9 μs)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 405.0 μs   (402.6 μs .. 407.8 μs)
std dev              8.039 μs   (6.568 μs .. 10.82 μs)
variance introduced by outliers: 12% (moderately inflated)

benchmarking manual/direct/twitter100
time                 387.0 μs   (383.2 μs .. 391.5 μs)
                     0.999 R²   (0.999 R² .. 1.000 R²)
mean                 385.2 μs   (383.0 μs .. 387.6 μs)
std dev              8.185 μs   (7.162 μs .. 9.457 μs)
variance introduced by outliers: 14% (moderately inflated)

benchmarking th/direct/twitter100
time                 406.7 μs   (402.7 μs .. 409.9 μs)
                     0.999 R²   (0.999 R² .. 1.000 R²)
mean                 404.4 μs   (401.7 μs .. 407.7 μs)
std dev              9.895 μs   (8.084 μs .. 12.91 μs)
variance introduced by outliers: 16% (moderately inflated)

With:

stack build aeson  179.39s user 4.51s system 97% cpu 3:09.30 total
stack build        567.26s user 16.09s system 100% cpu 9:41.51 total

benchmarking generic/direct/twitter100
time                 399.7 μs   (395.2 μs .. 405.4 μs)
                     0.999 R²   (0.999 R² .. 1.000 R²)
mean                 400.3 μs   (397.5 μs .. 403.5 μs)
std dev              9.911 μs   (7.580 μs .. 12.47 μs)
variance introduced by outliers: 17% (moderately inflated)

benchmarking manual/direct/twitter100
time                 385.4 μs   (383.1 μs .. 387.9 μs)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 386.9 μs   (384.4 μs .. 390.0 μs)
std dev              9.782 μs   (7.519 μs .. 15.14 μs)
variance introduced by outliers: 18% (moderately inflated)

benchmarking th/direct/twitter100
time                 404.5 μs   (400.5 μs .. 407.8 μs)
                     0.999 R²   (0.999 R² .. 1.000 R²)
mean                 402.1 μs   (399.1 μs .. 406.3 μs)
std dev              12.35 μs   (9.032 μs .. 18.06 μs)
variance introduced by outliers: 24% (moderately inflated)

[bergmark]

Usually packages don't specify -02 themselves, but aeson does. Is removing this a solution?

[RyanGlScott]

FWIW, I removed all INLINE pragmas in Data.Aeson.Types.Generic as part of #335. Not only did it significantly help compilation times when Generic ToJSON/FromJSON instances are being derived, but it also didn't appear to appreciably change performance.

That's not to say we should necessarily nuke every INLINE pragma from aeson, but perhaps we should only annotate the smaller functions with it.

[bergmark]

What do you think about my suggestion to remove -O2 from being a default?

[lspitzner]

Bump, I currently see 10min+ compile time (and 600MB ram usage) for a single module on github/travis-CI (which is then terminated), in large part due to aeson instance generation (and the module itself is only 350 loc, but there are several instances).

[hvr]

@bergmark given that with new-build it's simple to request an -O2 version of aeson via

-- cabal.project

package aeson
   optimization: 2

nowadays it makes more sense to remove the -O2 from ghc-options (especially since ghc-options: -O2 overrides any optimization requested by the user currently)

@phadej do we have any performance comparison of -O1 vs -O2 for aeson?

[phadej]

@hvr I run aeson-benchmark-typed encodeGeneric, in 6306cec commit (as after that master is faster than aeson-1.2.1.0), wth and without -O2 in aeson-benchmark.cabal.

My machine is quite noisy atm, so I cannot really tell a difference. It tries to tell me that without -O2 is faster.

[phadej]

With -O2

benchmarks and baseline are the same, as it should:

benchmarking encodeGeneric/direct/twitter100
time                 263.5 μs   (261.7 μs .. 265.7 μs)
                     0.999 R²   (0.999 R² .. 1.000 R²)
mean                 264.2 μs   (262.2 μs .. 267.9 μs)
std dev              8.789 μs   (5.205 μs .. 14.35 μs)
variance introduced by outliers: 29% (moderately inflated)

benchmarking encodeGeneric/direct/jp100
time                 247.7 μs   (245.8 μs .. 250.0 μs)
                     0.999 R²   (0.999 R² .. 1.000 R²)
mean                 248.7 μs   (247.0 μs .. 251.7 μs)
std dev              7.393 μs   (5.082 μs .. 10.37 μs)
variance introduced by outliers: 24% (moderately inflated)

benchmarking encodeGeneric/direct/twitter100 baseline
time                 269.4 μs   (267.0 μs .. 272.0 μs)
                     0.999 R²   (0.998 R² .. 1.000 R²)
mean                 270.3 μs   (267.9 μs .. 276.2 μs)
std dev              11.53 μs   (6.310 μs .. 20.79 μs)
variance introduced by outliers: 40% (moderately inflated)

benchmarking encodeGeneric/direct/jp100 baseline
time                 253.0 μs   (250.9 μs .. 255.4 μs)
                     0.999 R²   (0.999 R² .. 1.000 R²)
mean                 253.3 μs   (251.6 μs .. 257.5 μs)
std dev              8.151 μs   (4.271 μs .. 14.50 μs)
variance introduced by outliers: 27% (moderately inflated)

benchmarking encodeGeneric/viaValue/twitter100
time                 1.070 ms   (1.068 ms .. 1.073 ms)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 1.076 ms   (1.071 ms .. 1.108 ms)
std dev              30.13 μs   (5.403 μs .. 76.37 μs)
variance introduced by outliers: 17% (moderately inflated)

benchmarking encodeGeneric/viaValue/jp100
time                 1.072 ms   (1.061 ms .. 1.085 ms)
                     0.999 R²   (0.998 R² .. 0.999 R²)
mean                 1.075 ms   (1.066 ms .. 1.096 ms)
std dev              40.78 μs   (22.28 μs .. 77.21 μs)
variance introduced by outliers: 26% (moderately inflated)

benchmarking encodeGeneric/viaValue/twitter100 baseline
time                 1.085 ms   (1.075 ms .. 1.097 ms)
                     0.999 R²   (0.998 R² .. 0.999 R²)
mean                 1.090 ms   (1.080 ms .. 1.124 ms)
std dev              52.25 μs   (24.41 μs .. 111.7 μs)
variance introduced by outliers: 38% (moderately inflated)

benchmarking encodeGeneric/viaValue/jp100 baseline
time                 1.039 ms   (1.032 ms .. 1.044 ms)
                     0.999 R²   (0.998 R² .. 1.000 R²)
mean                 1.051 ms   (1.045 ms .. 1.078 ms)
std dev              35.57 μs   (3.989 μs .. 81.14 μs)
variance introduced by outliers: 23% (moderately inflated)

Without -O2

encoding via Value is considerably faster, otherwise no chang

last-bar has 98 variance introduced by outliers, as it shows.

benchmarking encodeGeneric/direct/twitter100
time                 255.6 μs   (253.4 μs .. 257.9 μs)
                     0.999 R²   (0.998 R² .. 1.000 R²)
mean                 256.9 μs   (254.7 μs .. 262.1 μs)
std dev              11.23 μs   (4.729 μs .. 22.85 μs)
variance introduced by outliers: 41% (moderately inflated)

benchmarking encodeGeneric/direct/jp100
time                 241.8 μs   (239.8 μs .. 244.0 μs)
                     0.999 R²   (0.998 R² .. 0.999 R²)
mean                 242.4 μs   (241.1 μs .. 245.3 μs)
std dev              6.408 μs   (3.889 μs .. 11.20 μs)
variance introduced by outliers: 20% (moderately inflated)

benchmarking encodeGeneric/direct/twitter100 baseline
time                 270.7 μs   (268.4 μs .. 273.2 μs)
                     0.999 R²   (0.999 R² .. 0.999 R²)
mean                 271.9 μs   (269.9 μs .. 276.2 μs)
std dev              9.617 μs   (5.058 μs .. 16.30 μs)
variance introduced by outliers: 31% (moderately inflated)

benchmarking encodeGeneric/direct/jp100 baseline
time                 255.2 μs   (253.3 μs .. 257.5 μs)
                     0.999 R²   (0.998 R² .. 1.000 R²)
mean                 256.5 μs   (254.8 μs .. 260.5 μs)
std dev              8.144 μs   (4.554 μs .. 14.73 μs)
variance introduced by outliers: 26% (moderately inflated)

benchmarking encodeGeneric/viaValue/twitter100
time                 792.4 μs   (788.9 μs .. 795.6 μs)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 798.4 μs   (797.4 μs .. 799.5 μs)
std dev              3.591 μs   (2.875 μs .. 4.781 μs)

benchmarking encodeGeneric/viaValue/jp100
time                 782.6 μs   (773.7 μs .. 794.0 μs)
                     0.998 R²   (0.998 R² .. 0.999 R²)
mean                 789.4 μs   (782.9 μs .. 799.7 μs)
std dev              27.08 μs   (18.09 μs .. 39.78 μs)
variance introduced by outliers: 24% (moderately inflated)

benchmarking encodeGeneric/viaValue/twitter100 baseline
time                 1.075 ms   (1.061 ms .. 1.089 ms)
                     0.998 R²   (0.997 R² .. 0.999 R²)
mean                 1.086 ms   (1.075 ms .. 1.109 ms)
std dev              51.24 μs   (28.94 μs .. 86.69 μs)
variance introduced by outliers: 36% (moderately inflated)

benchmarking encodeGeneric/viaValue/jp100 baseline
time                 2.400 ms   (1.894 ms .. 2.709 ms)
                     0.878 R²   (0.837 R² .. 0.915 R²)
mean                 1.483 ms   (1.315 ms .. 1.719 ms)
std dev              602.4 μs   (472.6 μs .. 773.5 μs)
variance introduced by outliers: 98% (severely inflated)

[lspitzner]

I'd think that -O2 has little/nothing to do with INLINE and that the latter affects rev-dev compilation, while the former only affects compilation of the aeson package.

[bergmark]

Let's remove it then.

[lspitzner]

I have done some more testing around my issue, and I must admit that I am uncertain if any INLINE pragmas are at fault. Neither using -O1 nor removing all INLINEs has much influence on what I see. Just compiling a separate module that contains nothing but (orphan) aeson instance definitions for my types takes up ~480MB "total memory in use" (according to +RTS -s)/~580MB (according to time).

So, sorry if this belongs in a separate issue. Before I do that: is such memory usage to be expected? Or should I bother the ghc people instead?

[lspitzner]

Not to say that I wouldn't welcome lower compilation times for aeson itself. 👍