Compilation Regression with GHC 7.10.3

[ecaustin]

There is a massive increase in the amount of memory required to compile hermit-shell with GHC 7.10.3, to the point that it is crashing the VM I'm using.

The memory usage skyrockets when compiling the HERMIT.RemoteShell.Orphanage module which leads me to believe that the problem may be related to this GHC/binary bug:
https://ghc.haskell.org/trac/ghc/ticket/9630

Does anyone have time to confirm this and possibly replace some of the generic instances with manually written ones as a short term fix?

I'd volunteer to do it myself, but my hands are tied for the time being.

[ecaustin]

I should note that this problem exists for all versions of GHC 7.10.x based on my limited testing, but is particularly bad for GHC 7.10.3 for whatever reason.

I can at least get compilation to finish with GHC 7.10.2, so I've rolled back to that version for now.

[andygill]

@roboguy13 David - is this something you could look at?

[RyanGlScott]

Urgh - I'm 99% sure this is an aeson issue. For whatever reason, default implementations GHC generics-related functions don't inline very well, and if you force them to be inlined via INLINE pragmas, they can result in insane memory usage. pandoc-types has also experienced a similar issue, and their workaround was to roll back from aeson-0.10 to aeson-0.9.

I don't know the root cause of the issue, but I speculate that if they removed all of the INLINE pragmas from Data.Aeson.Types.Generics' generic class methods, they might not have such a bad time. Unfortunately, @bos is pretty hard to get ahold of, so I don't know how soon this issue will be addressed...

[xich]

The aeson dep in this cabal file already limits to <0.10. Maybe try forcing back to 0.8? Could also just try splitting up that module into one-per-datatype and see if that helps.

Workarounds, of course, but might be easier than writing instances by hand.

[ecaustin]

I can confirm that I am using aeson-0.10 at this point. More specifically, it's the master branch from github because I was tired of running into this warning:
haskell/aeson#290

Shame on me for changing the cabal file, but it seemed like the easier solution than manually inlining the default implementation everywhere.

That being said, I'm pretty sure I was having the same problems with aeson-0.9.0.1, but I'd be happy to retest using that and earlier versions.

[ecaustin]

Maximum memory residency during compilation with GHC-7.10.2 and aeson-0.8.1.1was ~1.4GB on my machine which which is only marginally less than with aeson-0.10.

Once Drew wraps up porting HERMIT to GHC 8, I'll repeat the tests with it.

[RyanGlScott]

My hope is that memory usage involving GHC generics decreases with GHC 8.0, since it was redesigned to encode the generic representation with -XDataKinds rather than generating a jillion empty datatypes an proxy instances.

[roboguy13]

@andygill I should be able to look into it this weekend. I don't know too much about the details of this aspect of GHC compilation, but I'll see what I can do.

[andygill]

We just need a workaround.

[ecaustin]

I've been more rigorously testing this all day and can confirm it's an issue that is amplified by aeson-0.10.

The maximum memory residency for any combination of GHC-7.10.x and aeson-0.8 or 0.9 is under 2GB which is obviously not ideal, but it's good enough for me to continue with my work.

I'll apologize for screaming fire here since, as @xich pointed out, the hermit-shell.cabal file already disallows aeson-0.10.

[RyanGlScott]

The maximum memory residency for any combination of GHC-7.10.x and aeson-0.8 or 0.9 is under 2GB which is obviously not ideal, but it's good enough for me to continue with my work.

That's somewhat encouraging to hear, since aeson-0.8 and -0.9 both have substantially simpler Data.Aeson.Types.Generic modules than -0.10. Perhaps that's worth investigating some more.

@roboguy13, can you try this:

1. Get aeson-0.10 as a control, but also prepare a version that removes all INLINE pragmas from Data.Aeson.Types.Generic
2. Install each and compare the memory usage when compiling pandoc-types.

Someone reported using 7GB of memory when compiling pandoc-types with aeson-0.10, so if removing the INLINE pragmas makes it better, it's probably worth opening a pull request for.

[roboguy13]

@RyanGlScott Do you happen to know if the GHC developers are aware of these things yet? This might make a good test case for them (although it might be an unavoidable issue where the solution is just fewer INLINEs).

[ecaustin]

I believe what we're running into is an instance of this bug:
https://ghc.haskell.org/trac/ghc/ticket/9630

[RyanGlScott]

@roboguy13, I believe that's precisely the issue. Aside from the aforementioned inlining issues, a comment on Trac #9630 has another nifty hack that might help speed things up: make each generic class have exactly one method. Apparently, if you have multiple class methods like this (taken from Data.Aeson.Types.Class):

class GToJSON f where
    gToJSON :: Options -> f a -> Value
    gToEncoding :: Options -> f a -> Encoding

it hurts the optimizer a lot. Try taking all of the generic classes in Data.Aeson.Types.Class and Data.Aeson.Types.Generic and splitting them up, e.g.,

class GToJSON f where
    gToJSON :: Options -> f a -> Value

class GToEncoding g where
    gToEncoding :: Options -> f a -> Encoding

To see if that improves things.