Cocoa WebView rendering / interactivity issues (video/audio), setWantsLayer vs. CSS 3D trick

[danielweck]

Fixed layout issue only.

1. Problem: without Cocoa-NSView-setWantsLayer and CSS-3D (-webkit-transform: translateZ(0)), OSX-WebView renders the iframe contents partially (top-left corner only, rectangular surface area equivalent to the percentage of the transform-scale applied to the "scaler" div (iframe parent element)). The HTML inspector reveals that the content actually exists (text can be selected with the mouse, etc.), but somehow it does not get composited by the WebView / Cocoa rendering pipeline.

2. With setWantsLayer applied to the NSView control (either the WebView itself, or its superview), there are some improvements but not across the board (i.e. not with every FXL titles) For example, highly-graphical content that makes use of CSS animations (sprites, etc.) fails to render correctly. Sometimes, pages are blank (white empty background), yet the HTML/CSS inspector shows that the tree is rendered correctly in the backstore, just not flushed correctly in the display layer.

3. With CSS-3D applied to the iframe or its parent element, there are no display issues (i.e. no "invisible" or cropped content), but unfortunately there are nasty side effects, such as broken HTML5 audio/video interactivity, lack of text selection, jagged font rendering.

[danielweck]

See: 71fb04c

[danielweck]

(note: above description updated)
Possible alternative solution: complete re-think of the FXL view architecture (which is implemented in readium-shared-js), migrate away from HTML iframe towards native WebViews (one for each EPUB HTML/SVG content document). This major application-specific refactoring would be costly to design, implement and test. Readium's document loading infrastructure and feature-injection mechanism depend on the known behaviours of HTML iframe (security, cross-origin communication), so this would unfortunately not be just a straight swap from iframe to WebView (this would include replacing the reader.html top-level wrapper by a Cocoa NSView hierarchy). This would require a lot of native ObjectiveC code to replicate the layout routines that fixed_view.js and one_page_view.js currently implement, and an entirely new bridge API between the reader.js, annotations.js, media_overlays.js (etc.) who would need to access the DOM hosted in the native WebViews.

[danielweck]

Ah, I figured-out a pure-CSS trick to work around the WebView rendering / clipping bug. Basically, the scaling mechanism for fixed-layout documents has to apply to the HTML document itself, so that its full composited surface area is picked-up by Cocoa / NSView.

Because this is a CSS transform, it does not affect the computed CSS dimensions of wrapper div and iframe, so it works just as in the normal case (no side effects / regression bugs in coordinate /size computations, and the visual result is the same). Caveat: authored HTML documents may originally contain a CSS transform on the root html element, in which case it gets overridden by Readium's own (but, because this edge-case is unlikely to occur, I think it is a perfectly acceptable trade-off).

See: readium/readium-shared-js@ca4e1d0 and e909269#diff-2

[danielweck]

Although the fix referred-to above works fine for general rendering (including rich CSS animations), there may still be occasional issues with audio/video. See #22

[rkwright]

I just tested with my SmokeTest files and it looks good. Will also run through the relevant EPUB Test Suite files next.

[danielweck]

Thanks @rkwright If you have observations about video/audio and non-selectable text, feel free to populate the discussion in this issue: #22 (I usually manage to "restore" normal behaviour by resizing the viewport...but may you'll have a different experience)