Skip to content

Progress statements

Jump to bottom Edit New page
Jostein Austvik Jacobsen edited this page May 25, 2016 · 8 revisions

By stating the progress of a job at specific points in the code, DP2 clients can display a progressbar or a percentage to the end-user.

Due to the nature of XProc and XSLT, there is not always a clear execution order of the different parts of a script. So the way to state progress in Pipeline 2 is to state the amount of work that you expect will take place until the next progress statement arrives.

In addition, when stating the expected amount of work that will take place, another "level" of progress statements can be added, which will allow you to further refine the progress information and reuse them across scripts in a modular way.

For instance, say you have the script px:dtbook-to-epub3, then you can state the following:

  • loading the DTBook takes 10% of the total job running time
  • converting from DTBook to ZedAI takes 40% of the total time
  • converting from ZedAI to EPUB 3 takes 40% of the total time
  • storing the EPUB 3 takes 10% of the total time

Within the "DTBook to ZedAI" step however, you can add more statements like the following:

  • loading external resources takes 30% of the time spent in DTBook to ZedAI, (translating to 12% of the total job time since 30% of 40% is 12%)
  • The XSLT that converts from DTBook to ZedAI uses the remaining 70% of the time, (translating to 28% of the total job time since 70% of 40% is 28%)

And you may nest progress statements deeper inside the DTBook to ZedAI XSLT or wherever you wish.

This mechanism allows the client to know where in the process a job is, and estimate how much time is remaining.

Sending progress statements

Progress statements are sent to the client as part of the execution log. Messages in the execution log starting with the following pattern are progress statements:

[progress NAME PROGRESS SUBNAME]

XProc

From XProc, messages can be sent to the execution log by using the px:message step:

<px:message message="[progress NAME PROGRESS SUBNAME]"/>

If the following step contains pipes, for instance if the result of another step is piped into a port or an option of the step you are writing a progress message for, then you will want to use cx:depends-on on <px:message> to make sure that no other progress messages from the current XProc file are sent to the execution log before the step you are writing a progress message for finishes.

XSLT

From XSLT, messages can be sent to the execution log by using the xsl:message step:

<xsl:message select="'[progress NAME PROGRESS SUBNAME]'"/>

Java

There is currently no mechanism to send messages to the execution log from Java. See: https://github.com/daisy/pipeline-issues/issues/477

The progress syntax

[progress NAME PROGRESS SUBNAME]

PROGRESS

The PROGRESS is a fraction on the form NUMERATOR/DENOMINATOR. Both the numerator and denominator must be integers. The denominator part can be omitted, in which case the denominator will default to 100 so that the number in practice represents a percentage ([progress 5] = 5%, [progress 5/50] = 10%).

(It is also possible to use a ranged PROGRESS on the form FROM-TO, where FROM and TO are integers in the range [0,100]. This is a way of stating that the current progress is FROM % and the next step will take TO-FROM % of the current progress. However, this is rarely useful, since this requires you to know the current progress.)

NAME and SUBNAME

NAME is the name of the current progress level. SUBNAME is the name of the next progress level. The top-level progress level does not have a NAME.

SUBNAME is optional, but recommended. Progress statements whose NAME does not equal the SUBNAME of the first preceding progress statement with a NAME different than itself are ignored. (Unless the NAME is empty.)

Note that there cannot be two progress statements with the same NAME at different levels because there would be no way to know which of the two levels the progress statement belongs to. In case this does occur, it is always assumed that the progress statement belongs to the top-most level of the two. For recursive steps/templates/methods this means that you should not create a new level for each recursion.

Conventions
  • for public XProc steps, use the step type including the namespace prefix, such as px:dtbook-to-pef.
  • for public XSLTs, use the filename of the XSLT, such as dtbook-to-pef.xsl.
  • for public Java classes, use the package name + class name + method name, such as org.daisy.saxon.functions.file.FileExists.exists
  • for internal structures such as xsl:for-each, p:group, for () {} etc., append a descriptive string, such as px:dtbook-to-pef.check-arguments, dtbook-to-pef.xsl.iterate-paragraphs or org.daisy.saxon.functions.file.FileExists.exists.iterate-files.

Example

dtbook-to-pef.xpl

<p:declare-step type="px:dtbook-to-pef" ...>
(...)

<px:message message="[progress 40 px:dtbook.load] Converting from DTBook to PEF"/>
<px:dtbook.load>
  (...)
</px:dtbook.load>

<px:message message="[progress 40 px:dtbook-to-pef.convert] Converting from DTBook to PEF"/>
<px:dtbook-to-pef.convert>
  (...)
</px:dtbook-to-pef.convert>

<px:message message="[progress 20 px:pef.store] Converting from DTBook to PEF"/>
<px:pef.store>
  (...)
</px:pef.store>

</p:declare-step>

dtbook-to-pef.convert.xpl

<p:declare-step type="px:dtbook-to-pef.convert" ...>
(...)

<px:message message="Translating text to braille"/>
<px:message message="[progress px:dtbook-to-pef.convert 30 text-transform.xsl]"/>
<p:xslt>
    (...)
    <p:input port="stylesheet">
        <p:document href="../xslt/text-transform.xsl"/>
    </p:input>
</p:xslt>

<px:message message="Formatting DTBook as PEF"/>
<px:message message="[progress px:dtbook-to-pef.convert 60 px:dtbook-to-pef.convert.for-each-document]"/>
<p:for-each name="for-each-document">
    <px:message>
        <p:with-option name="message" select="concat('[progress px:dtbook-to-pef.convert.for-each-document 1/',p:iteration-size(),'] px:format-dtbook-as-pef')"/>
    </px:message>
    <px:format-dtbook-as-pef>
        (...)
    </px:format-dtbook-as-pef>
</p:for-each>

<px:message message="Finalizing"/>
<px:message message="[progress px:dtbook-to-pef.convert 10]"/>
<p:group>
    (...)
</p:group>

</p:declare-step>

text-transform.xsl

<xsl:stylesheet ...>

    (...)
    
    <xsl:variable name="total-paragraphs" select="count(//dtbook:p)"/>
    
    <!-- a single paragraph -->
    <xsl:template match="dtbook:p">
        <pef:row>
            <xsl:message select="concat('[progress text-transform.xsl 1/',$total-paragraphs,' org.daisy.saxon.functions.braille.Transform.transformText]')"/>
            <pf:transform-text>
                <xsl:with-param name="text" select="text()"/>
            </pf:transform-text>
        </pef:row>
    </xsl:template>
    
    <!-- many paragraphs at once -->
    <xsl:template match="dtbook:poem">
        <pef:page>
            <xsl:message select="concat('[progress text-transform.xsl ',count(.//dtbook:p),'/',$total-paragraphs,' org.daisy.saxon.functions.braille.Transform.transformPoem]')"/>
            <pf:transform-text>
                <xsl:with-param name="poem" select="."/>
            </pf:transform-text>
        </pef:page>
    </xsl:template>
    
</xsl:stylesheet>

Receiving progress statements

pipeline-clientlib-java filters out \[progress[^\]*\] * from the messages before displaying them, so progress statements are never rendered for the end-user. It also calculates the percentage and provides an API for estimating (interpolating) the current progress.

Add a custom footer
Add a custom sidebar
Clone this wiki locally