Extract content from a file

[dadoonet]

Today we are somehow building static content. I mean content provided within .asciidoc files only.

In Maven land, when you generate the website which contains documentation, you can use Snippets.

It's really handy because you can extract a content to your documentation and this content comes from the source code itself.
It basically means that when a developper changes whatever part of code, the documentation is automatically updated with the new content. If the developper does not update the code, it will most likely not compile anymore.

I'd encourage doing the same thing with our doc build process and try to use the same convention as in Maven land:

// START SNIPPET: snip-id
    public static void main( String[] args )
    {
        System.out.println( "Hello World!" );
    }
// END SNIPPET: snip-id

The snippet is extracted within two lines containing START SNIPPET and END SNIPPET. Optionally the snipper might have an id in case you have more than one snippet per file.

Most likely snippet start and end tags are provided within comments depending on the language you are using.

For example, in xml, it could be:

<!-- START SNIPPET: snip-id -->
            <dependency>
                <groupId>org.elasticsearch</groupId>
                <artifactId>elasticsearch</artifactId>
                <version>1.5.0</version>
            </dependency>
<!-- END SNIPPET: snip-id -->

To use it, you can add the following to your asciidoc file (example/java.asciidoc):

== This is my doc

[source,java]
--------------------------------------------------
%{snippet|id=snip-id|url=file:///path/to/Sample.java}
--------------------------------------------------

The idea of this change is to first parse all existing documentation files and copy them to another build_dir in case they changed (or at each run).

In the previous example, a file named build_dir/example/java.asciidoc will be created and will contain:

== This is my doc

[source,java]
--------------------------------------------------
    public static void main( String[] args )
    {
        System.out.println( "Hello World!" );
    }
--------------------------------------------------

Opening this issue now here to open discussion and see if the process I described is the right way for doing that. If so, I think this is something which should be implemented in perl within the build_docs.pl script.

@clintongormley WDYT?

[MaineC]

+1 from my side to keep documentation examples under build and test cycles.

Not sure if we should first take a step back maybe and review what we want to accomplish.

Here's my gist after updating the docs for the Java API query dsl examples:

What we have at the moment is the following:

https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/bool.html

IMHO on the one hand this is not enough as for most queries it doesn't cover all options, so no real reference documentation there. It duplicates documentation as well as what is written there to some part is written in JavaDoc already. On top it's never built so easily forgotten when ppl change the API.

Another approach I've seen in the wild:

https://github.com/randomizedtesting/randomizedtesting/tree/master/examples/maven/src/main/java/com/carrotsearch/examples/randomizedrunner

To me those are complimentary to the reference style docs JavaDoc typically provides: It's giving real world examples centred around some topics. On top it has the advantage of being under build and test cycles.

Lastly there's the Lucene JavaDoc only approach:

https://lucene.apache.org/core/4_1_0/core/index.html?overview-summary.html

Close to the code, examples still not under build and test cycles.

Me personally if I was a user I would love to see the second type of docs - could be as real code examples in some test package. If it's done using the style @dadoonet outlined above it would be better in line with the rest of our documentation - great as well. For such documentation I personally find it frustrating if there are errors in the docs, so to me it would be really important to keep them under build and test cycles.

In addition I would wish for detailed reference docs in our JavaDocs, potentially linked from the examples above.

[ppf2]

It basically means that when a developer changes whatever part of code, the documentation is automatically updated with the new content.

+1 and will help our doc from becoming stale over time :)

[dadoonet]

Hmmm. Just found this doc today: http://mrhaki.blogspot.fr/2014/04/awesome-asciidoc-include-partial-parts.html

May be this is something we can use easily? I'm going to give it a try.

[dadoonet]

Closed by elastic/elasticsearch#24354. Thanks to @nik9000 !