Documentation points folks to ESIntegTestCase

[nik9000]

This page points users to inheriting form ESIntegTestCase to start and stop the cluster. This approach has fallen out of favor with the Elasticsearch project and we don't write new code using it internally. Instead we tend to use http to test against a cluster started by gradle using Elasticsearch's standard startup scripts. This gives us a "more real" test.

The trouble with this disconnect is that we no longer publish the jars that you'd need to run a fully stood up cluster to maven central. The jars are available in central but inside the zip distribution. The ESIntegTestCase way of testing looks promising and convenient so users start down that path only to be thwarted by unavailable jars.

We need to fix this documentation so it expresses our current recommendations. And we'll need to keep it up to date as we come to a conclusion on #21119.

[timbaer]

Hi @nik9000 ,

first of all, I really appreciate your work on es5. The changes look promising! Because of that, I'm currently looking for what needs to be done to migrate my 2.4.x setup to elasticsearch 5.

One of my findings so far is that you no longer support running es in embedded mode. However, I have like hundreds of integration tests for my client functionality which are based on starting and stopping an embedded node for each test suite. After some investigation and playing around with changed Node Java Api and the es5 test framework, I found this issue and the one you linked and was somewhat shocked that neither the embedded node, nor the testing framework are the desired way by you guys to write integration tests.
What worked for me so far is using the maven-elasticsearch-plugin, however I'm missing the control of when to start / stop a cluster, e.g. if I want to run tests simultaneously.

So, I just want to ask if there is something I missed and simply moved into the wrong direction because of the lack of documentation. Or if using a maven/gradle plugin is the best I can do right now.

Best regards and thanks for the work!!!

[nik9000]

What worked for me so far is using the maven-elasticsearch-plugin, however I'm missing the control of when to start / stop a cluster, e.g. if I want to run tests simultaneously.

You can't really do that with the maven plugin. One at a time, sadly.

I think the maven plugin is the way to go because it is "more real" but it does sadly lose a few things like parallel test runs. Running ES was never really supported. It worked and folks would help with it but it was a maintenance nightmare and requires poking holes in the security which was bad.

ESIntegTestCase works and we use it when we need to be able to reach into the running cluster and mess with it but we try not write new tests that use that unless we need it because we're worried about how much fakery in injects. The loss of parallel tests is sad, but maybe worth it. At least for now we think it is. For us, at least.

[timbaer]

ESIntegTestCase works and we use it when we need to be able to reach into the running cluster and mess with it but we try not write new tests that use that unless we need it because we're worried about how much fakery in injects.

The problem with ESIntegTestCase for me is, that it brings in way too many dependencies which heavily interfere with mine, e.g. mockito vs securemock, log4j2 vs slf4j and many more...I tried the shading of those dependencies as described here, but then I'm running into even more worse problems.

Actually, I would love to use these test utilities. If you have some more hints about what I can change in order to use the lib, I would be really happy. I bet the community would love to see a testing-library with minimal side-effects (almost no dependencies).

Btw, there is a solution out there on how to work with embedded Node in es 5. Works for me so far, and I'm still arguing whether to use it, or the maven plugin. Maybe it would be good to show your point (the one of the core devs) to the outer world by commenting on threads like this, so that frustration about it keeps at a minimum. :-)

[nik9000]

The problem with ESIntegTestCase for me is, that it brings in way too many dependencies which heavily interfere with mine

Yeah, it really is for testing Elasticsearch plugins and Elasticsearch plugins have to suffer through stuff like securemock and live with log4j2 because we depend on it directly (though I don't feel that bad about it).

I bet the community would love to see a testing-library with minimal side-effects (almost no dependencies).

I do too but I think it is more I don't think there is any willingness from the Elasticsearch team to make something like you are describing. On the whole we're of the opinion that you should use an external Elasticsearch node, maybe started by maven or gradle or esvm or Vagrant or something. As I said, this loses parallel test runs which is bad but the "realness" you get from depending on a node started with real configs is worth the price. We think. We've moved many of our tests to this model and liked it for the most part. I think @alexcojocaru is working getting the maven plugin to support Elasticsearch 5.0: https://github.com/alexcojocaru/elasticsearch-maven-plugin/tree/es5

Btw, there is a solution out there on how to work with embedded Node in es 5.

I expect there are ways to get it to work but I wouldn't trust them not to break on upgrade. Like, minor or even patch version upgrade.

Maybe it would be good to show your point (the one of the core devs)

@dadoonet commented. I'll comment again.

[alexcojocaru]

Just a comment on the Elasticsearch maven plugin: I finished rewriting it to work with ES 5; the code is in the master branch and the artifact was deployed to maven central: https://search.maven.org/#artifactdetails|com.github.alexcojocaru|elasticsearch-maven-plugin|5.3|maven-plugin

[nik9000]

I finished rewriting it to work with ES 5

❤️

[jillesvangurp]

An alternative approach to using maven or gradle plugins is using docker. I implemented some integration tests last week using docker-java. I wrote a simple Junit rule called EsRule for this that pulls and launches a docker container with elasticsearch. Works great whether you are running tests from an IDE or a build and you can launch whatever you need with this approach. I made it so that it reuses any running elasticsearch if it is already up and running and added a simple jvm shut down hook to ensure the docker container gets killed. We run our CI builds with travis ci where all I had to do to make this work was add docker support to the services section.

[jeacott1]

@jillesvangurp care to share your EsRule?

[elasticmachine]

Pinging @elastic/es-core-infra

[jillesvangurp]

@jeacott1 here you go, not perfect but it works. I'd prefer to use something using docker compose long term so we can also run other dependencies but this was good enough last year.

Note this also includes some dependencies on our internal client that you probably want to swap out for the official one at this point. Also includes some logic for creating indicices that you may want to skip.

package com.matmatch.search.testutils;

import com.fasterxml.jackson.databind.ObjectMapper;
import com.github.dockerjava.api.DockerClient;
import com.github.dockerjava.api.command.CreateContainerResponse;
import com.github.dockerjava.api.command.StartContainerCmd;
import com.github.dockerjava.api.model.ExposedPort;
import com.github.dockerjava.api.model.Ports;
import com.github.dockerjava.core.DefaultDockerClientConfig;
import com.github.dockerjava.core.DockerClientBuilder;
import com.github.dockerjava.core.DockerClientConfig;
import com.github.dockerjava.core.command.PullImageResultCallback;
import com.matmatch.search.IndexMapping;
import com.matmatch.search.ObjectMapperService;
import com.matmatch.search.esclient.EsClientService;
import com.matmatch.search.esclient.EsRestClientService;
import com.matmatch.search.esclient.exceptions.EsNotFoundException;
import lombok.extern.slf4j.Slf4j;
import okhttp3.OkHttpClient;
import org.apache.commons.lang3.RandomStringUtils;
import org.apache.commons.lang3.StringUtils;
import org.junit.rules.ExternalResource;

import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
import java.util.concurrent.locks.ReentrantLock;

import static org.assertj.core.api.Assertions.assertThat;

/**
 * Simple @{@link org.junit.ClassRule} for ensuring elasticsearch is running.
 *
 * Note, if something is already running on the assigned port, it will skip starting the docker container
 * and use that instead. Useful if you want to run in an IDE.
 *
 * TODO logging, error handling, config, etc.
 */
@Slf4j
public class EsRule extends ExternalResource {
    // lets not bind to the standard port so we never 'accidentally' let tests write to an existing es cluster unintentionally
    public static final int ES_TEST_PORT = 9600;

    private final ReentrantLock lock = new ReentrantLock();
    private boolean running=false;
    private EsClientService esClientService;
    private EsRestClientService esRestClientService;
    private List<String> temporaryIndices = new ArrayList<>();


    private void startEs() {
        DockerClientConfig config = DefaultDockerClientConfig.createDefaultConfigBuilder()
                .build();
        DockerClient docker = DockerClientBuilder.getInstance(config).build();

        ExposedPort tcp9200 = ExposedPort.tcp(9200);

        Ports portBindings = new Ports();
        portBindings.bind(tcp9200, Ports.Binding.bindPort(ES_TEST_PORT));


        PullImageResultCallback callback = new PullImageResultCallback();
        docker.pullImageCmd("elasticsearch:5.1.2").exec(callback);
        callback.awaitSuccess();

        CreateContainerResponse containerResponse = docker
                .createContainerCmd("elasticsearch:5.1.2")
                .withPortBindings(portBindings)
                .exec();

        String containerId = containerResponse.getId();
        log.info("container id: {}", containerId);

        StartContainerCmd startContainerCmd = docker.startContainerCmd(containerId);
        startContainerCmd.exec();

        log.info("STARTING DOCKER CONTAINER FOR ES - note you can run your own ES on port " + ES_TEST_PORT + " in development to skip this and save some time");

        esClientService.awaitStarted();
        assertThat(esClientService.isUp()).isTrue();
        log.info("ES IS UP");
        running=true;

        // FIXME figure out if there's a more elegant way to do this
        Runtime.getRuntime().addShutdownHook(new Thread(){
            @Override
            public void run() {
                docker.stopContainerCmd(containerId).exec();
                log.info("STOPPED CONTAINER");
            }
        });
    }

    @Override
    protected void before() throws Throwable {
        super.before();
        ObjectMapperService objectMapperService = new ObjectMapperService(new ObjectMapper());
        esRestClientService = new EsRestClientService(objectMapperService, new OkHttpClient(), "http://localhost:"+ES_TEST_PORT);
        esClientService = new EsClientService(objectMapperService, esRestClientService);
        // check if we are running a usable es already; this allows you to run from the IDE without having to wait for es to start
        running = esClientService.isUp();
        if(!isRunning()) {
            lock.lock();
            try {
                if(!isRunning()) {
                    startEs();
                }
            } finally {
                lock.unlock();
            }
        }
    }

    public String createTemporaryIndex(IndexMapping mapping) {
        return createTemporaryIndexWithAlias(mapping,null);
    }

    public String createTemporaryIndexWithAlias(IndexMapping mapping, String alias) {
        String index = "testindex_" + RandomStringUtils.randomAlphanumeric(10).toLowerCase(Locale.ENGLISH);
        esClientService().createIndex(index, mapping);
        temporaryIndices.add(index);
        if(StringUtils.isNotBlank(alias)) {
            esClientService().addAlias(index,alias);
        }
        return index;
    }

    public void deleteIndicesAfterTest(String...indexes) {
        for(String index: indexes) {
            temporaryIndices.add(index);
        }
    }

    @Override
    protected void after() {
        super.after();
        if(temporaryIndices.size()>0) {
            temporaryIndices.forEach(indexName -> {
                try {
                    esClientService().deleteIndex(indexName);
                } catch (EsNotFoundException e) {
                    // ignore, the index we wanted delete did not exist
                }
            });
            esClientService().refresh();
        }
    }

    public boolean isRunning() {
        return running;
    }

    public EsRestClientService esRestClientService() {
        return esRestClientService;
    }

    public EsClientService esClientService() {
        return esClientService;
    }
}