Skip to content

Curl /_cat/indices causes null pointer exception #27046

@mdmarek

Description

@mdmarek

Elasticsearch version

curl -s localhost:9200/
{
  "name" : "es-c2-m1",
  "cluster_name" : "es-c2",
  "cluster_uuid" : "zc5Ak5LoRRy_zfaC39JTFA",
  "version" : {
    "number" : "5.5.2",
    "build_hash" : "b2f0c09",
    "build_date" : "2017-08-14T12:33:14.154Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.0"
  },
  "tagline" : "You Know, for Search"
}

Plugins installed: []

      "plugins" : [
        {
          "name" : "repository-gcs",
          "version" : "5.5.2",
          "description" : "The GCS repository plugin adds Google Cloud Storage support for repositories.",
          "classname" : "org.elasticsearch.repositories.gcs.GoogleCloudStoragePlugin",
          "has_native_controller" : false
        }
      ],

JVM version (java -version):

java -version
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)

OS version

uname -a
Linux es-c2-m1 4.9.0-3-amd64 #1 SMP Debian 4.9.30-2+deb9u3 (2017-08-06) x86_64 GNU/Linux

Description of the problem including expected versus actual behavior:
Request indices via a curl, like so:

curl localhost:9200/_cat/indices

The expected result is a table of indices in the cluster, but instead a null pointer exception is returned:

curl localhost:9200/_cat/indices
{"error":{"root_cause":[{"type":"null_pointer_exception","reason":null}],"type":"null_pointer_exception","reason":null},"status":500}

Steps to reproduce:

  1. ssh into any node in the cluster, master or data node
  2. run command curl localhost:9200/_cat/indices

Provide logs (if relevant)
The following stack trace shows up in the logs right after the curl command is issued:

[2017-10-18T22:52:40,381][DEBUG][o.e.a.a.i.s.TransportIndicesStatsAction] [es-c2-m1] failed to execute [indices:monitor/stats] on node [6QkTt8qWSiKU1sZSTDa9dg]
org.elasticsearch.transport.RemoteTransportException: [es-c2-4][10.240.0.234:9300][indices:monitor/stats[n]]
Caused by: java.lang.IllegalStateException: Negative longs unsupported, use writeLong or writeZLong for negative numbers [-4134408331751]
	at org.elasticsearch.common.io.stream.StreamOutput.writeVLong(StreamOutput.java:219) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.index.search.stats.SearchStats$Stats.writeTo(SearchStats.java:211) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.index.search.stats.SearchStats.writeTo(SearchStats.java:353) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.common.io.stream.StreamOutput.writeOptionalStreamable(StreamOutput.java:723) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.action.admin.indices.stats.CommonStats.writeTo(CommonStats.java:255) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.action.admin.indices.stats.ShardStats.writeTo(ShardStats.java:102) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.common.io.stream.StreamOutput.writeOptionalStreamable(StreamOutput.java:723) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$NodeResponse.writeTo(TransportBroadcastByNodeAction.java:574) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.transport.TcpTransport.buildMessage(TcpTransport.java:1235) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.transport.TcpTransport.sendResponse(TcpTransport.java:1184) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.transport.TcpTransport.sendResponse(TcpTransport.java:1165) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.transport.TcpTransportChannel.sendResponse(TcpTransportChannel.java:67) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.transport.TcpTransportChannel.sendResponse(TcpTransportChannel.java:61) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.transport.DelegatingTransportChannel.sendResponse(DelegatingTransportChannel.java:60) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.transport.RequestHandlerRegistry$TransportChannelWrapper.sendResponse(RequestHandlerRegistry.java:111) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.messageReceived(TransportBroadcastByNodeAction.java:425) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.messageReceived(TransportBroadcastByNodeAction.java:399) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.transport.TcpTransport$RequestHandler.doRun(TcpTransport.java:1544) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.5.2.jar:5.5.2]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_144]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_144]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]
[2017-10-18T22:52:40,383][WARN ][r.suppressed             ] path: /_cat/indices, params: {format=json}
java.lang.NullPointerException: null
	at org.elasticsearch.rest.action.cat.RestIndicesAction.buildTable(RestIndicesAction.java:368) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.rest.action.cat.RestIndicesAction$1$1$1.buildResponse(RestIndicesAction.java:116) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.rest.action.cat.RestIndicesAction$1$1$1.buildResponse(RestIndicesAction.java:113) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.rest.action.RestResponseListener.processResponse(RestResponseListener.java:37) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.rest.action.RestActionListener.onResponse(RestActionListener.java:47) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:88) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:84) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$AsyncAction.onCompletion(TransportBroadcastByNodeAction.java:391) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$AsyncAction.onNodeFailure(TransportBroadcastByNodeAction.java:376) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$AsyncAction$1.handleException(TransportBroadcastByNodeAction.java:335) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1067) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.transport.TcpTransport.lambda$handleException$16(TcpTransport.java:1467) ~[elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.common.util.concurrent.EsExecutors$1.execute(EsExecutors.java:110) [elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.transport.TcpTransport.handleException(TcpTransport.java:1465) [elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.transport.TcpTransport.handlerResponseError(TcpTransport.java:1457) [elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:1401) [elasticsearch-5.5.2.jar:5.5.2]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:74) [transport-netty4-5.5.2.jar:5.5.2]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.11.Final.jar:4.1.11.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:297) [netty-codec-4.1.11.Final.jar:4.1.11.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:413) [netty-codec-4.1.11.Final.jar:4.1.11.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-codec-4.1.11.Final.jar:4.1.11.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.11.Final.jar:4.1.11.Final]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions