-
Notifications
You must be signed in to change notification settings - Fork 25.6k
Closed
Description
es-version: 2.3.3:
plugins: [hq,ik,kopf,licence,marvel-agent,search-guard-2,search-guard-ssl ]
JVM version:
1.7.0_79
OS version:
sata disk
Description of the problem including expected versus actual behavior:
Hi, I have a index(shard=60,replica=1,node=48,jvm:30G), Now total 7 billion docs,everyday 200 million doc index(with uid ), The job is very slow and I print the stack of es, the most bulk thread are loading version. Then os load is very high(because my sata disk).
my data update one day one time(one doc update by one thread), like the sense.whether I can disable the version load or any suggestion ?
"elasticsearch[N'Garai][bulk][T#28]" #449 daemon prio=5 os_prio=0 tid=0x00007f4b35317800 nid=0xbdc runnable [0x0000000054666000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.FileDispatcherImpl.pread0(Native Method)
at sun.nio.ch.FileDispatcherImpl.pread(FileDispatcherImpl.java:52)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:220)
at sun.nio.ch.IOUtil.read(IOUtil.java:197)
at sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:762)
at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:748)
at org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:179)
at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:342)
at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:54)
at org.apache.lucene.store.DataInput.readVInt(DataInput.java:125)
at org.apache.lucene.store.BufferedIndexInput.readVInt(BufferedIndexInput.java:221)
at org.apache.lucene.codecs.blocktree.SegmentTermsEnumFrame.loadBlock(SegmentTermsEnumFrame.java:157)
at org.apache.lucene.codecs.blocktree.SegmentTermsEnum.seekExact(SegmentTermsEnum.java:507)
at org.elasticsearch.common.lucene.uid.PerThreadIDAndVersionLookup.lookup(PerThreadIDAndVersionLookup.java:88)
at org.elasticsearch.common.lucene.uid.Versions.loadDocIdAndVersion(Versions.java:124)
at org.elasticsearch.common.lucene.uid.Versions.loadVersion(Versions.java:140)
at org.elasticsearch.index.engine.InternalEngine.loadCurrentVersionFromIndex(InternalEngine.java:1050)
at org.elasticsearch.index.engine.InternalEngine.innerIndex(InternalEngine.java:504)
- locked <0x000000024df935a8> (a java.lang.Object)
at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:460)
at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:601)
at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836)
at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnReplica(TransportIndexAction.java:196)
at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnReplica(TransportShardBulkAction.java:436)
at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnReplica(TransportShardBulkAction.java:68)
at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncReplicaAction.doRun(TransportReplicationAction.java:392)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at org.elasticsearch.action.support.replication.TransportReplicationAction$ReplicaOperationTransportHandler.messageReceived(Tra
Metadata
Metadata
Assignees
Labels
No labels