StackExchange.Redis.RedisConnectionException: SocketClosed on <redis server>

[razvan007]

Hello,

I have seen some similar posts in regards of this but it does not seem to have the answer. We seems to struggle for a while now with the new version of the StackExchange.Redis.

We run a performance test against two azure redis caches (P4) in two regions (EUN, EUW) at a load of around 15k RPS for 48 hours.

After like 14 hours in the test suddenly we are getting some awkward errors on multiple of our machines which we assumed the package should be able to handle for us. From my understanding the socket connection gets dropped and the connection is not re-established.

StackExchange.Redis.RedisConnectionException: No connection is available to service this operation: MGET ; SocketClosed on :6380/Interactive, Idle/MarkProcessed, last: MGET, origin: ReadFromPipe, outstanding: 4, last-read: 0s ago, last-write: 0s ago, keep-alive: 60s, state: ConnectedEstablished, mgr: 9 of 10 available, in: 0, last-heartbeat: 0s ago, last-mbeat: 0s ago, global: 0s ago, v: 2.0.593.37019; IOCP: (Busy=2,Free=998,Min=512,Max=1000), WORKER: (Busy=2,Free=32765,Min=256,Max=32767), Local-CPU: n/a

Note: our nodes Network, CPU and memory allocation is looking good when the errors are happening, which makes us conclude that we are not running in a starvation mode.

Our connection strings configuration to redis servers is something like:
.redis.cache.windows.net:6380,password=abc=,ssl=True,abortConnect=False,syncTimeout=500
.redis.cache.windows.net:6380,password=abc=,ssl=True,abortConnect=False,syncTimeout=500

We are using MGet sync operations to retrieve data from the redis cache and for the connection multiplexer, we are using two of them for both regions and we are picking through a mechanism the one that responds the best in a window. The connection multiplexer for each of these redis cache instances are singleton.

Package details:
Pipelines.Sockets.Unofficial/2.0.22
System.Runtime.CompilerServices.Unsafe/4.5.2

Any help or suggestions are more than welcome.

Regards,
Raz

[StfnStrn]

Seeing similar issues, also with an Azure Redis Cache. Usually happens after swap slots on the WebApp after deployment. I suspected that step to cause the trouble.

[NickCraver]

This is the issue identified in #1120 and should be resolved by #1374 - collapsing this down into those discussions but a fix is on the way for the 2.1.x release.

[opendomain]

I upgraded my APP yesterday to the new release 2.1.0 and I just got an error

SocketClosed on myCacheName.redis.cache.windows.net:6380/Interactive, Idle/MarkProcessed, last: INFO, origin: ReadFromPipe, outstanding: 2, last-read: 0s ago, last-write: 0s ago, keep-alive: 60s, state: ConnectedEstablished, mgr: 8 of 10 available, in: 0, last-heartbeat: 0s ago, last-mbeat: 0s ago, global: 0s ago, v: 2.1.0.1

No changes to our Redis code
We were using 2.0.601 with no issues

.net 4.7.2
64 bit
Deployed to Azure 64 bit

[opendomain]

I am now getting another error

Timeout performing EXISTS (5000ms), next: EXISTS myKEY001, inst: 1, qu: 0, qs: 1, aw: False, rs: ReadAsync, ws: Idle, in: 0, serverEndpoint: myServer.redis.cache.windows.net:6380, mgr: 9 of 10 available, clientName: VF10-PC111FLE, IOCP: (Busy=1,Free=999,Min=250,Max=1000), WORKER: (Busy=26,Free=32741,Min=250,Max=32767), v: 2.1.0.1

We did get timeouts a year ago, but some suggestions from Microsoft and they all went away. Since upgrading to 2.1, they have come back. We will revert to the new library.

[NickCraver]

@opendomain Which version were you on previously?

[opendomain]

We were using 2.0.601, and since we switch back to this version, the issues above are gone