Redis Timeout

[vnc4567]

Hello, recently we have been experiencing a large number of timeouts.

A RedisTimeoutException has been thrown with the following message:

RedisTimeoutException has been thrown with message : "The message timed out in the backlog attempting to send because no connection became available, command=EXPIRE, timeout: 10000, outbound: 0KiB, inbound: 1KiB, next: HMGET J9XaFVDGBEaJ2XbPuMO5Aw==, inst: 0, qu: 0, qs: 0, aw: False, bw: Inactive, rs: ReadAsync, ws: Idle, in: 0, last-in: 0, cur-in: 0, sync-ops: 8042, async-ops: 21095, serverEndpoint: master.omni-cache.ghibp1.euw3.cache.amazonaws.com:6379, conn-sec: 7275.7, aoc: 0, mc: 1/1/0, mgr: 10 of 10 available, clientName: ip-172-17-37-82(SE.Redis-v2.7.27.49176), IOCP: (Busy=0,Free=1000,Min=1,Max=1000), WORKER: (Busy=25,Free=32742,Min=2,Max=32767), POOL: (Threads=25,QueuedItems=8,CompletedItems=2576585,Timers=82), v: 2.7.27.49176 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts)". From method ExecuteSyncImpl of StackExchange.Redis.ConnectionMultiplexer

This exception originates from the ExecuteSyncImpl method of StackExchange.Redis.ConnectionMultiplexer.

We tried to understand what could be causing these timeouts by following the guidance at https://stackexchange.github.io/StackExchange.Redis/Timeouts but without success.

Here is some information about our environment:

Infrastructure:

AWS ElastiCache Valkey Engine 8.2.0

Cluster mode disabled

Application client:

.NET 9

Microsoft.Extensions.Caching.StackExchangeRedis v9.0.1

StackExchange.Redis v2.7.27

In Program.cs

services.AddStackExchangeRedisCache(options =>
{
options.Configuration = "***:6379,ssl=true,abortConnect=false,connectTimeout=10000,syncTimeout=10000,connectRetry=3"
});

ConfigurationOptions redisOptions = ConfigurationOptions.Parse(redisConfiguration);

        ConnectionMultiplexer multiplexer = ConnectionMultiplexer.Connect(redisOptions);

        services.AddDataProtection()
                .SetApplicationName("Omni")
                .PersistKeysToStackExchangeRedis(multiplexer, "DataProtection-Keys");

Here are the metrics of our Valkey service:

Should I set minimum thread like this :

ThreadPool.SetMinThreads(200, 200);