Need ConnectionRestored Event subscribe in the CacheManager ,to identify connection is up after it went down.

[vijay8059]

Need ConnectionRestored event in the CacheManager that available in the StackExchange.Redis client .
so be used as a part of WithRedisConfiguration so to identify connection is up after it went down

    var cacheConfig = ConfigurationBuilder.BuildConfiguration(settings =>
        {
            settings
            .WithDictionaryHandle("inProcessCacheA1").WithExpiration(ExpirationMode.Sliding, TimeSpan.FromSeconds(30))
            .And
            .WithRedisConfiguration("redisCacheA1", config =>
            {
                config.WithAllowAdmin()
                    .WithDatabase(0).WithConnectionTimeout(8000)
                    .WithEndpoint(server, 6379);
            })
            .WithMaxRetries(1000)
            .WithRetryTimeout(100)
            .WithRedisBackplane("redisCacheA1", "redisCacheA1")
            .WithRedisCacheHandle("redisCacheA1", true); //.WithExpiration(ExpirationMode.Absolute, TimeSpan.FromSeconds(40))//.
            
        });

And also can remove the redis configuration on fly. Suppose if i get an exception i want to remove the redis configuration and only use in memory cache. After redis is up then i add the redis configuration back. I am seeing some issue
cacheConfig.CacheHandleConfigurations.Remove(redisConfiguration);
But i am see exception as ""At least one cache handle must be marked as the backplane source if a backplane is defined via configuration."" i got it as put in the configuration as

 .WithRedisBackplane("redisCacheA1", "redisCacheA1")
            .WithRedisCacheHandle("redisCacheA1", true);

i have put the backplane=false in the redisconfiguration but how to remove the back plane from cache config.

[TMiNus]

My initial ideas on this feature are discussed in #58. I have a fork with this feature implemented but it needs improvements. Basically you need a way of throwing an event detailing which handle failed, from that point forward you remove that handle completely from the handle collection, put it on a list of failed handles and return a totally new CacheManager instance with only the good handles. Although I am not 100% sure but I think with the new version you can pass handles around... @MichaCo will need to confirm #118

[MichaConrad]

@vijay8059 The configuration is not really meant to be modified after the cache manager has been instantiated. You can do that, but it doesn't really do anything. For example, if you remove a cache handle configuration after you created a BaseCacheManager with it, this does not remove the cache handle, and the configuration is still copied to the cache handle.
That's by design, and I still have to review how to make it harder to mess with it ;)
Like right now, cacheConfig.CacheHandleConfigurations is a list, but that's only because I need to add stuff to it internally. It was never intended to expose that and have you mess with it later.
That being said, certain configuration changes actually do have an impact, like expiration type/timeout of a cache handle. I'm pretty sure everything else is only read while initializing the manager.

That being said, to create a different configuration without Redis, just create a completely new configuration without Redis handle and/or backplane, and create a new BaseCacheManager instance with that.

As @TMiNus mentioned, there was the idea to disable, or temporarily remove certain cache handles, but that could cause all kinds of issues and I'm probably never going to implement that.

Also, after #118 was implemented, you can now pass in the cache vendor's client to the CacheManager configuration. That allows you to control the redis or memcached client. Pretty important if you want to use Redis for other things than just caching, too and only want one connection open ever!

So, back to the original issue, you want to disable the distributed cache whenever the connection goes down and then continue to cache stuff "normally".
I think the following might fix that:
Implement a class which manages two CacheManager instances, one distributed+inMemory and one inMemory only cache.
If the connection goes down, we just switch to the inMemory cache, if the connection comes up again, we switch back.
Full working example: Just throw that into a console app:

internal class Program
{
    private static void Main(string[] args)
    {
        var cacheKeeper = new CacheKeeper<int>();

        while (true)
        {
            var value = cacheKeeper.Cache.AddOrUpdate("key", 1, (v) => v + 1);
            Console.WriteLine("Current value is " + value);
            Thread.Sleep(500);
        }
    }

    public class CacheKeeper<T>
    {
        private readonly ICacheManager<T> _distributed;
        private readonly ICacheManager<T> _inMemory;
        private bool _distributedEnabled = true;

        public CacheKeeper()
        {
            var multiplexer = ConnectionMultiplexer.Connect("localhost");

            multiplexer.ConnectionFailed += (sender, args) =>
            {
                _distributedEnabled = false;

                Console.WriteLine("Connection failed, disabling redis...");
            };

            multiplexer.ConnectionRestored += (sender, args) =>
            {
                _distributedEnabled = true;

                Console.WriteLine("Connection restored, redis is back...");
            };

            _distributed = CacheFactory.Build<T>(
                s => s
                    .WithJsonSerializer()
                    .WithDictionaryHandle()
                        .WithExpiration(ExpirationMode.Absolute, TimeSpan.FromSeconds(5))
                    .And
                    .WithRedisConfiguration("redis", multiplexer)
                    .WithRedisCacheHandle("redis"));

            _inMemory = CacheFactory.Build<T>(
                s => s
                    .WithDictionaryHandle()
                        .WithExpiration(ExpirationMode.Sliding, TimeSpan.FromSeconds(5)));
        }

        public ICacheManager<T> Cache
        {
            get
            {
                if (_distributedEnabled)
                {
                    return _distributed;
                }

                return _inMemory;
            }
        }
    }
}

While running this, you should periodically kill the Redis server and short after, start it again.

You should see console messages like

Current value is 1
Current value is 2
Current value is 3
Connection failed, disabling redis...
Connection failed, disabling redis...
Current value is 1
Current value is 2
Current value is 3
Current value is 4
Current value is 5
Connection restored, redis is back...
Connection restored, redis is back...
Current value is 1
Current value is 2
Current value is 3
Current value is 4
Current value is 5

Yey, seems to work just fine.
But be aware of the fact that the data of course also changes. You cannot rely on the cached data if you actually work with it. In this case, I have a counter, and the counter resets every time I restart Redis, because I'm not persisting it in Redis. The In-Memory cache has a totally different counter which also eventually expires independent from the other cache.

Even if we would "share" the in-memory instance and keep the same counter while Redis is down, the moment Redis comes up again, the key is not in Redis and/or might have a different value and will eventually replace the one found in-memory.

So, in short, if you just cache some data which always has the same state, that might totally work fine for you. If not, don't do it. If you rely on Redis, your system is supposed to stop working properly if Redis goes down and you have to deal with that, e.g. show a warning or placeholder to the user or whatever...

wow that was a lot of writing...
Let me know if you have any more comments, questions or ideas ;)

I agree though, that I have to expose the Redis client from the Redis CacheHandle, so that you can attach those events even if you do not control the ConnectionMulitplexer. I'll do that soon

[TMiNus]

@MichaCo exposing those events from Redis CacheHandle will be great, right now we have to many layers in between. In your example you are using DictionaryHandle as local cache, I wonder if this will work with RuntimeCaching giving that each one of those has a unique signature...although wait it seems like you are not sharing the in-memory cache

[MichaConrad]

@TMiNus no I'm never sharing the in-memory cache in different CacheManager instances. The dictionary handle just creates a new dictionary for every CacheManager instance. And for SystemRuntime I prefix the keys with an instance token.
That being said, I might add that eventually in the future, as a configuration option.

[TMiNus]

@MichaCo that's what I meant for unique signature for every SystemRuntime. But you are right even if you share it, once the Redis handle is up again and someone put data in, an evict message will be issued making whatever data you have in local invalid

[MichaConrad]

@TMiNus yup, exactly. And not only that. Even if you just increment the counter (in the example) after Redis came back up, the change will be applied on the Redis instance first and then removes the in-memory version to keep your "view" consistent. You could try to sync in the other direction manually though. But that's pretty tricky, too, imagine you have n instances doing the same thing now...

[vijay8059]

Ya i Agree you different counters for local and distributed. When the connection of the redis is restored . I will invalidate/remove the cache from redis and in memory. So that it will get data from database. Now the data in redis will be in sync with database. So that always my data will be insync.