How to implement a shared state between multiple risotto instances

[matthieugouel]

right now the state is written in a file, because it's very fast to read/write it and hold it in memory.
I've made non conclusive experiments with Redis, and I'm not sure a SQLite approach would work well (to test maybe).

With a file, it's difficult to share the state between different instances, for redanduncy purposes. It's not impossible thought, and maybe it's still a good approach to keep everything in memory and sync with a file.

See #7 which is another limitation of the file approach.

[matthieugouel]

One idea could be to use an object storage (S3/R2/MinIO like), but that's not enough to share the state between multiple Risotto instances, it would need to sync the file/object each time we receive a BMP message.

[matthieugouel]

To re-scope, I don't necessarily want to offer a solution in the app, which imo should stay simple and generic. But I would be interested in having a satisfactory answer for this, so this issue is more about testing different solutions.

[mfreeman451]

I'm interested in this as well and am considering implementing it using NATS or Iggy

[matthieugouel]

The simplest solution I thought of so far to handle high availability (having multiple risotto instances, without messing with state) is:

• spin up multiple risotto instances, each one managing it's state
• configure routers to send BMP messages to those instances, or have something HA that mirror the messages to risotto instances
• implement a leader election, for instance with Redis, Valkey etc. to ensure only one risotto instance is sending messages downstream. To share the load it can be one lock per router.