What Happened?
On Sunday November 22, 2021 our Desmos validator had a double-signing incident at approximately 04:45:10 UTC. We have worked quickly to debug the issue and manage to find the cause as well as put a fix in order to prevent a similar situation from happening again.
We found the incident was caused by a misconfiguration in our validator setup which caused 2 background services to run in parallel for the same node in the same server. This caused the validator node to start back up after being put to a short stop while we were in the process of moving the validator key to a different server and preparing for the upgrade on Monday.
Since this discovery, we have rechecked all our running nodes to make sure there is only one service running and setup additional alerts in order to prevent another incident such as this.
We sincerely apologize to all our delegators and supporters. Moving forward, we will continue to be transparent and work hard to regain your trust with goals of contributing to the Desmos community.
Cat Boss
