May I ask if you are using the eWeLink app with the device too? If you experience frequent unresponsiveness with Matter, would you be interested in using a test firmware that could fix the issue?
If not, an official patch is coming soon to solve the Matter SDK bug.