e2e-tests: Reset user settings at end of test

Created by: mrnugget

This is a follow-up to #5210.

It makes sure that the notice banner is removed after the test. Otherwise it persists through multiple test runs on the same instance.