Skip to content

Delete expired security log entries automatically

Warren Gifford requested to merge flying-robot/evict-old-security-logs into main

Created by: flying-robot

To preserve space, a background process should occasionally sweep through the security_event_logs table to hard delete expired entries[0].

The event log janitor deletes events older than 93 days (31 * 3), which ensures we have at least 3 full months worth of data available. The security event long janitor has a longer time horizon, currently set to 6 months.

I confirmed via spinup that the janitor removed old events and preserved the newer ones:

localhost sourcegraph@sourcegraph=# select * from security_event_logs;
┌────┬──────┬────────────────────┬─────────┬───────────────────┬────────┬──────────┬─────────┬───────────────────────────────┐
│ id │ name │        url         │ user_id │ anonymous_user_id │ source │ argument │ version │           timestamp           │
├────┼──────┼────────────────────┼─────────┼───────────────────┼────────┼──────────┼─────────┼───────────────────────────────┤
│ 10 │ EV1  │ http://example.com │       1 │ 1                 │ test   │ {}       │ 1       │ 2020-11-23 14:34:04.452748-05 │
│ 11 │ EV2  │ http://example.com │       1 │ 1                 │ test   │ {}       │ 1       │ 2021-06-09 14:34:06.868866-04 │
└────┴──────┴────────────────────┴─────────┴───────────────────┴────────┴──────────┴─────────┴───────────────────────────────┘
(2 rows)

Time: 0.331 ms

localhost sourcegraph@sourcegraph=# select * from security_event_logs;
┌────┬──────┬────────────────────┬─────────┬───────────────────┬────────┬──────────┬─────────┬───────────────────────────────┐
│ id │ name │        url         │ user_id │ anonymous_user_id │ source │ argument │ version │           timestamp           │
├────┼──────┼────────────────────┼─────────┼───────────────────┼────────┼──────────┼─────────┼───────────────────────────────┤
│ 11 │ EV2  │ http://example.com │       1 │ 1                 │ test   │ {}       │ 1       │ 2021-06-09 14:34:06.868866-04 │
└────┴──────┴────────────────────┴─────────┴───────────────────┴────────┴──────────┴─────────┴───────────────────────────────┘
(1 row)

Time: 0.239 ms

[0] https://sourcegraph.atlassian.net/browse/COREAPP-103

Merge request reports

Loading