Skip to content

insights: handle errors and skipped events in streaming search

Administrator requested to merge insights/stream-search-errors into main

Created by: leonore

closes #33290 (closed).

the search streaming API gives us information on skipped events (see different reasons) that we are currently ignoring. this fixes this by collecting them and logging it.

working on this also revealed that we were handling errors incorrectly because we were appending them to a slice, and without a pointer this wouldn't actually get returned.

overall cleans up the decoder function with a new StreamingResult struct for clarity, the number of return values was getting too big.

naming and logging changes welcome!

Test plan

Created a backend insight with a query a count:1 to trigger a shard-match-limit skipped event, and observed this was logged correctly: image

Merge request reports