RFC 624: (M4) Design new recognizer interface

Created by: efritz

Implements a portion of RFC 624: Configurable auto-indexing recognizers.

Re-implement the existing recognizer behaviors into a greenfield API in Lua and get team alignment on the user ergonomics, behavior, and execution model. See this gist for an example API.

Update: Scope of ticket has changed to improve the existing API as merged in #33756. This must be informed by the results of #33047 and blocks #33046.