codeintel: create & implement rate limiting criteria for autoindexing dependencies

Created by: shrouxm

Now that we are opening up autoindexing to include dependencies, and dependency graphs are likely to be both large and highly autoindexable, we'll need a way to scale the amount of indexing happening up & down easily as our infra comes under new strains. I think this is a new thing we need to solve but I'm not actually sure? @efritz?