codeintel: Add file content limits to inference service

Created by: efritz

The introduction of the Lua inference service has ballooned worker memory because we're reading an unbounded amount of content from a tar archive of the target repo. This PR adds a configurable maximum number of files as well as a configurable maximum file size when trying to infer project structure. Additionally, we ensure that we don't call Archive with an empty list of paths (which, unfortunately, returns the entire content).

Test plan

Existing unit tests. Also tested locally to make sure default limits were reasonable.