1624 index large files
Created by: ijsnow
This PR is part of the solution for #1624. We are adding a configuration option called "search.largeFiles" which is a list of glob patterns. Any files matching the patterns in the list will be indexed and searched regardless of the size.
This PR:
- adds the setting
- adds an internal endpoint that serves this setting for zoekt to consume
- uses the setting in searcher
Test plan:
Testing searcher implementation:
- Run sourcegraph
- Make sure basic text search is ran by disabling indexing
"search.index.enabled": false,
- Add a large file to a repo. (e.g. ijsnow/tmp)
- Make sure we don't search it
- Add
"search.largeFiles": ["bigfile.txt"]
to config - Make sure it is in search results