authz/github: allocate less memory for groupsCache disabled path

Created by: bobheadxi

We assign to seenUsers and seenRepos while syncing users, this can double the memory usage of a sync for no use because the groupsCache disabled code path does not make use of the maps for deduplication (none is necessary).

Follow-up to the groupsCache permissions work. Prompted by some performance discussion on Slack. Tests already cover behaviour of both code paths, and they passed locally.