Skip to content

codeintel: SQLite batch write

Administrator requested to merge sqlite-batch-write into master

Created by: efritz

Before: Write to SQLite within the writer were batched but synchronous, and each of the Write{Meta,Document,Defs,Refs} were run in goroutines to achieve concurrent writes.

After: Calls to writer methods are done synchronously, and each method within the writer controls its own concurrency.

Result: We are able to achieve higher throughput as we now serialize across goroutines. This shaves about another 15 seconds (from ~2m to ~1m45s) off of the precise code intel integration suite's upload step. Additionally, this allows us to do parallel batch writes within migrations, instead of just within the worker.

Merge request reports