Skip to content

codeintel: Unify bundle reader/writer into a store interface

Administrator requested to merge bundle-store into main

Created by: efritz

What: This combines the bundle reader (which reads and caches data from a SQLite file) and the bundle writer (which pumps data into a new SQLite file) into a new interface and removes references to the old interface.

Why: The proof of concept work @gbrik and I are doing is going to benefit from having a unified interface. The new code both reads from an old bundle while modifying it, which is an action we didn't require before (hence separate interfaces). This is also going to aid the effort of https://github.com/sourcegraph/sourcegraph/issues/13070 when replacing the SQLite layer with Postgres (as it now looks/behaves like all the other Postgres-backed store).

How/How to review: I'd suggest reviewing commit by commit. This seems like a large change, but the individual commit diffs are pretty small if you ignore generated code. The steps of the commits are described below:

  • Make transactions explicit in bundle writer: When the bundle writer was opened, it would start a transaction and create tables implicitly. It's now up to the caller to do this. This is so we can introduce control over when tables are created (if we're modifying an existing bundle we don't need to). The createTables function is also exposed.

  • Unify the types of reader and writer's Close methods: Both readers and writers have a Close method with different signatures. Unify them so that the interfaces can coexist in a composite interface without name clashing.

  • Combine reader and writer into a store interface: This creates a new interface that's just a composite of Reader and Writer named Store. The sqliteWriter's methods are usurped by sqliteReader so that the reader now conforms to the combined interface. The ObservedReader interface is also expanded to conform to the new combined interface.

  • Update mocks: Add a line to gen.go.

  • Update reader/writer with store in internal usage: Rename ReaderCache -> StoreCache, Reader -> Store, sqliteReader -> sqliteStore. The reader cache files were renamed, but GitHub doesn't pick it up as a rename so it looks like there are changes. This commit doesn't do anything but renames.

  • Update reader/writer with store in service usage: Update the bundle manager and worker by replacing instances of persistence.Reader/Writer with persistence.Store.

  • Remove dead interfaces: Remove old Reader/Writer interfaces and regenerate mocks.

  • Ensure we start explicit transaction before writing a bundle: Add starting a transaction in the worker I missed in the earlier commits.

Merge request reports