Skip to content

can't update lsifstore testdata

Created by: slimsag

I hope I'm just losing my mind or have suddenly became a really bad programmer, because that would be better than the reality sinking in here. I'm 12 hours into trying to debug this and totally out of ideas. I can't for the life of me get the lsifstore SQL test file updated to include API docs search data in it for testing.

But what's really bothering me is I just cannot, at all, explain what I am seeing here. I'm going to revisit this on Monday, leaving notes here for myself to come back to.

This spawns from this slack convo

If I change the entire file to be just these two lines:

		SELECT pg_catalog.set_config('search_path', '', false);
		INSERT INTO public.lsif_data_definitions VALUES (1, 'gomod', 'github.com/sourcegraph/lsif-go/protocol:MonikerEdge.Edge', '\x1f8b08000000000000ffe2fd3f9f8991e9ff0206c6fff3181812fecf656664e4f1c94f4e2cc9cccf73492c49048933b2323287067932f23030720697241695f864e6a532b23030f28179ce19894589c925a945202176d7bc1498348f6b5e0ab2240383d7ff050c4c8c220545f925f9c9f939fa30865e7a3ee33f6613462e3029c4804b09cb06462630c9c500080000ffff4b16809dba000000', 2, 2);

Then I get:

=== RUN   TestDatabaseExists
connect to postgres://sourcegraph@localhost:5432/sourcegraph-test-8465867818003617086?sslmode=disable&timezone=UTC
    dbtest.go:101: testdb: postgres://sourcegraph@localhost:5432/sourcegraph-test-8465867818003617086?sslmode=disable&timezone=UTC
    helpers_test.go:38: unexpected error loading database data: ERROR: relation "lsif_data_definitions_schema_versions" does not exist (SQLSTATE 42P01)
    helpers_test.go:27: unexpected error finishing transaction: commit unexpectedly resulted in rollback
    dbtest.go:111: DATABASE sourcegraph-test-8465867818003617086 left intact for inspection

So it looks like that relation isn't being created for some reason. However, when I psql -d sourcegraph-test-8465867818003617086 and \d+ lsif_data_definitions_schema_versions it clearly shows that relation exists. So what gives?

Now, if I remove SELECT pg_catalog.set_config('search_path', '', false);, then it gets past that one error above, but then I get errors indicating that every single data point does not exist:

image

But again if I psql -d <the test DB> into the test DB, it seemingly shows the data is all there!?

So naturally I begin to think that "it must be connecting to the wrong database somehow", right? That's the only explanation I can think of.

So I run a single simple test, TestDatabaseExists, but this time I instrument the code with prints that show (1) what DB we're trying to connect to and (2) what DB we actually connect to.

=== RUN   TestDatabaseExists
dbtest.NewDB()
1) initTemplateDB postgres://sourcegraph@localhost:5432/sourcegraph?sslmode=disable&timezone=UTC
	initTemplateDB -> dbConn postgres://sourcegraph@localhost:5432/sourcegraph?sslmode=disable&timezone=UTC
	sql.Open registeredConnConfig0 postgres://sourcegraph@localhost:5432/sourcegraph?sslmode=disable&timezone=UTC

	initTemplateDB -> templateDB dbConn postgres://sourcegraph@localhost:5432/sourcegraph-test-template-1747993380162480695?sslmode=disable&timezone=UTC
	sql.Open registeredConnConfig1 postgres://sourcegraph@localhost:5432/sourcegraph-test-template-1747993380162480695?sslmode=disable&timezone=UTC

2) dbConn (copy template DB) postgres://sourcegraph@localhost:5432/sourcegraph?sslmode=disable&timezone=UTC
	sql.Open registeredConnConfig2 postgres://sourcegraph@localhost:5432/sourcegraph?sslmode=disable&timezone=UTC

3) testDB dbConn postgres://sourcegraph@localhost:5432/sourcegraph-test-10338157927812141277?sslmode=disable&timezone=UTC
	sql.Open registeredConnConfig3 postgres://sourcegraph@localhost:5432/sourcegraph-test-10338157927812141277?sslmode=disable&timezone=UTC
    dbtest.go:103: testdb: postgres://sourcegraph@localhost:5432/sourcegraph-test-10338157927812141277?sslmode=disable&timezone=UTC
    exists_test.go:27: unexpected exists result for cmd/lsif-go/main.go. want=true have=false
    exists_test.go:27: unexpected exists result for internal/index/indexer.go. want=true have=false
    dbtest.go:113: DATABASE sourcegraph-test-10338157927812141277 left intact for inspection
--- FAIL: TestDatabaseExists (5.46s)
FAIL
FAIL	github.com/sourcegraph/sourcegraph/enterprise/internal/codeintel/stores/lsifstore	5.849s
FAIL

And from what I can tell:

  1. Yes, the template DB should be initialized in just the generic sourcegraph database.
  2. Yes, copying of the template DB is also done via a sourcegraph database connection
  3. That only leaves 3) testDB dbConn in the logs above, which as you can see in the code is intended to connect to the test instance DB, and in the logs it clearly is!

So what am I left to conclude?

  • The data is there, according to psql.
  • The DB migrations did run, the schema exists.
  • The data in the testdata SQL file was inserted, according to psql.
  • We're connecting to the right DB, according to the tests' print lines I added
  • But the tests complain that relations don't exist, or that data is missing. How is that possible?