Use WAL with SQLite cache, fix close

[hauntsaninja]

This is the more modern way to manage concurrency with SQLite. Relevant to current discussion, it means concurrent mypy runs using the cache will wait for each other, rather than fail

SQLite also claims this is significantly faster, but I haven't yet done a good profile (If you are profiling this, note that WAL is a persistent setting, so you will want to delete the cache). This might also allow removing the PRAGMA synchronous=OFF

Finally, I also explicitly close the connection in main. This is relevant to this change, because it forces checkpointing of the WAL, which keeps reads fast, reduces disk space and means the cache.db remains a single self-contained file under regular use

Fixes #21136, see also discussion in #13916

For what it's worth, I feel there are many legitimate uses of concurrent mypy. At work, we often share cache between multiple projects. At home, I often end up having parallel runs with a debugger while working on mypy (although this PR just makes those ones hang waiting for the lock lol)

[github-actions]

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

[ilevkivskyi]

For what it's worth, I feel there are many legitimate uses of concurrent mypy

Can you give some more concrete examples? Also the problem is, as I mentioned in other issue, even if there are such uses, current incremental logic was not designed for this, and it would be tricky to guarantee correctness.

[ilevkivskyi]

Anyway, I am not strongly against this per se, as I mentioned in #13916 (comment) IMO the key point is to give a loud warning, the exact best effort semantics is not as important then.

Also, to be clear, although this will make SQLite cache behave like FS cache, this will not solve other concurrent-related crashes like #14521 or #18473, while disabling cache completely will fix those.