Skip to content

Commit 9ce15e5

Browse files
committed
Disable sequential scan on database tables in copy threads
This forces the DELETEs to use an index scan which is much faster. The underlying problem here is probably that the DELETEs are called for all OSM objects in the input file, regardless of whether they are even in the table. For most tables most ids will not be in there. So PostgreSQL overestimates the number of "hits" it is going to get and thinks a full table scan is more efficient when in fact an index lookup is much better. The COPYs used in the same database connection are not affected by this. In many tests I have done I only see this problem in some cases, but when it hits it can slow down processing considerably. This could explain the occasional slow processing that some people have been reporting over the years and that we could never really nail down (#1674, #1971).
1 parent fb6bd80 commit 9ce15e5

File tree

1 file changed

+8
-0
lines changed

1 file changed

+8
-0
lines changed

src/db-copy.cpp

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,14 @@ void db_copy_thread_t::thread_t::operator()()
131131
// Let commits happen faster by delaying when they actually occur.
132132
m_conn->exec("SET synchronous_commit = off");
133133

134+
// Disable sequential scan on database tables in the copy threads.
135+
// The copy threads only do COPYs (which are unaffected by this
136+
// setting) and DELETEs which we know benefit from the index. For
137+
// some reason PostgreSQL chooses in some cases not to use that index,
138+
// possibly because the DELETEs get a large list of ids to delete of
139+
// which many are not in the table which confuses the query planner.
140+
m_conn->exec("SET enable_seqscan = off");
141+
134142
bool done = false;
135143
while (!done) {
136144
std::unique_ptr<db_cmd_t> item;

0 commit comments

Comments
 (0)