Why does the semantic engine add a delay to scheduling semantic analysis tasks when it triggers completion?

[howmushroomsay]

Description

I want to learn code related to DBeaver's code completion. When I tested the semantic completion engine, I found that it took more than 500ms for the completion prompt to appear. After debugging, it was found that the semantic parsing task added a delay. I would like to ask why this delay was chosen and whether this value can be adjusted

DBeaver Version

25.0.3.202504201842

Operating System

win11

Database and driver

No response

Steps to reproduce

No response

Additional context

No response

[davestokes]

@howmushroomsay Thank you for your report. We need to ask why and report back to you.

[E1izabeth]

The description of your findings looks as if you found DBeaver always introducing unwanted delay, which simply is not true. It highly depends on the circumstances, query structure, state of the metadata cache, concrete editing sequences and precise key press timings in the whole scenario. To better understand what is really happening in your case, it would be helpful to introduce a certain amount of debug logging and measuring all around the SQLBackgroundParsingJob and SQLCompletionProcessor. Particular scheduling delay is just a small piece in a bigger picture. Let me try to describe how it works.

• There are two subsystems: semantic analysis and completion. They are connected, but different.
  The analysis triggered each time DBeaver's SQL Editor component state changes (any of edit/input/paste/delete/scroll to not-yet-parsed fragment).
  The completion triggered either due to auto-activation (character input) or due to explicit assistance command (Ctrl+Space) by the underlying Eclipse Platform logic, not by DBeaver-specific logic.
  These subsystems don't trigger each other.

• Completion engine API is a part of the Eclipse Platform and, by design, has no connections with our semantic analysis besides the fact that they both serve the text editor component. DBeaver's completion engine implementation does NOT trigger semantic analysis, it just asks for analysis results (here calling this method). If the text fragment of interest is parsing (or scheduled for parsing) at the moment, completion just waits for it. Then it takes steps to prepare completion proposals (all the following SQLQueryCompletionAnalyzer logic). These extra steps also take time, so by the moment of proposals being ready, editor state may have been changed again. In this case parsing will already be rescheduled and running by the time of completion engine discovering it, so then proposals will be recomputed according to the actual editor state (see here).
  Both completion proposals preparation and background parsing are asynchronous for editing operations and for each other, so consider the following hypothetical sequence of events:

 - key press
 - completion triggered
 - completion initializing
(potentially more key presses)
 - completion asks for analysis results
(potentially more key presses)
 - completion prepares proposals
(potentially more key presses)
 - completion delivers proposals
(potentially more key presses)
 - completion proposals appear on the screen

Any key press here might or might not change the editor state, might or might not trigger parsing and invalidate already prepared pieces of information from the previous steps.

• More details about parsing timeouts then. You found this schedule() method in the SQLBackgroundParsingJob class, so, i believe, you can also find its usages. Let's now read the fragment of your interest line by line. Remember, it works for any editor state change, not only when completion activation follows, and completion is not always prepended with parsing in time.

If parsing is not running yet, but was already scheduled,
    cancel and reschedule from scratch.

If DOT character was entered (heuristic for member access expression being typed in)
    and parsing is ALREADY RUNNING
        then take half of the default timeout 
            (doesn't matter when the current run will actually end, 
             it may even finish later then the delay, and restarted immediately in this case)
    if parsing it NOT RUNNING YET
        then take ZERO delay 
    (no decisions about the completion here, but the chance of it being triggered is known to be higher)
otherwise for ANY other editor state changes
    if parsing is ALREADY RUNNING
        then take twice the default timeout 
            (heuristic is that user stopped typing, then parsing started, 
             then user started typing again and won't stop for some time more, 
             because they didn't bother to wait for the current parse, 
             and then there'll be another parse when he'll stop)
    otherwise
        take just one timeout

So we are actually reducing the delay when possible and adding only when the parsing is already running but will be triggered later again.

Note that by the time the schedule method is called, parsing may already be running, and we don't know how much time it will take. There may be HUGE queries, whose analysis requires more than 2 seconds.
There may follow any amount of upcoming input operations, including removals for Backspace key at any point in time, and we know nothing about them in advance, so there is no point in reacting to any input immediately.

• On parsing vs completion and highlighting.
  Parsing takes time. Sometimes a noticeable amount of it. Updating text coloring, error markers and outline view also takes time. Meanwhile, characters' typing flow takes much less time between key presses, so we have two situations here:

1. User does his input and editing operations without really stopping and reading what is presented on the screen.
2. User stops typing and takes pause to think about what he should do next, then maybe proceeds with the next sequence of editing operations (1).

When the user just types, not really reading what is on the screen (1), there is no point in parsing and doing complex highlighting, because they just type no matter what is currently on the screen, until their current considerations' part will be expressed. For example, you are entering a word (a name of something, or a keyword). You won't stop pressing keys until you accomplish the intention of entering this word. Then you think about what to do next and whether it is correct or not (2) - and here is the moment to assist! Or you remembered something while entering the word and now you are immediately deleting it, without waiting for the editor to react and highlight it (1), to then type in the other thing. Or to stop typing and do something else (2).

Summing it all up, there are at least two points to have delays between input events and parsing activities:

• To do parse and analysis inbetween of continuous editing sequences.
• To not parse inside of one continuous editing sequence and/or when the editing happens without waiting for already scheduled parse.

There are well-established foundations of UI responsiveness time limits (see here for some references), of which it is known that an expected computer reaction within one second doesn't break the train of the user's thought.
We can easily measure input timeouts by making a video record of text editing and then measuring the average time delta between key presses of continuous edits.
This is the base of how the present value of scheduling timeout was established.

Basically, if you are using any other modern IDE, like JetBrains IDEA for example, you can notice how editor reacts for input key presses: there is always a fraction of second between the input and the editor reaction (coloring update and/or completion list appearance), so that the editor has its chance to know if it faces with continuous input or the user is ready to observe the feedback.

P.S. Note that all of these details don't cover metadata queries required by both analysis and completion engines when the connection was recently established and its metadata cache has not yet been populated. Cold metadata queries may take a noticeable amount of time. Upcoming activities using cached information work much faster.