Quarto extension in VSCode should better support Python syntax highlighting

[ntluong95]

Currently, the code syntax highlighting for Python is very poor.
Consider these example below to see the different

In quarto

In Python source code

so at least module name like pandas, pd, numpy, np or function argument should be highlighted.
It seems no scope has been capture for these keywords.

Is there away that I can custom define the language myself in your quarto.tmLanguage file?

### Tasks

[kunqian1997]

any more updates now? the issue persist

[bakerbs]

I'd like to be able to use .qmd files for my python work in VSCode, but the lack of semantic highlighting makes development really challenging. I'm currently rendering .ipynb files using the quarto engine strictly because they have better syntax highlighting support. +1 for this issue.

EDIT: Ideally, this would be as simple as associating {python} code chunks within .qmd files as being "in scope" for Pylance to apply semantic highlighting... not sure how all the backend stuff works and/or where the scope of responsibility lies in VSCode extensions, but Pylance has great semantic highlighting and is ubiquitous for VSCode (Desktop) python development.

[austin3dickey]

I've noticed that other LSP features like hover, completions, go-to-definition, etc. all work in Python chunks in Quarto files. They work across many different LSP extensions like pylance, pyrefly, or ty, and either in VS Code or Positron. In all cases, it's just the semantic tokens that are missing. (That's the more precise term for what highlighting is missing here.)

[juliasilge]

I think whatever that LSP feature is needs to be added here to the "middleware":

quarto/apps/vscode/src/lsp/client.ts

Lines 104 to 111 in 9838c73

	const middleware: Middleware = {
	handleDiagnostics: createDiagnosticFilter(),
	provideCompletionItem: embeddedCodeCompletionProvider(engine),
	provideDefinition: embeddedGoToDefinitionProvider(engine),
	provideDocumentFormattingEdits: embeddedDocumentFormattingProvider(engine),
	provideDocumentRangeFormattingEdits: embeddedDocumentRangeFormattingProvider(
	engine
	),

[juliasilge]

Support for semantic tokens in .qmd has been implemented in #868 and merged in, and it will be available in the next Quarto extension release!