Fix(rust-tokenizer)!: return token vector in tokenize even on failure#5155

georgesittas merged 1 commit intomaintobymao/sqlglot:mainfrom

jo/always_return_tokens_in_sqlglotrs_tokenizetobymao/sqlglot:jo/always_return_tokens_in_sqlglotrs_tokenizeCopy head branch name to clipboard