Does suggest query support LANG search

[sincerefly]

When push or query data, I use { LANG: "cmn" }, It's Mandarin Chinese, maybe the same as Simple Chinese

It's works well, testing data:

sonicChannelIngest.push("testing", "area_code", "230102", "黑龙江省哈尔滨市道里区", { LANG: "cmn" })
sonicChannelIngest.push("testing", "area_code", "230103", "黑龙江省哈尔滨市南岗区", { LANG: "cmn" })
sonicChannelIngest.push("testing", "area_code", "230104", "黑龙江省哈尔滨市道外区", { LANG: "cmn" })

# query
sonicChannelSearch.query("testing", "area_code",  "黑龙江省", { LANG: "cmn" })

# result
["230102", "230103", "230104"]

when use suggest:

sonicChannelSearch.suggest("testing", "area_code",  "黑龙江省")

got the error:

Error: { code: 'unexpected_error', message: 'query_error' }

so, does suggest support LANG params query, thanks

[valeriansaliou]

Suggest should work yes. What do you have in Sonic debug logs?

[sincerefly]

@valeriansaliou

I use the v1.2.0 sonic deployed with docker images

the log when suggest "黑龙江省":

(DEBUG) - got channel message: SUGGEST testing area_code "黑龙江省"
(DEBUG) - will dispatch search command: SUGGEST
(DEBUG) - parsed text parts (still needs post-processing): "黑龙江省"
(DEBUG) - parsed text parts (post-processed): 黑龙江省
(DEBUG) - dispatching search suggest #c9vmnBCo on collection: testing and bucket: area_code
(DEBUG) - will suggest for #c9vmnBCo with text: 黑龙江省, limit: 5
(DEBUG) - not detecting locale from lexer text: 黑龙江省
(DEBUG) - fst store acquired from pool for collection: testing (pool key: <c89d0f8>/<1c01776c>)
(DEBUG) - lexer yielded word: 黑
(DEBUG) - lexer yielded word: 龙
(DEBUG) - wrote response with values: ERR (query_error)
(INFO) - took 0ms/308us/308505ns to process channel message

[valeriansaliou]

Thanks. Filing a bug. Will process it when I have time.

[18601673727]

I'm having same issue on valeriansaliou/sonic:v1.2.3

{ code: 'unexpected_error', message: 'query_error' }

[valeriansaliou]

@18601673727 are you sending Chinese characters in the command? Is that UTF-8 or UTF-8 extended or UTF-16?

[18601673727]

@valeriansaliou I'm sending Chinese characters via node-sonic-channel just like @sincerefly did, but one small difference is for single character like 黑 is OK, if I pass more than 1 like 黑龙 it gives that error, and I believe all of these characters are in UTF-8, no fancy staff say emojis. 😂

[jukbot]

got the same issue 'query_error' when parsing Thai language in suggest method.

[valeriansaliou]

Hello! SUGGEST does not support the LANG attr, as suggestions are langage unaware by design.

[zanpen2000]

got same issue on 1.2.6

(DEBUG) - got channel message: SUGGEST origin title "鲁迅" LIMIT(10)
(DEBUG) - will dispatch search command: SUGGEST
(DEBUG) - parsed text parts (still needs post-processing): "鲁迅"
(DEBUG) - parsed text parts (post-processed): 鲁迅
(DEBUG) - dispatching search suggest #CVaH6tE2 on collection: origin and bucket: title
(DEBUG) - parsed meta part as: LIMIT = 10
(DEBUG) - handle suggest meta: LIMIT = 10
(DEBUG) - will suggest for #CVaH6tE2 with text: 鲁迅, limit: 10
(DEBUG) - not detecting locale from lexer text: 鲁迅
(DEBUG) - fst store acquired from pool for collection: origin (pool key: <8b732c7e>/<7932423e>)
(DEBUG) - lexer yielded word: 鲁
(DEBUG) - lexer yielded word: 迅
(DEBUG) - wrote response with values: ERR (query_error)