Conversation
Add test case for Vars, France pelias/api#1568
|
Wow, quite comprehensive testing. We'll run this through our own tests as well and report back. Thanks for taking the time to try out lots of options. The So hopefully we can improve things even more someday when we fix scoring issues. |
|
Hey this looks pretty good! We tested it out and there don't really look to be any unexpected effects. Queries with a focus point do now prefer things closer to the focus point by a tiny bit. The acceptance tests provide the best examples, especially with the Union Square tests that have always been challenging to make pass:
The flip side is that some city/locality/high population results come up lower. Both the following examples are only slight regressions but worth looking more closely at.
Also NYC and SF are hard queries for various reasons (aliases, duplicate names, etc), so while they're obviously very prominent examples, if they are the only things affected then that's not so bad. |
2 participants
Hi there, this is the PR for #1567
I kept in mind what @orangejulius says in #1567 (comment):
I did several tests to find out what would be the best configuration.
Results with population and popularity boost
I started by decreasing population and popularity boosts as I said in my issue. I tried with three values, 8, 10 and 12 and ran acceptance tests (issues are from
autocomplete_acushnet_antiques.jsonandautocomplete_focus.json).With boost decreased to 10 for popularity and population I had 2 improvements, 1 regression and 2 changes.
Improvements
/v1/autocomplete?focus.point.lat=41.652889&focus.point.lon=-70.922898&text=antiq/v1/autocomplete?focus.point.lat=41.652889&focus.point.lon=-70.922898&text=antiqueThere is no doubt, this is an improvement
Regressions
/v1/autocomplete?focus.point.lat=40.744131&focus.point.lon=-73.990424&text=San FranciscoThis one is very weird,
South San Franciscotook the place ofSan Francisco...Change
/v1/autocomplete?focus.point.lat=41.05343&focus.point.lon=-73.53873&text=New YorkHere the new position of New York City is 42 ! This may be considered as a regression, that's why I will not chose this configuration.
/v1/autocomplete?focus.point.lat=41.05343&focus.point.lon=-73.53873&text=New York, NYHere New York is loosing one place, but still good enough. At this point I decided to change my mind and increase focus point instead of decreasing population/popularity.
Results with focus point increase weight
So now I want to have the same scale between focus point boosts and population/popularity, so I set the weight of focust point to 20. With this configuration I had 2 improvements, 0 regression 🎉 and 1 change.
Improvements
/v1/autocomplete?focus.point.lat=41.652889&focus.point.lon=-70.922898&text=antiq/v1/autocomplete?focus.point.lat=41.652889&focus.point.lon=-70.922898&text=antiqueThere is no doubt, this is still an improvement
Regressions
No regressions ✔️
Change
/v1/autocomplete?focus.point.lat=41.05343&focus.point.lon=-73.53873&text=New YorkHere New York is loosing one place, but still good enough.
Resume
So instead of decreasing population and popularity boosts, I chose to increase focus point weight. The overall result seems good and close to current responses.
Results are also better in my initial test with
Vars(the lang is important here)./v1/autocomplete?lang=fr&focus.point.lat=48.03661925338169&focus.point.lon=6.580299229512&text=vars📝 TODO: write acceptance test for
Varsrelated #1567