WCS solve fails for globular clusters

[emannenicholas]

A user is having issues getting WCS solves on images of globular clusters (dense fields with many possible saturated sources). He created a video of changes he'd like to request we make to our banzai.astrometry.py, specifically the SOURCE_LIMIT=50 parameter. This parameter got changed in 2021 from 200 to 50. Our documentation still says 200. Is there any way we can help this person get his solves?

[mfitzasp]

Hi! Thanks for looking at this.

Maybe make a little loop that tries 50 to start with, then 200, then 1000, then if it still fails then say it failed. But I think "50" isn't the WCS failing, it is that it didn't try hard enough in a field where the brightest 50 stars are in unuseful positions!

I give all of the sources in the M13 images to astrometry.net locally (running on linux) and it comes back at me in a couple of seconds max with a good fit to those images. Perhaps the alternative way to deal with the timeouts is not to restrict the number of sources, but put an actual timeout (e.g. using https://pypi.org/project/func-timeout/) on the astrometry.net routine... which is what I do (although I just use the timeout function in linux). My pipeline has all the time in the world, so I set it quite high, but not infinitely high. But you could set it to 30 seconds or whatever you like. Realistically if astrometry.net hasn't found a solution within 30 seconds, it is very unlikely to ever find a solution.

Here is the linux command I use that reduces those M13 images prefectly fine and within a second or two. As you can see I feed in all sources 3 sigma above the background threshold. I think the assumption that is being made here on LCO's behalf (knowingly or unknowingly) is that the "brightest 50 sources" are distributed randomly across the frame, whereas in the field of a globular cluster (and probably also open clusters and associations), this is probably not true. It is also true that if the 50 stars are all in M13 itself, perhaps the size of the quads in astrometry.net don't actually even get that small. The command below runs succesfully in 2 seconds on a Dell XPS from 2015 gathering dust in my garage, so I assume it should run faster than that at AWS.

timeout 120 /usr/local/astrometry/bin/solve-field --scale-units arcsecperpix --scale-low 0.54 --scale-high 0.65 --ra RA --dec DEC --radius 0.5 --cpulimit 90 --overwrite --no-verify --no-plots --use-source-extractor --source-extractor-path '/usr/bin/sex -DETECT_THRESH 3 -ANALYSIS_THRESH 3 -DETECT_TYPE CCD -DETECT_MINAREA 5 -BACK_SIZE 32 -BACK_FILTERSIZE 1' astromtemp_flipped.fit

[cmccully]

I suspect the 50 is really a red herring. How many of your bright stars are saturated? We by default don't try to centroid on saturated stars, but then it ends up trying to use fainter stars (quads) that aren't in the indexes.

[mfitzasp]

I would say approximately none are saturated. They are very well exposed. The highest value I can find in any of the stars is around 26,000.

Here are two of the proposals that the user sent me. They've tried multiple times and m13 always fails (https://observe.lco.global/?target=m13&limit=20) so it is something intriguing about the field of view
https://observe.lco.global/requests/3162406
https://observe.lco.global/requests/3156458

As you can see, all the bright stars aside from one or two are clustered in the center... so thats why I was thinking it is just picking 50 stars essentially in the center of the image and failing because it isn't spread out.

[mfitzasp]

Maybe if thats the case, the number shouldn't be 50.... perhaps it starts at 50 then calculates the field coverage and if it isn't above some threshold (75%), it then tries 100, then 200 etc. etc. until it has a good field coverage then sends it to astrometry.net. So 50 should always usually work but for more concentrated fields like this, it will find a larger number that will work for it.

[mfitzasp]

e.g. what is the size of the rectangle as a fraction of the full fov rectangle considering the largest and smallest RAs and DECs in the sourcelist. If that is above X% of the fov, send to astrometry.net but if it isn't above X%, then increase the number of sources until X% is reached.

[mfitzasp]

Actually you wouldn't need to know the field of view.... you could do it terms of the maximum POSSIBLE coverage just from the source list.

So the area of the "50" source list.... (max_ra50 - min_ra50) * (max_dec50 - min_dec50)

divided by the larger area of the full list of sources ... (max_ra_full - min_ra_full) * (max_dec_full - min_dec_full)

So you'd try 50 if it covered X% (perhaps 75) of the full area provided by the original source list. This would all depend on four single stars in each list describing the rectangle.... but that would be enough I reckon. It would only be a handwavey factor that just makes sure an adequate coverage is achieved and if it isn't to up the sources 50 -> 100 -> 200 -> 500 -> 1500 etc.

I imagine there are going to be a good collection of objects like this... globulars of various shapes and sizes, open clusters that are particular fractions of the true fov and just coincidental statistical fluctuations in the galactic plane that don't like just 50 stars.

[mfitzasp]

Hi Curtis, Resurrecting a year old thread for your interest. Taking the long route! On PTR we found WCS "fails" for a dense open cluster... sorta similar to this thread from a while ago. The WCS "fails" are quite visible when they happen on PTR because we use the WCS to reproject the image. A fail looks like this: [image: image.png] [image: image.png] You probably wouldn't notice it unless you reproject the image and it whacks you in the face... which we do. So I did a little investigation. If we take the brightest 200 stars which we were doing (and BANZAI was/is doing? maybe it is 50?) Then we see the following [image: image.png] Thats RA and DEC. You can see most of the brightest stars (blue dots) are in the middle and there is a corner where it is missing some sources - which is likely the problematic corner because there are no sources to constrain the wcs in that region. The orange dots were my first (not totally optimal) attempt to cover the full frame. So a.net needs X numbers of stars in descending flux order .... so instead of just the brightest 200 stars, I wrote a basic routine to pick the brightest 200 stars *that minimises the standard deviation of proportional source numbers across a 4x4 binned grid and pick the brightest set of 200 stars where that set equals the median standard deviation across the image. *Thats a mouthful. Once I do that I get this: [image: image.png] Where the green dots are a set of 200 dimmer sources in the image, but certainly bright enough for platesolving. This takes a.net a little longer to process as it has to wonder "why aren't those brighter stars there?" but it gets there and fits a wcs solution across all the positions in a frame. This probably emulates an image where those central stars were heavily overexposed. The routine to select the sources is pretty quick and it isn't particularly sophisticated. Thought you might like to know about that! The image above is of a relatively dense open cluster in the galactic plane, but I imagine the situation is even trickier with a globular away from the galactic plane because the brightest 200 (or 50) stars are likely mostly in the middle. Anyhow, I will let you know how it goes as we keep using it and if it seems to solve this problem. We will test it on a bunch of clusters and see how it goes.... but I thought you'd like to be aware of that. The globular cluster (M13) that this ancient (Feb 2023) thread referred to didn't work well with the LCO BANZAI a.net but worked fine with my local a.net..... but I think back then that was just dumb luck in that the way I photometered the image gave more coverage of the field for some reason. Michael

…

On Thu, Mar 2, 2023 at 3:20 PM Curtis McCully ***@***.***> wrote: I suspect the 50 is really a red herring. How many of your bright stars are saturated? We by default don't try to centroid on saturated stars, but then it ends up trying to use fainter stars (quads) that aren't in the indexes. — Reply to this email directly, view it on GitHub <#357 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEAHGNW5KSIOH3Q2SONTZMDW2D6KPANCNFSM6AAAAAAVNWO7RY> . You are receiving this because you commented.Message ID: ***@***.***>

-- Michael Fitzgerald Mobile: 0431 480007 Email: ***@***.***