Zone type Analysis: Run statistical tests to verify the density grouping results (4.5.2)

[manaswisaha]

For section 4.5.2, we need to test the statistical significance of the results showing that accuracies across the density based groups are similar.

Related to #19.

[misaugstad]

I don't think that we need a statistical test for this. A statistical test to say that the means are truly "similar" is actually really uncommon and not simple. Just testing to see if there is a difference between the means and failing to reject the null (the null being that the means are equal) is not sufficient to say that the means are similar. This really feels like one of those places where we can just let the data speak for themselves and we don't need a statistical test; it opens up this whole other can of statistical worms that I don't think is worth opening (the can contains things like Bland-Altman plots and estimation statistics)

[misaugstad]

@Manaswis thoughts?

[manaswisaha]

I will wait until I can confirm with @jonfroehlich about this. Obviously, this would require reading the sections. Hence, I would hold off on it until then.

[manaswisaha]

I have another idea for zone type analysis, which might show differences in accuracies.

I went through the definitions of the zones again. Based on the definitions of the given zone types, instead of two groups of densities, we should have low, moderate, medium and high density groups. We might get better results. I am sure the built infrastructure is different in zones with different densities. I think we just have to figure out the correct sub-groups for the zones, and I think a finer grouping based on density should work.

@misaugstad: Do you think we can divide routes based on these finer groups? Is it possible?

[misaugstad]

Do you think we can divide routes based on these finer groups? Is it possible?

I don't think so. When I tried to do the density analysis originally, that is what I wanted to do, but there is just not enough data in the high density range. I don't remember exactly how much there was, but I did try it before and saw that there wasn't enough.

[manaswisaha]

If there are significant sample sizes in other groups, I think it should be fine if there isn't much in one group. We won't analyse it. At least we would have three distinct groups. I am suggesting this because I was reading about sidewalk planning and density measures that they have for land use planning, the infrastructure is built differently based on the density of the land use type.

[misaugstad]

Upon further reflection, I have recalled the bigger problem with splitting into four density categories: each zone type doesn't really fit into just one category. They are each made up of a bunch of sub-zones, so each zones end up spanning multiple densities. And those sub-zones often say things like "low to moderate density"...

Putting each zone type into one of two density categories was sort of imprecise for the above reason, but trying to put each zone type into one of four density categories sounds suuuper imprecise.

It would be a bit better if we used the sub-zones to determine density, but this would require a ton of work (many sub-zones don't specify density, so I would have to go through DC's zoning documentation to assign densities to them myself; I would have to modify the zoning script and rerun; a bunch of the sub-zones specify a range of densities anyway).

[manaswisaha]

I see. That's a ton of work with unknown ROI.

[misaugstad]

Yeah, unknown and honestly it is more likely to be a very low ROI, since the interesting zones are still likely to be the ones with not enough data :)