Limitations and Flaws of the Modeling Approach

[salihakalender]

Please review and share your comments

Limitations and Flaws of the Modeling Approach

• Focus on PM2.5 Only:
  Our study focuses mainly on PM2.5 as the key pollutant. This means we may overlook the combined effects of other air pollutants like NO₂, O₃, or SO₂, which can also impact health.

• Country-Level Aggregation:
  Using country-level average pollution and health data may hide important differences within countries, such as urban vs. rural areas or pollution hotspots.

• Limited Sample Size:
  We analyze 25 countries with reliable data, but this limits the global generalizability of our results and may bias findings toward countries with better data reporting.

• Data Quality and Reporting Differences:
  Health data may vary in accuracy due to differences in healthcare systems, diagnosis criteria, and reporting standards across countries.

• Potential Confounding Factors Not Included:
  Factors such as smoking rates, socioeconomic status beyond SDI, healthcare access, age distribution, and genetic predispositions may influence health outcomes but are not fully controlled for in our analysis.

• COVID-19 Mortality as Secondary Context:
  Including COVID-19 death rates adds useful context but these rates are influenced by many other factors (vaccination rates, health infrastructure) beyond pollution, making interpretation complex.

• Assumption of Average Pollution Impact Across Countries:
  Due to data aggregation at the country level, we assume average pollution impacts across countries, acknowledging that local environmental, genetic, and lifestyle factors may cause variability that our model cannot capture.

[linahKhayri]

Hey! First of all, thank you so much for pulling together the limitations, great work as always!! 🙏 I just wanted to add a few quick notes on some of the points, just to make sure we're framing them accurately based on how the dataset and our approach actually work.

On the PM2.5 Focus
You're right that we only look at PM2.5, but that’s actually because the GBD data already defines “particulate matter” as a specific risk factor and that’s the one we chose to analyze. So while we don’t look at NO₂, O3, etc., it’s not a flaw, just a scope choice based on how the data is structured. We could mention that this includes particulate matter in general, and not just PM2.5 in the strictest sense, to make that clearer.

On Confounders like Smoking, SES, etc.
We didn’t directly model things like smoking or healthcare access, but the GBD estimates already adjust for a lot of those factors in the background modeling. Since we’re using their risk-attributed values for PM, the confounding is already (partly) taken care of. And again, we’re specifically sticking to PM2.5 as the defined risk factor, so that helps narrow our focus and reduce confusion.

On COVID-19 Mortality
It's true that COVID outcomes are super complex and influenced by tons of things. But just to clarify we’re using COVID mortality more as a secondary context, not suggesting it’s directly caused by PM2.5 in our model. So we could definitely flag this as a limitation in interpretation, but I wouldn’t say it’s a flaw in our approach since we’re not drawing any causal conclusions.

[salihakalender]

Thank you Linah, I've revised according to your comments

[salihakalender]

Limitations and Flaws of the Modeling Approach

1. Country-Level Aggregation
   The use of national-level averages for both pollution exposure and health outcomes may obscure important regional disparities. Differences between urban and rural settings or localized pollution hotspots are not captured, potentially masking significant subnational variation in risk.

2. Limited Sample Size
   The study includes 25 countries with reliable data. While this ensures data quality, it limits the global representativeness of the findings and may bias results toward countries with more advanced environmental and health surveillance systems.

3. Data Quality and Reporting Differences
   Differences in healthcare infrastructure, disease surveillance, diagnostic criteria, and mortality reporting across countries can introduce variability in the health outcome data, affecting the comparability and robustness of cross-national analyses.

4. Assumption of Homogeneous Pollution Impact
   Due to reliance on country-level aggregation, the analysis assumes a uniform health impact of PM2.5 within each country. In reality, individual vulnerability may vary based on environmental, genetic, socioeconomic, and lifestyle factors that cannot be captured at this level of abstraction.

5. Interpretation of COVID-19 Mortality Context
   COVID-19 mortality is included in the study solely as a secondary contextual indicator. We do not model any causal relationship between PM2.5 and COVID deaths. However, since COVID outcomes are shaped by many other factors—such as vaccination rates, healthcare system capacity, and comorbidities—readers should be cautious not to overinterpret any apparent associations as evidence of direct environmental causation.

[linahKhayri]

this is a summarized paragraph of the Limitations
This study focuses on ambient particulate matter as defined in the Global Burden of Disease (GBD) framework, which includes PM₂.₅ and may also encompass other particulate types depending on data availability. Our analysis uses this exposure as the main risk factor to explore its association with health burdens from cardiovascular and respiratory diseases. However, we do not attempt to establish causality—our findings are observational and intended to highlight patterns and potential relationships. As we rely on country-level data, individual-level variation is not captured, and exposure or health estimates may carry uncertainty, especially in regions with limited data. Finally, focusing on 25 countries may limit the global generalizability of our results.