ImportError: Numba needs NumPy 1.26 or less

[donbowen]

Hi @s3alfisc / @leostimpfle

After updating to the latest marginaleffects package to test the C()/i() patch, which requires numpy 2.4.3, via pip install -U marginaleffects, pyfixest throws an error on import:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[1], line 2
      1 from marginaleffects import hypotheses
----> 2 import pyfixest as pf
      4 data = pf.get_data() 
      5 fit = pf.feols("Y ~ i(f1)", data=data)

File c:\Users\DonsLaptop\anaconda3\Lib\site-packages\pyfixest\__init__.py:2
      1 # Import modules
----> 2 from pyfixest import (
      3     did,
      4     errors,
      5     estimation,
      6     report,
      7     utils,
      8 )
      9 from pyfixest.did import (
     10     SaturatedEventStudy,
     11     did2s,
   (...)
     14     panelview,
     15 )
     17 # Import frequently used functions and classes

File c:\Users\DonsLaptop\anaconda3\Lib\site-packages\pyfixest\did\__init__.py:1
----> 1 from pyfixest.did.estimation import (
      2     did2s,
      3     event_study,
      4     lpdid,
      5 )
      6 from pyfixest.did.saturated_twfe import SaturatedEventStudy
      7 from pyfixest.did.visualize import (
      8     panelview,
      9 )

File c:\Users\DonsLaptop\anaconda3\Lib\site-packages\pyfixest\did\estimation.py:5
      1 from typing import Optional, Union
      3 import pandas as pd
----> 5 from pyfixest.did.did2s import DID2S, _did2s_estimate, _did2s_vcov
      6 from pyfixest.did.lpdid import LPDID
      7 from pyfixest.did.saturated_twfe import SaturatedEventStudy

File c:\Users\DonsLaptop\anaconda3\Lib\site-packages\pyfixest\did\did2s.py:9
      6 from scipy.sparse.linalg import spsolve
      8 from pyfixest.did.did import DID
----> 9 from pyfixest.estimation.estimation import feols
     10 from pyfixest.estimation.feols_ import Feols
     11 from pyfixest.estimation.FormulaParser import FixestFormulaParser

File c:\Users\DonsLaptop\anaconda3\Lib\site-packages\pyfixest\estimation\__init__.py:2
      1 from pyfixest.estimation import literals
----> 2 from pyfixest.estimation.demean_ import (
      3     demean,
      4 )
      5 from pyfixest.estimation.detect_singletons_ import (
      6     detect_singletons,
      7 )
      8 from pyfixest.estimation.estimation import (
      9     feglm,
     10     feols,
     11     fepois,
     12     quantreg,
     13 )

File c:\Users\DonsLaptop\anaconda3\Lib\site-packages\pyfixest\estimation\demean_.py:3
      1 from typing import Any, Callable, Optional
----> 3 import numba as nb
      4 import numpy as np
      5 import pandas as pd

File c:\Users\DonsLaptop\anaconda3\Lib\site-packages\numba\__init__.py:55
     50             msg = ("Numba requires SciPy version 1.0 or greater. Got SciPy "
     51                    f"{scipy.__version__}.")
     52             raise ImportError(msg)
---> 55 _ensure_critical_deps()
     56 # END DO NOT MOVE
     57 # ---------------------- WARNING WARNING WARNING ----------------------------
     60 from ._version import get_versions

File c:\Users\DonsLaptop\anaconda3\Lib\site-packages\numba\__init__.py:42, in _ensure_critical_deps()
     40     raise ImportError(msg)
     41 elif numpy_version > (1, 26):
---> 42     raise ImportError("Numba needs NumPy 1.26 or less")
     43 try:
     44     import scipy

ImportError: Numba needs NumPy 1.26 or less

[donbowen]

I simply downgraded Numpy in my conda after upgrading pyfixest and marginaleffects and it worked.

[s3alfisc]

Great! Does marginal effects works as well?

Btw, because of problems like yours we are trying to deprecate our dependency on numba =)

[donbowen]

Yes, marginal effects 0.5.0 works with i(), but not with i(ref) or C(). I have pyfixest 0.40.1 installed.

Works with i() and no base:

from marginaleffects import hypotheses
import pyfixest as pf
data = pf.get_data()
fit = pf.feols("Y ~ i(f1)", data=data)
hypotheses(fit,f'Q({fit._coefnames[1]})+Q({fit._coefnames[2]})=0')

But it fails when you set the omitted level:

fit2 = pf.feols("Y ~ i(f1, ref=9)", data=data)
hypotheses(fit2,f'Q({fit2._coefnames[1]})+Q({fit2._coefnames[2]})=0')
# >     Q(C(f1, contr.treatment(base-(9))[T.0.0])+Q(C(f1, contr.treatment(base-(9))[T.1.0])-(0)
# >                                       ^
# > SyntaxError: invalid syntax. Perhaps you forgot a comma?

And going the C() route still fails, whether or not you set the base level. Whcih is

fitC = pf.feols("Y ~ C(f1)", data=data.assign(f1 = data['f1'].astype(str).astype('category')))  
hypotheses(fitC, f"{fitC._coefnames[1]} + {fitC._coefnames[2]} = 0") 

[leostimpfle]

Hi @donbowen, thanks for reporting! Looks like there's still something not quite right with the parsing of encoded variables, I'll open an issue in the marginaleffects monorepo. In the meantime you could try using the positional syntax b* , for example

fit2 = pf.feols("Y ~ i(f1, ref=9)", data=data)
hypotheses(fit2, "b1 + b2 = 0")

[donbowen]

That worked! I built a b# to coef name map, and I'm in business, thanks!

Parsing formulas is kind of nasty business, you're all quite brave.

[s3alfisc]

Parsing formulas is kind of nasty business, you're all quite brave.

Agreed! We (mostly @leostimpfle, actually) have been working really hard towards no more formula parsing in the code base ... 😄