Adding the react/jsx-filename-extension is just fine as a rule, but adding some direction that you will need to add the --ext=js --ext=jsx arguments to the eslint cli for it to work. It might cause issues for users who are not aware of this when they start using the eslint-preset-airbnb package. Adding a line to the README might be helpful, and I would be happy to do so. 😄

Ideally, you could configure this as an option in .eslintrc but that is not currently available:
eslint/eslint#801