Remote authentication with GitHub failing

[ghost]

I have the following configuration in NetBox:

REMOTE_AUTH_ENABLED: true
REMOTE_AUTH_BACKEND: "social_core.backends.github.GithubTeamOAuth2"
SOCIAL_AUTH_GITHUB_TEAM_ID: "xxxxxxx"
SOCIAL_AUTH_GITHUB_TEAM_KEY: ...
SOCIAL_AUTH_GITHUB_TEAM_SECRET: ...

And I have setup an OAuth App in my GitHub org. However, everytime I try to autheticate using GitHub it throws the following error post the GitHub prompt for authorizing NetBox:

<class 'social_core.exceptions.AuthFailed'>
Authentication failed: User doesn't belong to the organization
Python version: 3.10.6
NetBox version: 3.4.1

Any pointers on what could be the issue?