Overwriting session expirary value in session callback not working when using custom Credentials provider

[ghost]

Describe the bug
I'm not able to overwrite the session expirary using the session callback when using when using custom Credentials provider

Steps to reproduce

1. Implement custom Credentials provider

export default Providers.Credentials({
  id: "app-signin",
  name: "Signin",
  authorize: async (credentials) => {
    try {
      const { data } = await axios.post("/api/users/signin", {
        email: credentials.email,
        password: credentials.password,
      });

      if (data) {
        // data contains accessToken and accessTokenExpirary
        return data;
      } else {
        return null;
      }
    } catch (error) {
      return null;
    }
  },
});

2. in [...nextauth].js set default expirary in the session option e.g.

options = {
  session: {
    jwt: true,
    maxAge: 60, // expire after 60 seconds
  },
}

3. In session callback overwrite the session object

  callbacks: {
    async session(session, token) {
      // Add property to session, like an access_token from a provider
      if (token?.user) {
        const { user } = token;
        session.accessToken = user.accessToken;
        // where accessTokenExpirary = new Date(user.decodedToken.exp * 1000)
        session.expires = accessTokenExpirary; 
      }
      return session;
    },
    async jwt(token, user, account, profile, isNewUser) {
      if (account?.type === "credentials" && user) {
        token.user = user;
      }
      return token;
    },
  },

Expected behavior
By overwriting the session objects expires value, i would have expected to overwrite the default expired value. This is not happening. I'm not able to overwrite the expirary time

Additional context
I want to be able to control the expirary of the session based on the expirary of the accessToken returned from the server. So if the accessToken expires in 10 days, I also want the session to expire in 10 days.

Setting the maxAge works fine. It's just I'm not able to dynamically set the expirary. In my case I want the session to expire as soon as the jwt accessToken returned by the server expires.