How to use the localBorrow family of functions properly in the context of an effect action returning a function m a -> IO a?

[jaecam]

Hi, I just wanted to start off by mentioning I haven't been using effectful for that long so it's possible I'm missing something crucial here. So the TLDR is that I'm trying to essentially wrap the monad transformer type ScottyT from scotty with effectful, so that I can have a given Eff es monad in ActionT m a context and thus run all the stuff I need to do like talking to the DB and such without having to unlift every single thing to IO or sprinkle IOE :> es everywhere (scotty needs m in ActionT m a to have an instance of MonadUnliftIO). Most of the work is done for this, but I'm just playing around with including what is essentially an effect action like UnliftResponse :: m (m Response -> IO Response) in my effect, and how exactly to deal with lifting the inner monad during handling.

The setup looks like so:

ScottyIO effect:

data ScottyIO :: Effect

type instance DispatchOf ScottyIO = Static WithSideEffects
data instance StaticRep ScottyIO  = ScottyIO

runScottyIO :: (HasCallStack, IOE :> es) => Eff (ScottyIO : es) a -> Eff es a
runScottyIO = evalStaticRep ScottyIO

This effect is essentially just a marker to let me write a newtype monad transformer ScottyInner (see below) with its own MonadIO and MonadUnliftIO instances when the ScottyInner-transformed monad is some Eff es where Scotty :> es, ScottyIO :> es (Scotty here necessary as I pass some options through that - right now this is an UnliftStrategy for the MonadUnliftIO instance).

Just as a small explanation, I'm creating MonadIO and MonadUnliftIO instances since those are constraints are imposed by my usage of scotty.

Scotty effect:

data Scotty :: Effect where
    Opts           :: Scotty m ScottyOpts
    UnliftResponse :: Scotty m (m Response -> IO Response)

type instance DispatchOf Scotty = Dynamic

scottyOpts :: (Scotty :> es) => Eff es ScottyOpts
scottyOpts = send Opts

... where ScottyOpts is just a product type with some field optics defined (from optics):

data ScottyOpts = ScottyOpts
    { unliftStrategy :: UnliftStrategy
    }
makeFieldLabelsNoPrefix ''ScottyOpts

Now, the issue I have is in the handler for this Scotty effect. Here's what I'm currently working with:

runWithOpts :: (HasCallStack, ScottyIO :> es) => ScottyOpts -> Eff (Scotty : es) a -> Eff es a
runWithOpts opts = interpret $ \env -> \case
    Opts           -> pure opts
    UnliftResponse -> localSeqBorrow @'[Scotty] env $ \borrow -> do
        --fmap (. raise) $ borrow $ defaultUnliftResponse
        fun <- borrow defaultUnliftResponse
        let fun2 = fun . raise
        pure fun2

defaultUnliftResponse :: (HasCallStack, Scotty :> es, ScottyIO :> es) => Eff es (Eff es Response -> IO Response)
defaultUnliftResponse = runInner $ withRunInIO $ \unlift -> pure $ unlift . lift

defaultUnliftResponse just presents the unlifting function provided within the MonadUnliftIO instance for ScottyInner where m ~ Eff es, Scotty :> es, ScottyIO :> es in ScottyInner m (so essentially the (unlift . runInner) expression below):

newtype ScottyInner m a = ScottyInner { runInner :: m a }
    deriving (Functor, Applicative, Monad)

instance MonadTrans ScottyInner where
    lift = ScottyInner

instance MFunctor ScottyInner where
    hoist nat (ScottyInner m) = ScottyInner (nat m)

type ScottyEff es = ScottyInner (Eff es)

-- | Explicit constraining of the effect list to include `ScottyIO` such that `IOE` has to be in there ultimately.
instance (ScottyIO :> es) => MonadIO (ScottyEff es) where
    liftIO = lift . unsafeEff_

-- | Explicit constraining of the effect list to include `ScottyIO` such that `IOE` has to be in there ultimately.
instance (Scotty :> es, ScottyIO :> es) => MonadUnliftIO (ScottyEff es) where
    withRunInIO inner =
        lift $ do
            opts <- scottyOpts
            let strategy = opts ^. #unliftStrategy
            unsafeEff $ \env ->
                unliftIOFromStrategy env strategy $ \unlift ->
                    inner (unlift . runInner)

type ScottyE es a = ScottyT (ScottyEff (Scotty : ScottyIO : es)) a

unliftIOFromStrategy
    :: (HasCallStack)
    => Env es
    -> UnliftStrategy
    -> ((forall r. Eff es r -> IO r) -> IO a)
    -> IO a
unliftIOFromStrategy env strategy cont = case strategy of
    SeqUnlift      -> seqUnliftIO env cont
    SeqForkUnlift  -> seqForkUnliftIO env cont
    ConcUnlift p b -> concUnliftIO env p b cont

Now, my issue is that I get the following error:

C:\Users\jaeca\Documents\haskell\testrio\src\KubeController\Web.hs:78:14: error: [GHC-25897]
    * Could not deduce `localEs ~ es'
      from the context: a1 ~ (Eff localEs Response -> IO Response)
        bound by a pattern with constructor:
                   UnliftResponse :: forall (m :: * -> *).
                                     Scotty m (m Response -> IO Response),
                 in a \case alternative
        at C:\Users\jaeca\Documents\haskell\testrio\src\KubeController\Web.hs:74:5-18
      Expected: a1
        Actual: Eff es Response -> IO Response
      `localEs' is a rigid type variable bound by
        a type expected by the context:
          Effectful.Internal.Monad.EffectHandler Scotty es
        at C:\Users\jaeca\Documents\haskell\testrio\src\KubeController\Web.hs:(72,32)-(78,17)
      `es' is a rigid type variable bound by
        the type signature for:
          runWithOpts :: forall (es :: [Effect]) a.
                         (HasCallStack, ScottyIO :> es) =>
                         ScottyOpts -> Eff (Scotty : es) a -> Eff es a
        at C:\Users\jaeca\Documents\haskell\testrio\src\KubeController\Web.hs:70:1-94
    * In the first argument of `pure', namely `fun2'
      In a stmt of a 'do' block: pure fun2
      In the expression:
        do fun <- borrow defaultUnliftResponse
           let fun2 = fun . raise
           pure fun2
    * Relevant bindings include
        fun2 :: Eff es Response -> IO Response
          (bound at C:\Users\jaeca\Documents\haskell\testrio\src\KubeController\Web.hs:77:13)
        fun :: Eff (Scotty : es) Response -> IO Response
          (bound at C:\Users\jaeca\Documents\haskell\testrio\src\KubeController\Web.hs:76:9)
        borrow :: forall r.
                  Eff ('[Scotty] Effectful.Internal.Effect.++ es) r -> Eff es r
          (bound at C:\Users\jaeca\Documents\haskell\testrio\src\KubeController\Web.hs:74:56)
        env :: Effectful.Internal.Monad.LocalEnv localEs es
          (bound at C:\Users\jaeca\Documents\haskell\testrio\src\KubeController\Web.hs:72:33)
        runWithOpts :: ScottyOpts -> Eff (Scotty : es) a -> Eff es a
          (bound at C:\Users\jaeca\Documents\haskell\testrio\src\KubeController\Web.hs:72:1)
   |
78 |         pure fun2
   |

So I'm calling the borrow function handed out by localSeqBorrow in order to cleave off the Scotty effect from the outer monad from defaultUnliftResponse (i.e. Eff (Scotty : es) (Eff (Scotty : es) Response -> IO Response) to Eff es (Eff (Scotty : es) Response -> IO Response) where ScottyIO :> es), and after I'm trying to compose with raise such that I can return the Eff es Response -> IO Response function required to match the signature of UnliftResponse :: m (m Response -> IO Response).

I'm simply struggling to wrap my head around how this equality constraint localEs ~ es has even been introduced. Is it an issue with calling raise? I can't see any other way to return the required function, since borrow :: forall a. Eff ('[Scotty] ++ es) a -> Eff es a (wrong way round) and calling runWithOpts :: ScottyOpts -> Eff (Scotty : es) -> Eff es also wouldn't work (again, wrong way round).

In the end the point is to be able to write something like this to turn a scotty router into a WAI application to then run manually on WARP or something:

httpApp :: (Reader String :> es, Scotty :> es, ScottyIO :> es) => ScottyInner (Eff es) Application
httpApp = do
    unliftResponse <- lift (send UnliftResponse)
    scottyAppT options (unliftResponse . runInner) httpRoutes
    where
        httpRoutes :: (Reader String :> es, Scotty :> es, ScottyIO :> es) => ScottyT (ScottyInner (Eff es)) ()
        httpRoutes =
            get "/:word" $ do
                _ignore :: String <- pathParam "word"
                html "Hello World"
        options = undefined -- TODO

---
[scotty-0.22]:

module Web.Scotty.Trans (...) where

[...]

scottyAppT :: (Monad m, Monad n)	 
=> Options	 
-> (m Response -> IO Response)	
-> ScottyT m ()	 
-> n Application

Sorry that my explanation has been super long and I suppose it's a somewhat contrived example since for all intents and purposes I'll probably only be switching out the relevant UnliftStrategy for unlifting the Response actions, but in a sense this has become a bit of a learning example for how to use the library more generally, so any help decoding this error would be greatly appreciated.

[jaecam]

On taking another look at this, I think it's very possible I've simply "switched" the directions of borrow/lend / lift/unlift in my head... I've just tried rewriting it as:

[...]
UnliftResponse -> 
        localSeqUnlift env $ \unlift ->
            localSeqLend @'[ScottyIO] env $ \lend -> 
                fmap (. raise) . unlift . lend $ defaultUnliftResponse

and this compiles. So it seems I just totally confused myself and in fact was supposed to transform the input parameter of the unlift response function to Eff localEs Response rather than Eff es Response (localEs and es in the handler context), hence the type error, which in hindsight should have been a very obvious explanation for the localEs ~ es constraint 🤦‍♂️.

Just wanted to say thanks for writing this brilliant library!