Developing Web Apps with Haskell and Yesod, Second Edition (2015)

Part II. Advanced

Chapter 13. Yesod's Monads

As you’ve progressed through this book so far, a number of monads have appeared: Handler, Widget, and YesodDB (for Persistent). As with most monads, each one provides some specific functionality: Handler gives access to the request and allows you to send responses; a Widgetcontains HTML, CSS, and JavaScript; and YesodDB lets you make database queries. In model-view-controller (MVC) terms, we could consider YesodDB to be the model, Widget to be the view, and Handler to be the controller.

So far, we’ve presented some very straightforward ways to use these monads: your main handler will run in Handler, using runDB to execute a YesodDB query and defaultLayout to return a Widget, which in turn was created by calls to toWidget.

However, if we have a deeper understanding of these types, we can achieve some fancier results.

Monad Transformers

Monads are like onions. Monads are not like cakes.

Variation on a quote from Shrek

Before we get into the heart of Yesod’s monads, we need to understand a bit about monad transformers. (If you already know all about monad transformers, you can likely skip this section.) Different monads provide different functionality: Reader allows read-only access to some piece of data throughout a computation, Error allows you to short-circuit computations, and so on.

Oftentimes, however, you’ll want to be able to combine a few of these features together. After all, why not have a computation with read-only access to some settings variable, that could error out at any time? One approach to this would be to write a new monad like ReaderError, but this has the obvious downside of exponential complexity: you’ll need to write a new monad for every single possible combination.

Instead, we have monad transformers. For example, in addition to Reader, we have ReaderT, which adds reader functionality to any other monad. So, conceptually, we could represent our ReaderError as follows:

typeReaderError=ReaderTError

In order to access our settings variable, we can use the ask function. But what about short-circuiting a computation? We’d like to use throwError, but that won’t exactly work. Instead, we need to lift our call into the next monad up. In other words:

throwError ::errValue ->Error

lift . throwError ::errValue ->ReaderTError

There are a few things you should pick up here:

§ A transformer can be used to add functionality to an existing monad.

§ A transformer must always wrap around an existing monad.

§ The functionality available in a wrapped monad will be dependent not only on the monad transformer, but also on the inner monad that is being wrapped.

A great example of that last point is the IO monad. No matter how many layers of transformers you have around an IO, there’s still an IO at the core, meaning you can perform I/O in any of these monad transformer stacks. You’ll often see code that looks like liftIO $ putStrLn "Hello There!".

The Three Transformers

We’ve already discussed two of our transformers: Handler and Widget. Remember that these are each application-specific synonyms for the more generic HandlerT and WidgetT. Each of those transformers takes two type parameters: your foundation data type, and a base monad. The most commonly used base monad is IO.

NOTE

In earlier versions of Yesod, Handler and Widget were far more magical and scary. Since version 1.2, things are much simplified. So, if you remember reading some scary stuff about fake transformers and subsite parameters, rest assured: you haven’t gone crazy, things have actually changed a bit. The story with Persistent is likewise much simpler.

In Persistent, we have a typeclass called PersistStore. This typeclass defines all of the primitive operations you can perform on a database, like get. There are instances of this typeclass for each database backend supported by Persistent. For example, for SQL databases, there is a data type called SqlBackend. We then use a standard ReaderT transformer to provide that SqlBackend value to all of our operations. This means that we can run a SQL database with any underlying monad that is an instance of MonadIO. The takeaway here is that we can layer our Persistent transformer on top of Handler or Widget.

In order to make it simpler to refer to the relevant Persistent transformer, the yesod-persistent package defines the YesodPersistBackend associated type. For example, if I have a site called MyApp and it uses SQL, I would define something like type instance YesodPersistBackend MyApp = SqlBackend. And for more convenience, we have a type synonym called YesodDB, which is defined as:

typeYesodDB site =ReaderT (YesodPersistBackend site) (HandlerT site IO)

Our database actions will then have types that look like YesodDB MyApp SomeResult. In order to run these, we can use the standard Persistent unwrap functions (like runSqlPool) to run the action and get back a normal Handler. To automate this, we provide the runDB function.Putting it all together, we can now run database actions inside our handlers.

Most of the time in Yesod code, and especially thus far in this book, widgets have been treated as actionless containers that simply combine HTML, CSS, and JavaScript. But in reality, a Widget can do anything that a Handler can do, by using the handlerToWidget function. So, for example, you can run database queries inside a Widget by using something like handlerToWidget . runDB.

Example: Database-Driven Navbar

Let’s put some of this new knowledge into action. We want to create a Widget that generates its output based on the contents of the database. Previously, our approach would have been to load up the data in a Handler, and then pass that data into a Widget. Now, we’ll do the loading of data in the Widget itself. This is a boon for modularity, as this Widget can be used in any Handler we want, without any need to pass in the database contents:

{-# LANGUAGE FlexibleContexts #-}

{-# LANGUAGE GADTs #-}

{-# LANGUAGE GeneralizedNewtypeDeriving #-}

{-# LANGUAGE MultiParamTypeClasses #-}

{-# LANGUAGE OverloadedStrings #-}

{-# LANGUAGE QuasiQuotes #-}

{-# LANGUAGE TemplateHaskell #-}

{-# LANGUAGE TypeFamilies #-}

import Control.Monad.Logger (runNoLoggingT)

import Data.Text (Text)

import Data.Time

import Database.Persist.Sqlite

import Yesod

share [mkPersist sqlSettings, mkMigrate "migrateAll"] [persistLowerCase|

Link

title Text

url Text

added UTCTime

dataApp=AppConnectionPool

mkYesod "App" [parseRoutes|

/ HomeR GET

/add-link AddLinkRPOST

instanceYesodApp

instanceRenderMessageAppFormMessagewhere

renderMessage __=defaultFormMessage

instanceYesodPersistAppwhere

typeYesodPersistBackendApp=SqlBackend

runDB db =do

App pool <-getYesod

runSqlPool db pool

getHomeR ::HandlerHtml

getHomeR =defaultLayout

[whamlet|

<p>

Add a new link to

titled

<h2>Existing links

^{existingLinks}

existingLinks ::Widget

existingLinks =do

links <-handlerToWidget $ runDB $ selectList [] [LimitTo 5, DescLinkAdded]

[whamlet|

<ul>

$forall Entity_ link <-links

<li>

<a href=#{linkUrl link}>#{linkTitle link}

postAddLinkR ::Handler ()

postAddLinkR =do

url <-runInputPost $ ireq urlField "url"

title <-runInputPost $ ireq textField "title"

now <-liftIO getCurrentTime

runDB $ insert $ Link title url now

setMessage "Link added"

redirect HomeR

main ::IO ()

main =runNoLoggingT $ withSqlitePool "links.db3" 10 $ \pool ->liftIO $ do

runSqlPersistMPool (runMigration migrateAll) pool

warp 3000 $ App pool

Pay attention in particular to the existingLinks function. Notice how all we needed to do was apply handlerToWidget . runDB to a normal database action. And from within getHomeR, we treated existingLinks like any ordinary Widget, with no special parameters at all.Figure 13-1 shows the output of this app.

wahy 1301

Figure 13-1. Screenshot of the navbar

Example: Request Information

Likewise, you can get request information inside a Widget. Here we can determine the sort order of a list based on a GET parameter:

{-# LANGUAGE MultiParamTypeClasses #-}

{-# LANGUAGE OverloadedStrings #-}

{-# LANGUAGE QuasiQuotes #-}

{-# LANGUAGE TemplateHaskell #-}

{-# LANGUAGE TypeFamilies #-}

import Data.List (sortBy)

import Data.Ord (comparing)

import Data.Text (Text)

import Yesod

dataPerson=Person

{ personName ::Text

, personAge ::Int

}

people :: [Person]

people =

[ Person "Miriam" 25

, Person "Eliezer" 3

, Person "Michael" 26

, Person "Gavriella" 1

]

dataApp=App

mkYesod "App" [parseRoutes|

/ HomeRGET

instanceYesodApp

instanceRenderMessageAppFormMessagewhere

renderMessage __=defaultFormMessage

getHomeR ::HandlerHtml

getHomeR =defaultLayout

[whamlet|

<p>

<a href="?sort=name">Sort by name

<a href="?sort=age">Sort by age

<a href="?">No sort

^{showPeople}

showPeople ::Widget

showPeople =do

msort <-runInputGet $ iopt textField "sort"

let people' =

case msort of

Just "name" ->sortBy (comparing personName) people

Just "age" ->sortBy (comparing personAge) people

_ -> people

[whamlet|

<dl>

$forall person <-people'

<dt>#{personName person}

<dd>#{show $ personAge person}

main ::IO ()

main =warp 3000 App

Notice that in this case, we didn’t even have to call handlerToWidget. The reason is that a number of the functions included in Yesod automatically work for both Handler and Widget, by means of the MonadHandler typeclass. In fact, MonadHandler will allow these functions to be “autolifted” through many common monad transformers.

But if you want to, you can wrap up the call to runInputGet using handlerToWidget, and everything will work the same.

Performance and Error Messages

At this point, you may be just a bit confused. As I already mentioned, the Widget synonym uses IO as its base monad, not Handler. So how can Widget perform Handler actions? And why not just make Widget a transformer on top of Handler, and then use lift instead of this specialhandlerToWidget? And finally, I mentioned that Widget and Handler were both instances of MonadResource. If you’re familiar with MonadResource, you may be wondering why ResourceT doesn’t appear in the monad transformer stack.

NOTE

You can consider this section extra credit. It gets into some of the design motivation behind Yesod, which isn’t necessary for usage of Yesod.

The fact of the matter is that there’s a much simpler (in terms of implementation) approach we could take for all of these monad transformers. Handler could be a transformer on top of ResourceT IO instead of just IO, which would be a bit more accurate. And Widget could be layered on top of Handler. The end result would look something like this:

typeHandler=HandlerTApp (ResourceTIO)

typeWidget =WidgetT App (HandlerTApp (ResourceTIO))

Doesn’t look too bad, especially considering you mostly deal with the friendlier type synonyms instead of directly with the transformer types. The problem is that any time those underlying transformers leak out, these larger type signatures can be incredibly confusing. And the most common time for them to leak out is in error messages, when you’re probably already pretty confused! (Another time is when working on subsites, which happens to be confusing too.)

One other concern is that each monad transformer layer does add some amount of performance penalty. This will probably be negligible compared to the I/O you’ll be performing, but the overhead is there.

So, instead of having properly layered transformers, we flatten out each of HandlerT and WidgetT into a one-level transformer. Here’s a high-level overview of the approach we use:

§ HandlerT is really just a ReaderT monad. (We give it a different name to make error messages clearer.) This is a reader for the HandlerData type, which contains request information and some other immutable contents.

§ In addition, HandlerData holds an IORef to a GHState (badly named for historical reasons), which holds some data that can be mutated during the course of a handler (e.g., session variables). The reason we use an IORef instead of a StateT kind of approach is that IORef will maintain the mutated state even if a runtime exception is thrown.

§ The ResourceT monad transformer is essentially a ReaderT holding onto an IORef. This IORef contains the information on all cleanup actions that must be performed. (This is called InternalState.) Instead of having a separate transformer layer to hold onto that reference, we hold onto the reference ourselves in HandlerData. (And yes, the reason for an IORef here is also for runtime exceptions.)

§ A WidgetT is essentially just a WriterT on top of everything that a HandlerT does. But because HandlerT is just a ReaderT, we can easily compress the two aspects into a single transformer, which looks something like newtype WidgetT site m a = WidgetT (HandlerData -> m (a, WidgetData)).

The definitions of HandlerT and WidgetT in Yesod.Core.Types are useful if you want to better understand this.

Adding a New Monad Transformer

At times, you’ll want to add your own monad transformer in part of your application. As a motivating example, let’s consider the monadcryptorandom package from Hackage, which defines both a MonadCRandom typeclass for monads that allow generating cryptographically secure random values, and CRandT as a concrete instance of that typeclass. Say we want to write some code that generates a random Bytestring such as the following:

importControl.Monad.CryptoRandom

importData.ByteString.Base16 (encode)

importData.Text.Encoding (decodeUtf8)

getHomeR =do

randomBS <-getBytes 128

defaultLayout

[whamlet|

<p>Here's some random data: #{decodeUtf8 $ encode randomBS}

However, this results in an error message along the lines of:

No instance for (MonadCRandom e0 (HandlerT App IO))

arising from a use of 'getBytes'

In a stmt of a 'do' block: randomBS <- getBytes 128

How do we get such an instance? One approach is to simply use the CRandT monad transformer when we call getBytes. A complete example of doing so would be:

{-# LANGUAGE OverloadedStrings, QuasiQuotes, TemplateHaskell, TypeFamilies #-}

importYesod

importCrypto.Random (SystemRandom, newGenIO)

importControl.Monad.CryptoRandom

importData.ByteString.Base16 (encode)

importData.Text.Encoding (decodeUtf8)

dataApp=App

mkYesod "App" [parseRoutes|

/ HomeRGET

instanceYesodApp

getHomeR ::HandlerHtml

getHomeR =do

gen <-liftIO newGenIO

eres <-evalCRandT (getBytes 16) (gen ::SystemRandom)

randomBS <-

case eres of

Left e ->error $ show (e ::GenError)

Right gen ->return gen

defaultLayout

[whamlet|

<p>Here's some random data: #{decodeUtf8 $ encode randomBS}

main ::IO ()

main =warp 3000 App

Note that what we’re doing is layering the CRandT transformer on top of the HandlerT transformer. It does not work to do things the other way around: Yesod itself would ultimately have to unwrap the CRandT transformer, and it has no knowledge of how to do so. Notice that this is the same approach we take with Persistent: its transformer goes on top of HandlerT.

But there are two downsides to this approach:

§ It requires you to jump into this alternative monad each time you want to work with random values.

§ It’s inefficient: you need to create a new random seed each time you enter this other monad.

The second point could be worked around by storing the random seed in the foundation data type, in a mutable reference like an IORef, and then atomically sampling it each time we enter the CRandT transformer. But we can even go a step further, and use this trick to make our Handlermonad itself an instance of MonadCRandom! Let’s look at the code, which is in fact a bit involved:

{-# LANGUAGE FlexibleInstances #-}

{-# LANGUAGE MultiParamTypeClasses #-}

{-# LANGUAGE OverloadedStrings #-}

{-# LANGUAGE QuasiQuotes #-}

{-# LANGUAGE TemplateHaskell #-}

{-# LANGUAGE TypeFamilies #-}

{-# LANGUAGE TypeSynonymInstances #-}

import Control.Monad (join)

import Control.Monad.Catch (catch, throwM)

import Control.Monad.CryptoRandom

import Control.Monad.Error.Class (MonadError (..))

import Crypto.Random (SystemRandom, newGenIO)

import Data.ByteString.Base16 (encode)

import Data.IORef

import Data.Text.Encoding (decodeUtf8)

import Yesod

dataApp=App

{ randGen ::IORefSystemRandom

}

mkYesod "App" [parseRoutes|

/ HomeRGET

instanceYesodApp

getHomeR ::HandlerHtml

getHomeR =do

randomBS <-getBytes 16

defaultLayout

[whamlet|

<p>Here's some random data: #{decodeUtf8 $ encode randomBS}

instanceMonadErrorGenErrorHandlerwhere

throwError =throwM

catchError =catch

instanceMonadCRandomGenErrorHandlerwhere

getCRandom =wrap crandom

{-# INLINE getCRandom #-}

getBytes i =wrap (genBytes i)

{-# INLINE getBytes #-}

getBytesWithEntropy i e =wrap (genBytesWithEntropy i e)

{-# INLINE getBytesWithEntropy #-}

doReseed bs =do

genRef <-fmap randGen getYesod

join $ liftIO $ atomicModifyIORef genRef $ \gen ->

case reseed bs gen of

Left e -> (gen, throwM e)

Right gen' -> (gen', return ())

{-# INLINE doReseed #-}

wrap :: (SystemRandom->EitherGenError (a, SystemRandom)) ->Handler a

wrap f =do

genRef <-fmap randGen getYesod

join $ liftIO $ atomicModifyIORef genRef $ \gen ->

case f gen of

Left e -> (gen, throwM e)

Right (x, gen') -> (gen', return x)

main ::IO ()

main =do

gen <-newGenIO

genRef <-newIORef gen

warp 3000 App

{ randGen =genRef

}

This really comes down to a few different concepts:

1. We modify the App data type to have a field for an IORef SystemRandom.

2. Similarly, we modify the main function to generate an IORef SystemRandom.

3. Our getHomeR function has become a lot simpler: we can now simply call getBytes without playing with transformers.

4. However, we have gained some complexity in needing a MonadCRandom instance. This is a book about Yesod, not monadcryptorandom, so I’m not going to go into details on this instance, but I encourage you to inspect it and, if you’re interested, compare it to the instance forCRandT.

Hopefully, this helps get across an important point: the power of the HandlerT transformer. As it provides you with a readable environment, you’re able to re-create a StateT transformer by relying on mutable references. In fact, if you rely on the underlying IO monad for runtime exceptions, you can implement most cases of ReaderT, WriterT, StateT, and ErrorT with this abstraction.

Summary

If you completely ignore this chapter, you’ll still be able to use Yesod to great benefit. The advantage of understanding how Yesod’s monads interact that it enables you to produce cleaner, more modular code. Being able to perform arbitrary actions in a Widget can be a powerful tool, and understanding how Persistent and your Handler code interact can help you make more informed design decisions in your app.