Framework agnostic HTTP endpoint go handlers

February 11, 2019

I'm making a simple REST service in go. Like most, it's basically just a CRUD service. This project is my way of learning go, and also of learning serverless application development (on AWS specifically). I want each endpoint to be its own serverless function, so that they can each scale independently, and also so I'll never have to worry about lambda scaling limits 🤞.

First Attempt (The not so great way)

The problem seemed straightforward enough. With the above in mind I ended up with a dozen or so files looking something like this:

// imports and stuff removed

func handler(request events.APIGatewayProxyRequest) 
    (events.APIGatewayProxyResponse, error)
{
    db.Error = nil
    user, cerr := util.CreateReadOrUpdateAuthenticatedUser(db, &request)
    if cerr != nil {
        return util.CreateErrorResponse(cerr), nil
    }

    boardID, ok := request.PathParameters["boardId"]
    if !ok {
        return util.CreateErrorResponse(errorcodes.BoardIDRequired), nil
    }

    id, err := uuid.Parse(boardID)
    if err != nil {
        log.Error(err)
        return util.CreateErrorResponse(errorcodes.BoardIDMustBeUUID), nil
    }

    /***
     *** Just about everything above here is not specific to this function
     ***/

    board := model.Board{}
    err = db
        .Where(&model.Board{Model: model.Model{ID: id}, UserID: user.ID})
        .First(&board)
        .Error
    if err != nil {
        log.Error(err)
        return util.CreateErrorResponse(errorcodes.NotFound), nil
    }

    /***
     *** And just about everything below here is not specific to this function
     ***/

    boardJSON, err := json.Marshal(board)
    if err != nil {
        log.Error(err)
        return util.CreateErrorResponse(errorcodes.JSONMarshalFailure), nil
    }

    return util.CreateResponse(events.APIGatewayProxyResponse{
        StatusCode: 200,
        Body:       string(boardJSON),
    }), nil
}

func init() {
    db, log, cerr = util.Init()
    if cerr != nil {
        panic(util.CreateErrorResponse(cerr))
    }
}

func main() {
    lambda.Start(handler)
}

There were several problems with this approach:

Even putting aside the verbosity from the error checking, there was a ton of boilerplate and replicated code across my handlers. In the above example, there are only a couple lines that do the work that is actually unique to this endpoint.
Each handler got its own file and main package. This resulted in an overly complicated build setup, and a complicated directory tree.
This was a pain to test. Constructing the objects that lambda handlers expect (correctly) was a tedious process of reading documentation and experimenting with live lambdas. Even after this, it was hard to be sure I was constructing the input and validating the output the same way that ApiGateway would.
I couldn't easily run the endpoints locally for rapid development alongside my frontend. There are ways to run go lambdas locally, but not easily. I had chosen serverless as my "Infrastructure as Code" tool (for various reasons beyond the scope of this post) and while there is serverless-offline, it only works for Node.js functions. AWS SAM has a convenient sam local start-api command, but I didn't want to also maintain a separate yaml file for SAM in addition to my serverless.yml (the approach I ended up taking also requires extra maintenance, but it comes with more flexibility).

So how can we address those problems?

Step 1: Just one handler with knowledge of all functions

We can greatly simplify the build step and reduce replicated code by making it so that there is only one binary to compile, as opposed to one for each endpoint like above. It looks like this:

// imports and stuff removed

func handler(request events.APIGatewayProxyRequest)
    (events.APIGatewayProxyResponse, error)
{
    db.Error = nil
    ctx := CreateLambdaBaseContext(request, db, log)
    coreHandler(core.CreateContext(ctx))
    return ctx.Response, nil
}

func init() {
    db, log, cerr = util.Init()
    if cerr != nil {
        panic(cerr)
    }
    coreHandler = CoreHandlerFactory(os.Getenv("MYPROJECT_FUNCTION_NAME"))
}

func main() {
    lambda.Start(handler)
}

Notice how an environment variable is passed into a factory function in init which returns the implementation of the endpoint. Then in handler we call that implementation with the context of this request (which is Step 2 below). That environment variable is set in my serverless.yml for each endpoint to a unique and sensible string that the factory is able to map to the desired function.

To be clear, this is not the same as creating one lambda that handles all requests by routing internally. This is one go handler that is uploaded as many different lambdas, each with a different environment, so that they each do something different.

Step 2: Each function uses an abstracted Context

The body of the function that does the work now looks something like this:

func GetBoard(ctx Context) {
    board, cerr := ctx.GetBoardFromPath()
    if cerr != nil {
        ctx.MyProjectError(cerr)
        return
    }

    ctx.JSON(http.StatusOK, board)
}

A lot simpler than the mess at the top of this post right??

The straightforwardness of this code is achieved by a two layered Context. The first layer is what provides the JSON response function, as well as request related functions like GetHeader and GetPathParameter. This is common among (and is inspired by) popular go web frameworks like gin.

The second layer provides the reusable functions like GetBoardFromPath which looks at the path param, validates the boardId and gets the corresponding board from the database.

What do I mean by two layers? and why?

The first layer is just a small interface that can be easily implemented for any framework, or for lambda, or for no framework. This is the entire interface (so far):

// BaseContext is the interface that should be implemented by servers
type BaseContext interface {
    GetHeader(key string) string
    GetBody() string
    GetPathParameter(key string) string
    GetQueryParameter(key string) string

    JSON(code int, obj interface{})
}

And this is a piece of the lambda implementation.

type LambdaBaseContext struct {
    Request  events.APIGatewayProxyRequest
    Response events.APIGatewayProxyResponse
}

func (l *LambdaBaseContext) GetBody() string {
    return l.Request.Body
}

func (l *LambdaBaseContext) JSON(code int, obj interface{}) {
    var body string
    status := code
    jsonBody, err := json.Marshal(obj)
    if err != nil {
        log.WithField("obj", obj).Warn(err)
        body = "{ \"error\": \"RESPONSE_TYPE_IS_INVALID_JSON\" }"
        status = 500
    } else {
        body = string(jsonBody)
    }
    l.Response.Body = body
    l.Response.StatusCode = status
}

Aside from some error handling, pretty simple right?

The second layer is a wrapper around the BaseContext called just Context. Context uses the basic functions that the BaseContext provides, along with access to the database, to provide convenient reusable functions to the handlers.

To show what I mean, here is an look at the Context type and the GetBoardFromPath implementation.

type Context struct {
    base  BaseContext
}

func (c *Context) GetUUIDPathParameter(key string)
    (uuid.UUID, *errorcodes.MyProjectError)
{
    /*** This is a BaseContext method ***/
    idString := c.base.GetPathParameter(key)

    id, err := uuid.Parse(idString)
    if idString == "00000000-0000-0000-0000-000000000000" {
        err = errors.New("uuid cannot be 0")
    }
    if err != nil {
        log.Error(err)
        cerr := errorcodes.PathParamMustBeUUID
        cerr.Context["param"] = key
        return uuid.New(), &cerr
    }
    return id, nil
}

func (c *Context) GetBoardFromPath() (*model.Board, *errorcodes.MyProjectError) {
    user, cerr := c.GetCurrentUser()
    if cerr != nil {
        return nil, cerr
    }
    boardID, cerr := c.GetUUIDPathParameter("boardId")
    if cerr != nil {
        return nil, cerr
    }
    board := model.Board{}
    err := c
        .GetDB()
        .Where(&model.Board{Model: model.Model{ID: boardID}, UserID: user.ID})
        .First(&board)
        .Error
    if err != nil {
        log.Error(err)
        return nil, &errorcodes.BoardNotFound
    }
    return &board, nil
}

Notice how GetBoardFromPath relies on GetUUIDPathParameter which is just a layer of functionality over BaseContext's GetPathParameter. Similarly, GetCurrentUser (implementation not shown) just uses BaseContext's GetHeader.

All of the above results in more modularized, reusable, and testable code than I had before. Also...

The best part

The functionality of my service is only loosely coupled to the framework providing the context, so all it takes to completely switch frameworks is to implement BaseContext for the new framework!

I was able to setup a gin version of my service for running locally very easily by implementing the couple functions of BaseContext and setting up the router.

That's it!

Phew, that was a lot of dense information about a very specific implementation I recently did. I enjoy reading about others' decisions about software design, so hopefully this was enjoyable or helpful to you.

Thanks for reading!