Dusted Codes

Programming adventures

Stop doing security (yourself)

Security has a very special place in software engineering. It is very different than other disciplines, because it is a problem space which cannot be approached with the same mindset as other problem spaces in our trade.

What makes security stand out so much is that it doesn't follow the same principles as the rest of software development. There is no different set of opinions. There is only one right opinion and the rest. There is not many ways how to achieve a secure something. There's only one very narrow, very particular and very unforgiving way of how to achieve a secure system (based on latest knowledge) and if you don't follow these steps very precisely then you'll end up with something which has more holes than a swiss cheese.

The other difference (between security and the rest of software engineering) is this you cannot delegate that competency to a single person. Your organisation might have one software architect, maybe one principal developer, maybe one Java or one database specialist, but if you have only one security expert then you have a problem. Security is such a complex topic, that even the most knowledgable person on the planet will not know everything that is important.

Unfortunately, by nature, a system can never be assumed (or claimed) to be 100% secure, because there is no physics on this planet which could back this up. Therefore, the best and really only way which an organisation can use to its defence is to get their security model in front of as many eyes as possible. This is not an opinion, but a fact.

The most secure encryption is not the one which has been developed by the most knowledgable person, but the one which has been reviewed, hacked and revised by the most people. There is a reason why all sophisticated cryptography algorithms are open to the public. Cryptographers deliberately want their work to be exposed to the largest possible audience and have it validated by them, because even they don't know how secure it is until it's been tested. This also explains why we have organisations specialising in penetration testing or why big organisations run public bug bounty programs.

With that in mind, my universal recommendation to any organisation which is not deeply involved in the InfoSec community is always to not do security by yourself. This can really not be stressed enough, but if you are not an industry leading expert in security, then don't even think about implementing your own password hasing algorithm, don't secure your API with a custom built authentication scheme and please don't build your own identity provider.

There is three reasons why:

  1. You'll get it wrong
  2. You don't have to do it, because others have already done it for you
  3. You will get it wrong

If this little pep talk hasn't been convincing enough yet then there is another crucial reason why you shouldn't do security yourself: Security is out of your control and you'll probably live better by not having to deal with it.

Imagine your development team has made the choice to use Amazon's DynamoDb as their main data persistence layer. Two years down the line Amazon releases a new, improved and more state-of-the-art NoSQL database, but the development team doesn't find out about it until a few months later when one of the team members hears about it at a conference. After the conference the team sits together and decides to migrate to the new NoSQL database, but they won't do it for another six months, because they simply don't have the time and resources right now. Half of the team is on summer holiday, one person is sick, the development manager is on her honeymoon (so they lack formal approval anyway) and the person who originally came up with the current database architecture has left the company last year. It will take some time to come up with a good migration strategy and roll out in an efficient way. Luckily that is not an issue, because the current NoSQL database still works perfectly well and there's no immediate pressure to make a hasty change. The team is not bothered, nobody is stressing out and everyone enjoys their holidays before the team tackles the project later in the year.

Now imagine the same scenario but replace "database" with "identity provider" and replace "state-of-the-art" with "not full of security holes" and the whole story looks very different.

If someone drops a zero day vulnerability which affects your system then time is against you before you have even realised it. At this point you are already on the losing end and the problem which you're trying to tackle is not prevention, but damage control. The longer it will take you to fix, update and roll out a new version of your affected software the more likely you are going to be hit hard by this situation.

It will add very little consolation knowing that this whole disaster hasn't even been your fault. You might just have used a cryptographic implementation which was considred secure yesterday, but today you woke up to the news that some hackers have published a white paper on how to crack it in less than an hour. This was not a targeted attack against you, your business or your customers. This was simply a new discovery which has been dumped on the security community without much thought and now you and half of the world is affected by it. Before you think this type of stuff doesn't happen, let me quickly remind you of incidents like Heartbleed, Cloudbleed, Meltdown, Spectre, WannaCry, and so on.

In order to survive such a scenario you'll want to have certain things in place:

Unless your business meets all of these requirements it might be flatout irresponsible to even think of writing your own custom security software if you don't have the means to deal with issues that come with it.

Now my main point is not to scare you of writing your own software, but to create some awareness that in the context of security you are probably better off by deferring these responsibilities to a third party which is a specialist in this field. Someone who lives and breathes security every day is much better equipped to deal with all the unkowns which we're confronted with every day.

Comments

Giraffe 1.1.0 - More routing handlers, better model binding and brand new model validation API

Last week I announced the release of Giraffe 1.0.0, which (apart from some initial confusion around the transition to TaskBuilder.fs) went mostly smoothly. However, if you have thought that I would be chilling out much since then, then you'll probably be disappointed to hear that today I've released another version of Giraffe with more exciting features and minor bug fixes.

The release of Giraffe 1.1.0 is mainly focused around improving Giraffe's routing API, making model binding more functional and adding a new model validation API.

Some of these features address some long requested functionality, so let's not waste any more time and get straight down to it.

Routes with trailing slashes

Often I've been asked how to make Giraffe treat a route with a trailing slash equal to the same route without a trailing slash:

https://example.org/foo/bar
https://example.org/foo/bar/

According to the technical specification a route with a trailing slash is not the same as a route without it. A web server might want to serve a different response for each route and therefore Giraffe (rightfully) treats them differently.

However, it is not uncommon that a web application chooses to not distinguish between two routes with and without a trailing slash and as such it wasn't a surprise when I received multiple bug reports for Giraffe not doing this by default.

Before version 1.1.0 one would have had to specify two individual routes in order to make it work:

let webApp =
    choose [
        route "/foo"  >=> text "Foo"
        route "/foo/" >=> text "Foo"
    ]

Giraffe version 1.1.0 offers a new routing handler called routex which is similar to route except that it allows a user to specify Regex in the route declaration.

This makes it possible to define routes with more complex rules such as allowing an optional trailing slash:

let webApp =
    choose [
        routex "/foo(/?)" >=> text "Foo"
    ]

The (/?) regex pattern denotes that there can be exactly zero or one slash after /foo.

With the help of routex and routeCix (the case insensitive version of routex) one can explicitly allow trailing slashes (or other non-standard behaviour) in a single route declaration.

Parameterised sub routes

Another request which I have seen on several occasions was a parameterised version of the subRoute http handler.

Up until Giraffe 1.0.0 there was only a routef and a subRoute http handler, but not a combination of both.

Imagine you have a localised application which requires a language parameter at the beginning of each route:

https://example.org/en-gb/foo
https://example.org/de-at/bar
etc.

In previous versions of Giraffe one could have used routef to parse the parameter and pass it into another HttpHandler function:

let fooHandler (lang : string) =
    sprintf "You have chosen the language %s." lang
    |> text

let webApp =
    choose [
        routef "/%s/foo" fooHandler
    ]

This was all good up until someone needed to make use of something like routeStartsWith or subRoute to introduce additional validation/authentication before invoking the localised routes:

let webApp =
    choose [
        // Doesn't require authentication
        routef "/%s/foo" fooHandler
        routef "/%s/bar" barHandler

        // Requires authentication
        requiresAuth >=> choose [
            routef "/%s/user/%s/foo" userFooHandler
            routef "/%s/user/%s/bar" userBarHandler
        ]
    ]

The problem with above code is that the routing pipeline will always check if a user is authenticated (and potentially return an error response) before even knowing if all subsequent routes require it.

The workaround was to move the authentication check into each of the individual handlers, namely the userFooHandler and the userBarHandler in this instance.

A more elegant way would have been to specify the authentication handler only one time before declaring all protected routes in a single group. Normally the subRoute http handler would make this possible, but not if routes have parameterised arguments at the beginning of their paths.

The new subRoutef http handler solves this issue now:


let webApp =
    choose [
        // Doesn't require authentication
        routef "/%s/foo" fooHandler
        routef "/%s/bar" barHandler

        // Requires authentication
        subRoutef "%s-%s/user" (
            fun (lang, dialect) ->
                // At this point it is already
                // established that the path
                // is a protected user route:
                requiresAuth
                >=> choose [
                    routef "/%s/foo" (userFooHandler lang dialect)
                    routef "/%s/bar" (userBarHandler lang dialect)
                ]
        )
    ]

The subRoutef http handler can pre-parse parts of a route and group a collection of cohesive routes in one go.

Improved model binding and model validation

The other big improvements in Giraffe 1.1.0 were all around model binding and model validation.

The best way to explain the new model binding and validation API is by looking at how Giraffe has done model binding in previous versions:

[<CLIMutable>]
type Adult =
    {
        FirstName  : string
        MiddleName : string option
        LastName   : string
        Age        : int
    }
    override this.ToString() =
        sprintf "%s %s"
            this.FirstName
            this.LastName

    member this.HasErrors() =
        if this.Age < 18 then Some "Person must be an adult (age >= 18)."
        else if this.Age > 150 then Some "Person must be a human being."
        else None

module WebApp =
    let personHandler : HttpHandler =
        fun (next : HttpFunc) (ctx : HttpContext) ->
            let adult = ctx.BindQueryString<Adult>()
            match adult.HasErrors() with
            | Some msg -> RequestErrors.BAD_REQUEST msg
            | None     -> text (adult.ToString())

    let webApp _ =
        choose [
            route "/person" >=> personHandler
            RequestErrors.NOT_FOUND "Not found"
        ]

In this example we have a typical F# record type called Adult. The Adult type has an override for its ToString() method to output something more meaningful than .NET's default and an additional member called HasErrors() which checks if the provided data is correct according to the application's business rules (e.g. an adult must have an age of 18 or over).

There's a few problems with this implementation though. First you must know that the BindQueryString<'T> extension method is a very loose model binding function, which means it will create an instance of type Adult even if some of the mandatory fields (non optional parameters) were not present in the query string (or badly formatted). While this "optimistic" model binding approach has its own advantages, it is not very idiomatic to functional programming and requires additional null checks in subsequent code.

Secondly the model validation has been baked into the personHandler which is not a big problem at first, but means that there's a lot of boilerplate code to be written if an application has more than just one model to work with.

Giraffe 1.1.0 introduces new http handler functions which make model binding more functional. The new tryBindQuery<'T> http handler is a stricter model binding function, which will only create an instance of type 'T if all mandatory fields have been provided by the request's query string. It will also make sure that the provided data is in the correct format (e.g. a numeric value has been provided for an int property of the model) before returning an object of type 'T:

[<CLIMutable>]
type Adult =
    {
        FirstName  : string
        MiddleName : string option
        LastName   : string
        Age        : int
    }
    override this.ToString() =
        sprintf "%s %s"
            this.FirstName
            this.LastName

    member this.HasErrors() =
        if this.Age < 18 then Some "Person must be an adult (age >= 18)."
        else if this.Age > 150 then Some "Person must be a human being."
        else None

module WebApp =
    let adultHandler (adult : Adult) : HttpHandler =
        fun (next : HttpFunc) (ctx : HttpContext) ->
            match adult.HasErrors() with
            | Some msg -> RequestErrors.BAD_REQUEST msg
            | None     -> text (adult.ToString())

    let parsingErrorHandler err = RequestErrors.BAD_REQUEST err

    let webApp _ =
        choose [
            route "/person" >=> tryBindQuery<Adult> parsingErrorHandler None adultHandler
            RequestErrors.NOT_FOUND "Not found"
        ]

The tryBindQuery<'T> requires three parameters. The first is an error handling function of type string -> HttpHandler which will get invoked when the model binding fails. The string parameter in that function will hold the specific model parsing error message. The second parameter is an optional CultureInfo object, which will get used to parse culture specific data such as DateTime values or floating point numbers. The last parameter is a function of type 'T -> HttpHandler, which will get invoked with the parsed model if model parsing was successful.

By using tryBindQuery<'T> there is no danger of encountering a NullReferenceException or the need of doing additional null check any more. By the time the model has been passed into the adultHandler it has been already validated against any data contract violations (e.g. all mandatory fields have been provided, etc.).

At this point the semantic validation of business rules is still embedded in the adultHandler itself. The IModelValidation<'T> interface can help to move this validation step closer to the model and make use of a more generic model validation function when composing the entire web application together:

[<CLIMutable>]
type Adult =
    {
        FirstName  : string
        MiddleName : string option
        LastName   : string
        Age        : int
    }
    override this.ToString() =
        sprintf "%s %s"
            this.FirstName
            this.LastName

    member this.HasErrors() =
        if this.Age < 18 then Some "Person must be an adult (age >= 18)."
        else if this.Age > 150 then Some "Person must be a human being."
        else None

    interface IModelValidation<Adult> with
        member this.Validate() =
            match this.HasErrors() with
            | Some msg -> Error (RequestErrors.BAD_REQUEST msg)
            | None     -> Ok this

module WebApp =
    let textHandler (x : obj) = text (x.ToString())
    let parsingErrorHandler err = RequestErrors.BAD_REQUEST err
    let tryBindQuery<'T> = tryBindQuery<'T> parsingErrorHandler None

    let webApp _ =
        choose [
            route "/person" >=> tryBindQuery<Adult> (validateModel textHandler)
        ]

By implementing the IModelValidation<'T> interface on the Adult record type we can now make use of the validateModel http handler when composing the /person route. This functional composition allows us to entirely get rid of the adultHandler and keep a clear separation of concerns.

First the tryBindQuery<Adult> handler will parse the request's query string and create an instance of type Adult. If the query string had badly formatted or missing data then the parsingErrorHandler will be executed, which allows a user to specify a custom error response for data contract violations. If the model could be successfully parsed, then the validateModel http handler will be invoked which will now validate the business rules of the model (by invoking the IModelValidation.Validate() method). The user can specify a different error response for business rule violations when implementing the IModelValidation<'T> interface. Lastly if the model validation succeeded then the textHandler will be executed which will simply use the object's ToString() method to return a HTTP 200 text response.

All functions are generic now so that adding more routes for other models is just a matter of implementing a new record types for each model and registering a single route in the web application's composition:

let webApp _ =
    choose [
        route "/adult" >=> tryBindQuery<Adult> (validateModel textHandler)
        route "/child" >=> tryBindQuery<Child> (validateModel textHandler)
        route "/dog"   >=> tryBindQuery<Dog>   (validateModel textHandler)
    ]

Overall the new model binding and model validation API aims at providing a more functional counter part to MVC's model validation, except that Giraffe prefers to use functions and interfaces instead of the System.ComponentModel.DataAnnotations attributes. The benefit is that data attributes are often ignored by the rest of the code while a simple validation function can be used from outside Giraffe as well. F# also has the benefit of having a better type system than C#, which means that things like the [<Required>] attribute have little use if there is already an Option<'T> type.

Currently this new improved way of model binding in Giraffe only works for query strings and HTTP form payloads via the tryBindQuery<'T> and tryBindFrom<'T> http handler functions. Model binding functions for JSON and XML remain with the "optimistic" parsing model due to the underlying model binding libraries (JSON.NET and XmlSerializer), but a future update with improvements for JSON and XML is planned as well.

In total you have the following new model binding http handlers at your disposal with Giraffe 1.1.0:

HttpHandler Description
bindJson<'T> Traditional model binding. This is a new http handler equivalent of ctx.BindJsonAsync<'T>.
bindXml<'T> Traditional model binding. This is a new http handler equivalent of ctx.BindAsync<'T>.
bindForm<'T> Traditional model binding. This is a new http handler equivalent of ctx.BindFormAsync<'T>.
tryBindForm<'T> New improved model binding. This is a new http handler equivalent of a new HttpContext extension method called ctx.TryBindFormAsync<'T>.
bindQuery<'T> Traditional model binding. This is a new http handler equivalent of ctx.BindQueryString<'T>.
tryBindQuery<'T> New improved model binding. This is a new http handler equivalent of a new HttpContext extension method called ctx.TryBindQueryString<'T>.
bindModel<'T> Traditional model binding. This is a new http handler equivalent of ctx.BindModelAsync<'T>.

The new model validation API works with any http handler which returns an object of type 'T and is not limited to tryBindQuery<'T> and tryBindFrom<'T> only.

Roadmap overview

To round up this blog post I thought I'll quickly give you a brief overview of what I am planning to tackle next.

The next release of Giraffe is anticipated to be version 1.2.0 (no date set yet) which will mainly focus around improved authentication and authorization handlers (policy based auth support), better CORS support and hopefully better Anti-CSRF support.

After that if nothing else urgent comes up I shall be free to go over two bigger PRs in the Giraffe repository which aim at providing a Swagger integration API and a higher level API of working with web sockets in ASP.NET Core.

Comments

Older Posts