New Web (API) Foundations (Part Five)

Published: 2021-10-22

Author: Teddi

As a short recap over the four posts previous to this, we have:

• Reviewed what failed with v1 of the API and set some ground rules of what powers the underlying API.
• Set some logical rules as to what the API must fulfil in terms of being usable and released to the public.
• Decided to try using GraphQL instead of REST to try and improve things.
• Implemented a proof of concept which was successful and at this point we have a working API!

While things were maintainable and in general better performing what we hadn’t done was sort out authentication & authorization. As a quick thing because security is hard and people tend to get the two terms mixed up:

• Authentication is the process of working out if you should even have access to the thing you’re trying to access. This can take the form of an API key, a username / password; in short a key to a lock.
• Authorization is the process of working out what you should have access to. Are you an admin? Are you banned? Essentially what can you see, touch and do?

Even back in V1 of the API there were only really ever plans for authentication and authorization was never really a consideration, however I have fairly strong views towards privacy and believe that a person should always have the right to shield their data from others so authorization was absolutely at the top of my list of things to do.

As this post is going to be fairly lengthy we’ll be focusing on authentication.

The API actually has three ways of authenticating with it. That’s right, not one or two, but three! To complicate things further one of these routes is intended to be digested by humans while the other two routes are intended for automation, but why is that?

As part of the desire for this to be available to the public we need the ability for humans to fetch API keys which means we need to grant the ability for a human to “log in” to the API. Whilst they’re not actually directly interacting with the API itself what they are interacting is with a layer above the API: The ability to see information about the API, manage the keys and other “meta” actions. However I also have zero desire to have users register with a username, password and other details that websites otherwise typically have you register with so instead we leverage Steam’s OpenID and we authenticate you that way; this means we don’t need to store credentials, you don’t need another account but everyone gets a seamless process! Win win!

Although we have humans now logging in, what we don’t have yet is a form of automated login for computers to use.

Over the years engineers have come up with fun ways of dealing with authentication for automated systems. Many may not remember it thanks to the work done over the years, but once upon a time sites like Facebook had you “import” friends by you literally handing over your username & password to your email account at somewhere like GMail. You were simply expected to hand over the keys to your metaphorical kingdom for what was often very useless perks because it was the norm! Many a person lost time, accounts, money and even more due to scams and phishing all because humans were trained to do this.

Thankfully it’s 2021 and we’ve come up with overall better solutions even if they’re not always used in the correct ways.

Our second way of authenticating with the API is using authentication keys, or more commonly known as API keys which from this point on we’ll refer to them as. API keys take many forms; in our case we generate a key for you (which you need to keep a secret!) and this is tied to your account. Once the key is supplied we no longer hold a direct record of said key and instead only store a hash of it. We do this for a few reasons:

• Firstly we can’t guarantee Steam accounts are properly secure. We can’t tell if the user has 2FA enabled or anything about their account security at all. What this means is that if that account is compromised then the attacker can gain access anywhere that the account holder has access to with Steam OpenID.
  • We only display part of the hash for management purposes so the attacker doesn’t really have much to go on.
  • Equally however there isn’t much that can be done to stop the attacker deleting and generating new keys but it does give us an audit trail!
• Secondly if we were compromised at the database level then the attacker has immediate access to all the API keys and could use them merrily until we voided them all which isn’t a nice experience.
  • Instead by hashing them we buy time and the ability for users to rotate their keys out as the data isn’t available in its raw format. It’s not much but it is better than nothing.

This form of authentication requires checking against the database to ensure the key is valid every single time and includes other actions such as is the key blocked, has it hit the limit and so on. This is perfectly fine for systems where there’s a developer and an application querying data but what about users playing on the server? Generating keys dynamically only to last a certain amount of time is a pain and is there a way they don’t have to authenticate against the database?

Enter JWTs

JWTs, pronounced as “Jots” for some daft reason stands for “JSON Web Token”. Here’s the breakdown:

• JWTs are signed blobs of JSON data.
• The data is cryptographically signed
• But the data is not encrypted (so don’t go storing sensitive details in there).

I’m going to segue for a moment and have a minor rant. JWTs should not be used as de-facto authentication where user sessions are critical and important. Even with refresh tokens! Too many web developers misunderstand the fundamental best-case usage of JWTs and where they’re actually useful and are often used as a crutch. Too often many web applications use a JWT which has no expiry or worse yet - they use a single JWT which is valid forever (or if they ever bother to swap out the certs). If you are unable to remove a persons authentication without having to destroy all the other keys (i.e. other users authentication), your lock is terrible and you should feel bad.

In short: Only use JWTs for incredibly short sessions where long-term access isn’t expected or as a bridge to tell a server “it’s okay to give this person a proper key”. /rantover

Because JWTs are signed blobs of data and are generally authenticated using certificates it means that we can authenticate the user directly on the server and check the signature of the data instead of storing an API key and relaying any authentication requests back to the database.

This is what a JWT looks like in its Base64 encoded format: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJhY2NvdW50SUQiOiIxIiwiZXhwIjoxNjM0OTM5NTkzLCJpYXQiOjE2MzQ5Mzc3OTMsIm5iZiI6MTYzNDkzNzczMywic3RlYW1JRDY0IjoiNzY1NjExOTc5ODE3NTE3MjMifQ.LFyWwquR1rvFHvfhJC2ISuXDZXPfbW_Dqi2tKytsgRNqbQE1otEIVkvLjnkkuQfnhhQsaQhXDuYxB8xW7En3joCnDA_xLZ7-s3rDU2wUBexwlRI5Oal9B9aebfvkTxNzycszH_oaRcS3yqzP5Lmx_hZJnLaACj8GYMO1brzcZVX7PobKXeSVjM2BbjENtGhs6RBBatWeKyfWo-LhY0TdmrcRiUKiYJEOoYeUaibN198JPEEJp699VZYwits65DMIg2wfKexZyIENJhWnCyIXYMCqD7tIXUjgCEGLGDjdPeZUCH_oykjtLMnj8Zz_d58mqd18Z2f58SHiwqlV-118TwX5E-WpLjWfBPI-zibVGprtoVomg9INqqblezpaKxyGSd3L1feIcC0a3HQr4CIn5yHC2_iPKYYrrn2H7a4gPZ-SPFQLWv3sFKTyH5R5nu68320jlPIoepVjZGFeU1ZacrbqwX-xg7ejrFuPsVHitTNaueHvNSnp6Aa-RdQtzHGuCkJtU5SOxFT7nNY13VclK3GdOWdOQ94OL6dFJACGBEj4aslRWiEryekgVgDT7Jhee-ycZG4Ms1PoorANbv8pfGxm9Q7jtSwerKeQt8bWguYaxQqpIMw85qYaoXsgHp_azjHnC1-TaxIT_BlGPj9ENl5HYxxdkZyFRnpxsF5EIpc

Essentially a huge blob of text that makes no sense at all. When converted however we get:

{
  "accountID": "1",
  "exp": 1634939593,
  "iat": 1634937793,
  "nbf": 1634937733,
  "steamID64": "76561197981751723"
}

You can verify this @ jwt.io.

Every player that plays on BB now gets one of these delivered automatically and it’s used to query things like map names, title data and so on.

At one point in time this was a valid JWT for my account! From an authentication point of view all we actually care about the signature from the JWT in the Base64 data, as well as the exp and nbf fields. exp is when the token expires (Friday, October 22, 2021 21:53:13) and the nbf field is “not before”, effectively this token isn’t valid before the time of  Friday, October 22, 2021 21:22:13 to prevent potential replay attacks if the server was misconfigured for whatever reason. Because only we (within the API) hold the keys for these specific signatures it means that if someone were to try and send us a spoofed message the request would immediately fail.

JWTs are perfect for the problem we have with player authentication when playing on the server. Only the player needs to store a copy of the JWT and using the magic of maths and cryptography means we can verify a player is who they say they are with very little effort. Better yet, we can define on the key when it expires so users at most only get 30 minutes per JWT to make requests before it’s a useless blob of text. Furthermore we are the issuers of the JWT meaning that you can only request a fresh key every so often and only when we say so. This gives us a degree of control and helps to prevent abuse.

With that, we’ve covered the three different methods users can attempt to identify themselves with the API. The next post will be looking towards authorization and the importance of making sure players can only see the data they’re meant to see.