Carson Tang

Unpacking Python Generated HMAC's in Golang

The issue at hand: There is no format specifier in Golang to allow us to decrypt packed information.

In the snippet below, we first create a shared secret which will be a random byte string containing 32 number of bytes.

Using this secret and our API_KEY, we can create a DB_token which will be stored in the database and can be used for example, to reference the permissions of a user.

Using the user's user_id and the secret, we will create a user_api_token which is what will be returned back to the user.

The important thing to note here is the format string for packing the user_id and secret, which is <36s32s. Briefly, the < indicates the byte order to be little-endian, the numerical value before the character indicates the number of bytes and the s indicates the bytes type, for more details see here.

import base64
import hmac
import secrets
import struct

API_KEY = 'g58gdzomuf3zj5crbfecr6jqway4f7qt8fdgushywit2r5ipaxy186uxppypdeyu'

def generate_token(user_id):
    secret = secrets.token_bytes(32)
    DB_token = hmac.new(API_KEY.encode(), msg=secret digestmod='sha256).hexdigest()
    user_api_token = base64.urlsafe_b64encode(struct.pack('<36s32s', user_id, secret)).decode('utf-8')

Running some values as we walkthrough the code, we call generate_token() with user_id = b'87c61654-1515-4b08-b4fd-3cc56604f603'

The values of the variables in this run:

In the snippet below, we unpack the user_id and secret from the user_api_token, which allows us to then recreate token using the secret and API_KEY that should match the existing one stored in the database from earlier.

import base64
import hmac
import secrets
import struct

API_KEY = 'g58gdzomuf3zj5crbfecr6jqway4f7qt8fdgushywit2r5ipaxy186uxppypdeyu'

def verify_token(user_api_token):
    user_id, secret = struct.unpack('<36s32s', base64.
    urlsafe_b64decode(user_api_token))
    digest = hmac.new(API_KEY.encode(), msg=secret, digestmod='sha256').hexdigest()
    hmac.compare_digest(digest, DB_token)

Using the user_api_token from above, user_api_token = 'ODdjNjE2NTQtMTUxNS00YjA4LWI0ZmQtM2NjNTY2MDRmNjAzsVEMrdIPEd77duAe8pfnUK92Y53DRIjtLnTsLxngpBM='

The value of the variables:

As can be seen, the user_id and secret are consistent, which as a result causes the digest and the DB_token to match as well!

Issue

The challenge now, is recreating the token verifying process in Go, as the crypto/hmac library does not exactly allow us to specify the format of values to unpack from the user_api_token.

Taking the user_api_token generated by our python side,

import (
    "crypto/hmac"
    "crypto/sha256"
	"encoding/base64"
	"encoding/hex"
	"strings"
	"fmt"
)

func main() {
    user_api_token := "ODdjNjE2NTQtMTUxNS00YjA4LWI0ZmQtM2NjNTY2MDRmNjAzsVEMrdIPEd77duAe8pfnUK92Y53DRIjtLnTsLxngpBM="
    user_api_token = strings.TrimSuffix(user_token, "=")
    decoded, err := base64.RawURLEncoding.DecodeString(user_token)
	if err != nil {
		panic(err)
	}
    fmt.Printf("%#v\n", decoded)

    user_id := make([]byte, 36)
    copy(user_id, decoded)
    secret := make([]byte, 32)
    copy(secret, decoded[36:])

    API_KEY := "g58gdzomuf3zj5crbfecr6jqway4f7qt8fdgushywit2r5ipaxy186uxppypdeyu"
    mac := hmac.New(sha256.New, []byte(API_KEY))
    mac.Write(secret)
    expected_MAC := mac.Sum(nil)
    api_token := hex.EncodeToString(expected_MAC)
}

We can first print out decoded to see what we're dealing with. ctrl f for ad to see it appear in decoded below and in the secret

decoded = []byte{0x38, 0x37, 0x63, 0x36, 0x31, 0x36, 0x35, 0x34, 0x2d, 0x31, 0x35, 0x31, 0x35, 0x2d, 0x34, 0x62, 0x30, 0x38, 0x2d, 0x62, 0x34, 0x66, 0x64, 0x2d, 0x33, 0x63, 0x63, 0x35, 0x36, 0x36, 0x30, 0x34, 0x66, 0x36, 0x30, 0x33, 0xb1, 0x51, 0xc, 0xad, 0xd2, 0xf, 0x11, 0xde, 0xfb, 0x76, 0xe0, 0x1e, 0xf2, 0x97, 0xe7, 0x50, 0xaf, 0x76, 0x63, 0x9d, 0xc3, 0x44, 0x88, 0xed, 0x2e, 0x74, 0xec, 0x2f, 0x19, 0xe0, 0xa4, 0x13}

Is there anything we can notice here?

First, let us note that decoded is a byte array of length 68 which just so happens to be the sum of the length of the user_id (36) and the secret (32).

Bringing the secret value again, secret = b'\xb1Q\x0c\xad\xd2\x0f\x11\xde\xfbv\xe0\x1e\xf2\x97\xe7P\xafvc\x9d\xc3D\x88\xed.t\xec/\x19\xe0\xa4\x13'

You may notice that the last 32 bytes of decoded are very similar.

The last 32 bytes of decoded being []byte{0xb1, 0x51, 0xc, 0xad, 0xd2, 0xf, 0x11, 0xde, 0xfb, 0x76, 0xe0, 0x1e, 0xf2, 0x97, 0xe7, 0x50, 0xaf, 0x76, 0x63, 0x9d, 0xc3, 0x44, 0x88, 0xed, 0x2e, 0x74, 0xec, 0x2f, 0x19, 0xe0, 0xa4, 0x13}

Let's separate the first 36 bytes to be user_id and the last 32 bytes to be secret.

We print out user_id and get

user_id = [56 55 99 54 49 54 53 52 45 49 53 49 53 45 52 98 48 56 45 98 52 102 100 45 51 99 99 53 54 54 48 52 102 54 48 51]

string(user_id) = "87c61654-1515-4b08-b4fd-3cc56604f603"

Notice that the user_id here matches the user_id from earlier!

We try printing out secret as well, however...

secret = [177 81 12 173 210 15 17 222 251 118 224 30 242 151 231 80 175 118 99 157 195 68 136 237 46 116 236 47 25 224 164 19]

string(secret) = �Q ����v���P�vc��D��.t�/�

This isn't exactly what we're looking for.

Instead we write this secret to the hmac,

expected_MAC = [189 148 242 129 34 6 12 33 248 251 149 74 105 95 206 15 32 141 14 166 198 14 136 48 35 183 143 107 88 181 81 136]

And finally we get the hexadecimal of the hmac sum of the expected_MAC, to which we see the exact same value as the DB_token from the first python snippet. Yay!

api_token = "bd94f28122060c21f8fb954a695fce0f208d0ea6c60e883023b78f6b58b55188"


[ / /]