How to generate structured data with GPT-3 and make it 10x more useful

Photo by Alexander Sinn on Unsplash

Nowadays, everyone seems to do stuff with OpenAI’s GPT-3/ChatGPT. But how much more useful would it be, if OpenAI’s text completion APIs would return structured data, like JSON, instead of plain text?

Consider the following use case: you want to create a little app that uses the OpenAI GPT-3 API to generate fantasy characters for your role-playing crew — with background stories, attributes and everything.

Yep, that nerd stuff many of us developers love so much. 🧙 But this is for everyone, so stay with me. It will be short and sweet!

Now, let’s start with a simple prompt like this:

Create a character for a fantasy roleplaying adventure, with background story, physical attributes, mental attributes and appearance.

GPT-3’s answer would be something like this (obviously depending on temperature and other parameters):

Name: Evelynne Shadowblade
Background: Evelynne was born into a family of skilled assassins, and was trained in the art of stealth and subterfuge from a young age. However, she grew disillusioned with the family’s morally questionable work, and struck out on her own as a rogue adventurer.
Physical attributes: Evelynne is a lithe and agile half-elf, with short black hair and piercing green eyes. She stands at 5'7” and weighs 120 lbs. Her lithe build allows her to move quickly and quietly, and her elven heritage gives her heightened senses and reflexes.
Mental attributes: Evelynne is intelligent and cunning, with a quick wit and a sharp tongue. She is also fiercely independent and distrustful of authority figures, preferring to work alone or with a small, tight-knit group of allies.
Appearance: Evelynne wears tight-fitting leather armor that allows her to move silently and quickly. She carries a pair of curved daggers at her waist, and a quiver of poisoned arrows on her back. Her face is often obscured by a hood or mask, to further obscure her identity.

That’s cool! 🥷

But let’s be honest, if I were to display so much text in an app, it would look rather unappealing, and overall I couldn’t do a whole lot with plain text in terms of app features.

So how can we make this better? Well, we would first have to bring it into a format that…

1. is consistent across responses.
2. is easy to parse and store (even if the generated data would be in different languages than English).

Now, plain text GPT-3 output isn’t very well suited for this kind of requirements out of the box.

Instead, let’s try to get the API to spit out some JSON with a structure we define in the prompt:

And this is what we get from GPT-3:

{
  "name": "Evelynne Shadowblade",
  "background": "Evelynne was born into a family of skilled assassins, and was trained in the art of stealth and subterfuge from a young age. However, she grew disillusioned with the family's morally questionable work, and struck out on her own as a rogue adventurer.",
  "physical_attributes": "Evelynne is a lithe and agile half-elf, with short black hair and piercing green eyes. She stands at 5'7\" and weighs 120 lbs. Her lithe build allows her to move quickly and quietly, and her elven heritage gives her heightened senses and reflexes.",
  "mental_attributes": "Evelynne is intelligent and cunning, with a quick wit and a sharp tongue. She is also fiercely independent and distrustful of authority figures, preferring to work alone or with a small, tight-knit group of allies.",
  "appearance": "Evelynne wears tight-fitting leather armor that allows her to move silently and quickly. She carries a pair of curved daggers at her waist, and a quiver of poisoned arrows on her back. Her face is often obscured by a hood or mask, to further obscure her identity."
}

Not bad at all—already ticks all the boxes from the list above!

But we can do even better! In RPGs the loadout of a character plays a big role. So let’s put it into a separate JSON array.

And, sure enough, GPT follows the instructions to the point:

{
  "name": "Evelynne Shadowblade",
  "background": "Evelynne was born into a family of skilled assassins, and was trained in the art of stealth and subterfuge from a young age. However, she grew disillusioned with the family's morally questionable work, and struck out on her own as a rogue adventurer.",
  "physical_attributes": "Evelynne is a lithe and agile half-elf, with short black hair and piercing green eyes. She stands at 5'7\" and weighs 120 lbs. Her lithe build allows her to move quickly and quietly, and her elven heritage gives her heightened senses and reflexes.",
  "mental_attributes": "Evelynne is intelligent and cunning, with a quick wit and a sharp tongue. She is also fiercely independent and distrustful of authority figures, preferring to work alone or with a small, tight-knit group of allies.",
  "appearance": "Evelynne wears tight-fitting leather armor that allows her to move silently and quickly. She carries a pair of curved daggers at her waist, and a quiver of poisoned arrows on her back. Her face is often obscured by a hood or mask, to further obscure her identity.",
  "loadout": {
    "weapons": [
      {
        "name": "Curved Daggers",
        "description": "A pair of razor-sharp daggers with a slight curve to the blade. Ideal for close-range combat and stealthy kills.",
        "dps": 25.0
      },
      {
        "name": "Poisoned Arrows",
        "description": "Arrows dipped in a potent toxin that paralyzes or kills the target. Ideal for ranged combat and setting traps.",
        "dps": 15.0
      }
    ],
    "potions": [
      {
        "name": "Invisibility Potion",
        "description": "A potion that renders the drinker invisible for a short period of time. Ideal for sneaking past guards and avoiding detection.",
        "effect": "Grants temporary invisibility."
      },
      {
        "name": "Healing Potion",
        "description": "A potion that restores a portion of the drinker's health. Ideal for quick healing in combat or after a tough battle.",
        "effect": "Restores 50 health."
      }
    ]
  }
}

Obviously we could go much further here, but I think you get the point: You can easily get OpenAI’s GPT-3 API to respond with a JSON using literally any schema you want!

Every developer out there will know what this means: it opens up a whole new set of applications with a super low entry barrier. 🚀 So go, code the next generation of GPT-based applications before someone else does. Exciting times! 🤯

Let me finish with some gotcha’s:

1. Define value types as strictly as possible, such as units (“use SI units”, “use metric units”, “use g instead of kg”), types (“make “dps” a Double”) and the likes.
2. Be prepared for incomplete JSONs if the token limit is reached. Split up single big requests into multiple smaller ones or tell the API to keep it short (literally!) to avoid this situation. Repeat requests if necessary.
3. Following up on the previous point: When attempting to parse the JSON modify your parser to be as lenient as possible. For example: the generated JSON will at times have added commas at the end of lists and for the last key/value pair in objects. So you either need to remove those superfluous commas or use a JSON5 parser which by design allows trailing commas and more.
   Here is a possible implementation of such a trailing comma remover in Kotlin for your convenience:

private val trailingCommaInObjectRegex = Regex(""",\s*\}""")
private val trailingCommaInArrayRegex = Regex(""",\s*]""")

private fun String.removeSuperfluousTrailingComma(): String {
    return replace(trailingCommaInObjectRegex, "}")
      .replace(trailingCommaInArrayRegex, "]")
}

If this helped you, and you are now successfully tapping JSONs from OpenAI’s APIs, consider fueling my brain with some black liquid, too. ☕ Much appreciated!