Skip to main content

Getting Started

In this notebook, we will go through the basics of creating a RAIL spec and using Guardrails to enforce it.

Objective

Our goal is to extract detailed patient information from a medical record. We will use a simple example to demonstrate how Guardrails can be used to enforce a RAIL spec.

As an example, we will use the following medical record:

doctors_notes = """49 y/o Male with chronic macular rash to face & hair, worse in beard, eyebrows & nares.
Itchy, flaky, slightly scaly. Moderate response to OTC steroid cream"""

We want our extracted information to contain the following fields:

  1. Patient's gender
  2. Patient's age
  3. A list of symptoms, each with a severity rating and an affected area
  4. A list of medications, each with information about the patient's response to the medication

Installation

To get started, install the guardrails-ai package with pip. Also, install pydantic, typing, openai, and rich for this example.

%pip install guardrails-ai pydantic typing openai rich
    Note: you may need to restart the kernel to use updated packages.

Creating a RAIL spec

At the heart of Guardrails is the RAIL spec.

RAIL started as a flavor of XML (standing for Reliable AI markup Language) that describes the expected structure and type of the output of the LLM, the quality criteria for the output to be valid and corrective actions to be taken if the output is invalid. Rails can be written as Pydantic as well. In this example, we'll be developing a Guard using Pydantic.

A RAIL is composed of 2 main components:

  1. Output schema
  2. Prompt

Output schema

We start by creating the output schema. The output schema is a Pydantic object that describes the expected structure and type of the output of the LLM.

Specifying structure

As mentioned earlier, we want our extracted information to contain the following fields: gender, age, symptoms, and medications. Using a Pydantic RAIL, we can create fairly complex output schemas. For example, we can specify that a field is a list of objects, etc. We see an example of this below.

RAIL Spec

from pydantic import BaseModel, Field
from typing import List
class Symptom(BaseModel):
symptom: str = Field(description="Symptom that a patient is experiencing")
affected_area: str= Field(description="What part of the body the symptom is affecting")

class Medication(BaseModel):
medication: str = Field(description="Name of the medication the patient is taking")
response: str = Field(description="How the patient is responding to the medication")

class PatientInfo(BaseModel):
gender: str = Field(description="Patient's gender")
age: int = Field(description="Patient's age")
symptoms: List[Symptom] = Field(description="Symptoms that the patient is currently experiencing. Each symptom should be classified into separate item in the list.")
current_meds: List[Medication] = Field(description="Medications the patient is currently taking and their response")

That pydantic RAIL, when used with a guard, will produce JSON output that looks as follows:

{
"patient_info": {
"gender": ...,
"age": ...,
"symptoms": [
{
"symptom": ...,
"affected area": ...
},
...
],
"current_meds": [
{
"medication": ...,
"response": ...
},
...
]
}
}

Specifying quality criteria

Next, we want to specify the quality criteria for the output to be valid and corrective actions to be taken if the output is invalid. We can do this by adding a format tag to each field in the output schema. Format tags can either be enforced by Guardrails, or they can only be suggetions to the LLM. You can see the list of validators enforced by Guardrails here. Additionally, you can create your own custom validators, see examples here 1, 2, 3.

As an example, for our use case we specify that the affected_area of symptoms should be one of the following: ['head', 'neck', 'chest']. For this, we use the valid-choices validator.

Specifying corrective actions

Additionally, we can specify how to handle invalid output by adding a on-fail-... attribute to an element.

For example, we can specify that if the affected_area of a symptom is not one of the valid choices, we should re-prompt the LLM to correct its output.

We do this by adding the on-fail-valid-choices='reask' attribute to the affected_area field. To see the full list of corrective actions, see here.

Finally, our updated output schema looks like:

from guardrails.validators import ValidRange, ValidChoices

class Symptom(BaseModel):
symptom: str = Field(description="Symptom that a patient is experiencing")
affected_area: str= Field(
description="What part of the body the symptom is affecting",
#(2)!
validators=[ValidChoices(choices=['head', 'neck', 'chest'], on_fail='reask')]
)

class Medication(BaseModel):
medication: str = Field(description="Name of the medication the patient is taking")
response: str = Field(description="How the patient is responding to the medication")

class PatientInfo(BaseModel):
gender: str = Field(description="Patient's gender")
age: int = Field(
description="Patient's age",
#(1)!
validators=[ValidRange(min=0, max=100)]
)
symptoms: List[Symptom] = Field(description="Symptoms that the patient is currently experiencing. Each symptom should be classified into separate item in the list.")
current_meds: List[Medication] = Field(description="Medications the patient is currently taking and their response")
  1. We specify the valid-range validator for the age field. This validator takes the min and max arguments, which we pass as 0 and 100 respectively.
  2. We specify the valid-choices validator for the affected_area field. This validator will check that the value of the field is one of the choices specified in the format attribute. We specify that if the affected_area field fails the valid-choices validator, we should re-prompt the LLM to correct its output. This is done by setting the on-fail-valid-choices attribute to reask.

!!! note In order to pass in a list of choices, we wrap the list in curly braces -- {['head', 'neck', 'chest']}. Anything inside the curly braces will be evaluated as a Python expression. This allows us to pass in more complex data structures or objects as arguments to the validator.

Prompt

The prompt contains high level instructions for the LLM, as well as any parameters that need to be substituted at runtime.

For our use case, we create the following prompt:

prompt = """
Given the following doctor's notes about a patient,
please extract a dictionary that contains the patient's information. <!-- (1)! -->

${doctors_notes} <!-- (2)! -->

${gr.complete_json_suffix_v2} <!-- (3)! -->
"""
  1. This is the high level instructions for the LLM.
  2. This is a prompt parameter that will be substituted at runtime. We will see how to do this in the next section.
  3. This is some boilerplate text that formats helps the LLM output a valid JSON object. Guardrails will automatically add this text to the end of the LLM's output. You can see the full list of boilerplate text here.

Full rail spec

Putting it all together, you can see what the full rail spec for our use case looks like:

from pydantic import BaseModel, Field
from typing import List
from guardrails.validators import ValidRange, ValidChoices

prompt = """
Given the following doctor's notes about a patient, please extract a dictionary that contains the patient's information.

${doctors_notes}

${gr.complete_json_suffix_v2}
"""

class Symptom(BaseModel):
symptom: str = Field(description="Symptom that a patient is experiencing")
affected_area: str = Field(description="What part of the body the symptom is affecting", validators=[ValidChoices(choices=['head', 'neck', 'chest'], on_fail="reask")])

class Medication(BaseModel):
medication: str = Field(description="Name of the medication the patient is taking")
response: str = Field(description="How the patient is responding to the medication")


class PatientInfo(BaseModel):
gender: str = Field(description="Patient's gender")
age: int = Field(validators=[ValidRange(min=0, max=100, on_fail="fix")])
symptoms: List[Symptom] = Field(description="Symptoms that the patient is currently experiencing. Each symptom should be classified into a separate item in the list.")
current_meds: List[Medication] = Field(description="Medications the patient is currently taking and their response")
    /home/zayd/workspace/guardrails/.venv/lib/python3.9/site-packages/torch/cuda/__init__.py:611: UserWarning: Can't initialize NVML
warnings.warn("Can't initialize NVML")

This can also be written using the actual XML based rail spec, as found here.

Using Guardrails to enforce the RAIL spec

We use the RAIL spec to create a Guard object. The Guard object is used to wrap the LLM API call and enforce the RAIL spec on the output of the LLM call.

import guardrails as gd

# From pydantic:
guard = gd.Guard.from_pydantic(output_class=PatientInfo, prompt=prompt)

# From XML RAIL spec
# guard = gd.Guard.from_rail('getting_started.rail')

We can see that the Guard object compiles the RAIL output specification and adds it to the provided prompt.

Next, we call the Guard object with the LLM API call as the first argument and add any additional arguments to the LLM API call as the remaining arguments.

import openai
import json

# Wrap the OpenAI API call with the `guard` object
res = guard(
openai.chat.completions.create,
prompt_params={"doctors_notes": doctors_notes},
max_tokens=1024,
temperature=0.3,
)

# Print the validated output from the LLM
print(json.dumps(res.validated_output, indent=2))
    {
"gender": "Male",
"age": 49,
"symptoms": [
{
"symptom": "chronic macular rash",
"affected_area": "head"
},
{
"symptom": "itchy",
"affected_area": null
},
{
"symptom": "itchy",
"affected_area": null
},
{
"symptom": "itchy",
"affected_area": null
},
{
"symptom": "flaky",
"affected_area": null
},
{
"symptom": "slightly scaly",
"affected_area": null
}
],
"current_meds": [
{
"medication": "OTC steroid cream",
"response": "Moderate"
}
]
}

We can see that our output was structured correctly, and that the values of the affected_area field were one of the valid choices.

Inspecting logs to peek under the hood

Guardrails logs all of the steps it takes to enforce the RAIL spec. We inspect these logs below to see all the steps Guardrails took to enforce the RAIL spec.

Inspecting Step 1: Getting LLM output

from rich import print
print(guard.history.last.iterations.first.rich_group)
╭──────────────────────────────────────────────────── Prompt ─────────────────────────────────────────────────────╮
│ │
│ Given the following doctor's notes about a patient, please extract a dictionary that contains the patient's │
│ information. │
│ │
│ 49 y/o Male with chronic macular rash to face & hair, worse in beard, eyebrows & nares. │
│ Itchy, flaky, slightly scaly. Moderate response to OTC steroid cream │
│ │
│ │
│ Given below is XML that describes the information to extract from this document and the tags to extract it │
│ into. │
│ │
│ <output> │
│ <string name="gender" description="Patient's gender"/> │
│ <integer name="age" format="valid-range: min=0 max=100"/> │
│ <list name="symptoms" description="Symptoms that the patient is currently experiencing. Each symptom should │
│ be classified into a separate item in the list."> │
│ <object> │
│ <string name="symptom" description="Symptom that a patient is experiencing"/> │
│ <string name="affected_area" description="What part of the body the symptom is affecting" │
│ format="valid-choices: choices=['head', 'neck', 'chest']"/> │
│ </object> │
│ </list> │
│ <list name="current_meds" description="Medications the patient is currently taking and their response"> │
│ <object> │
│ <string name="medication" description="Name of the medication the patient is taking"/> │
│ <string name="response" description="How the patient is responding to the medication"/> │
│ </object> │
│ </list> │
│ </output> │
│ │
│ │
│ ONLY return a valid JSON object (no other text is necessary), where the key of the field in JSON is the `name` │
│ attribute of the corresponding XML, and the value is of the type specified by the corresponding XML's tag. The │
│ JSON MUST conform to the XML format, including any types and format requests e.g. requests for lists, objects │
│ and specific types. Be correct and concise. │
│ │
│ Here are examples of simple (XML, JSON) pairs that show the expected behavior: │
│ - `<string name='foo' format='two-words lower-case' />` => `{'foo': 'example one'}` │
│ - `<list name='bar'><string format='upper-case' /></list>` => `{"bar": ['STRING ONE', 'STRING TWO', etc.]}` │
│ - `<object name='baz'><string name="foo" format="capitalize two-words" /><integer name="index" │
│ format="1-indexed" /></object>` => `{'baz': {'foo': 'Some String', 'index': 1}}` │
│ │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭───────────────────────────────────────────────── Instructions ──────────────────────────────────────────────────╮
│ You are a helpful assistant, able to express yourself purely through JSON, strictly and precisely adhering to │
│ the provided XML schemas. │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭──────────────────────────────────────────────── Message History ────────────────────────────────────────────────╮
│ No message history. │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭──────────────────────────────────────────────── Raw LLM Output ─────────────────────────────────────────────────╮
│ { │
│ "gender": "Male", │
│ "age": 49, │
│ "symptoms": [ │
│ { │
│ "symptom": "chronic macular rash", │
│ "affected_area": "face" │
│ }, │
│ { │
│ "symptom": "itchy", │
│ "affected_area": "beard" │
│ }, │
│ { │
│ "symptom": "itchy", │
│ "affected_area": "eyebrows" │
│ }, │
│ { │
│ "symptom": "itchy", │
│ "affected_area": "nares" │
│ }, │
│ { │
│ "symptom": "flaky", │
│ "affected_area": "face" │
│ }, │
│ { │
│ "symptom": "slightly scaly", │
│ "affected_area": "face" │
│ } │
│ ], │
│ "current_meds": [ │
│ { │
│ "medication": "OTC steroid cream", │
│ "response": "Moderate" │
│ } │
│ ] │
│ } │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────── Validated Output ────────────────────────────────────────────────╮
│ { │
│ 'gender': 'Male', │
│ 'age': 49, │
│ 'symptoms': [ │
│ { │
│ 'symptom': 'chronic macular rash', │
│ 'affected_area': FieldReAsk( │
│ incorrect_value='face', │
│ fail_results=[ │
│ FailResult( │
│ outcome='fail', │
│ metadata=None, │
│ error_message="Value face is not in choices ['head', 'neck', 'chest'].", │
│ fix_value=None │
│ ) │
│ ], │
│ path=['symptoms', 0, 'affected_area'] │
│ ) │
│ }, │
│ { │
│ 'symptom': 'itchy', │
│ 'affected_area': FieldReAsk( │
│ incorrect_value='beard', │
│ fail_results=[ │
│ FailResult( │
│ outcome='fail', │
│ metadata=None, │
│ error_message="Value beard is not in choices ['head', 'neck', 'chest'].", │
│ fix_value=None │
│ ) │
│ ], │
│ path=['symptoms', 1, 'affected_area'] │
│ ) │
│ }, │
│ { │
│ 'symptom': 'itchy', │
│ 'affected_area': FieldReAsk( │
│ incorrect_value='eyebrows', │
│ fail_results=[ │
│ FailResult( │
│ outcome='fail', │
│ metadata=None, │
│ error_message="Value eyebrows is not in choices ['head', 'neck', 'chest'].", │
│ fix_value=None │
│ ) │
│ ], │
│ path=['symptoms', 2, 'affected_area'] │
│ ) │
│ }, │
│ { │
│ 'symptom': 'itchy', │
│ 'affected_area': FieldReAsk( │
│ incorrect_value='nares', │
│ fail_results=[ │
│ FailResult( │
│ outcome='fail', │
│ metadata=None, │
│ error_message="Value nares is not in choices ['head', 'neck', 'chest'].", │
│ fix_value=None │
│ ) │
│ ], │
│ path=['symptoms', 3, 'affected_area'] │
│ ) │
│ }, │
│ { │
│ 'symptom': 'flaky', │
│ 'affected_area': FieldReAsk( │
│ incorrect_value='face', │
│ fail_results=[ │
│ FailResult( │
│ outcome='fail', │
│ metadata=None, │
│ error_message="Value face is not in choices ['head', 'neck', 'chest'].", │
│ fix_value=None │
│ ) │
│ ], │
│ path=['symptoms', 4, 'affected_area'] │
│ ) │
│ }, │
│ { │
│ 'symptom': 'slightly scaly', │
│ 'affected_area': FieldReAsk( │
│ incorrect_value='face', │
│ fail_results=[ │
│ FailResult( │
│ outcome='fail', │
│ metadata=None, │
│ error_message="Value face is not in choices ['head', 'neck', 'chest'].", │
│ fix_value=None │
│ ) │
│ ], │
│ path=['symptoms', 5, 'affected_area'] │
│ ) │
│ } │
│ ], │
│ 'current_meds': [ │
│ {'medication': 'OTC steroid cream', 'response': 'Moderate'} │
│ ] │
│ } │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

We can see that the output returned by the LLM contained invalid values for the affected_area field.

On validation, Guardrails created ReAsk objects that contains all of the information needed to re-prompt the LLM to correct its output.

Inspecting Step 2: Re-prompting LLM

print(guard.history.last.iterations.at(1).rich_group)
╭──────────────────────────────────────────────────── Prompt ─────────────────────────────────────────────────────╮
│ │
│ I was given the following JSON response, which had problems due to incorrect values. │
│ │
│ { │
│ "gender": "Male", │
│ "age": 49, │
│ "symptoms": [ │
│ { │
│ "symptom": "chronic macular rash", │
│ "affected_area": { │
│ "incorrect_value": "face", │
│ "error_messages": [ │
│ "Value face is not in choices ['head', 'neck', 'chest']." │
│ ] │
│ } │
│ }, │
│ { │
│ "symptom": "itchy", │
│ "affected_area": { │
│ "incorrect_value": "beard", │
│ "error_messages": [ │
│ "Value beard is not in choices ['head', 'neck', 'chest']." │
│ ] │
│ } │
│ }, │
│ { │
│ "symptom": "itchy", │
│ "affected_area": { │
│ "incorrect_value": "eyebrows", │
│ "error_messages": [ │
│ "Value eyebrows is not in choices ['head', 'neck', 'chest']." │
│ ] │
│ } │
│ }, │
│ { │
│ "symptom": "itchy", │
│ "affected_area": { │
│ "incorrect_value": "nares", │
│ "error_messages": [ │
│ "Value nares is not in choices ['head', 'neck', 'chest']." │
│ ] │
│ } │
│ }, │
│ { │
│ "symptom": "flaky", │
│ "affected_area": { │
│ "incorrect_value": "face", │
│ "error_messages": [ │
│ "Value face is not in choices ['head', 'neck', 'chest']." │
│ ] │
│ } │
│ }, │
│ { │
│ "symptom": "slightly scaly", │
│ "affected_area": { │
│ "incorrect_value": "face", │
│ "error_messages": [ │
│ "Value face is not in choices ['head', 'neck', 'chest']." │
│ ] │
│ } │
│ } │
│ ], │
│ "current_meds": [ │
│ { │
│ "medication": "OTC steroid cream", │
│ "response": "Moderate" │
│ } │
│ ] │
│ } │
│ │
│ Help me correct the incorrect values based on the given error messages. │
│ │
│ Given below is XML that describes the information to extract from this document and the tags to extract it │
│ into. │
│ │
│ <output> │
│ <string name="gender" description="Patient's gender"/> │
│ <integer name="age" format="valid-range: min=0 max=100"/> │
│ <list name="symptoms" description="Symptoms that the patient is currently experiencing. Each symptom should │
│ be classified into a separate item in the list."> │
│ <object> │
│ <string name="symptom" description="Symptom that a patient is experiencing"/> │
│ <string name="affected_area" description="What part of the body the symptom is affecting" │
│ format="valid-choices: choices=['head', 'neck', 'chest']"/> │
│ </object> │
│ </list> │
│ <list name="current_meds" description="Medications the patient is currently taking and their response"> │
│ <object> │
│ <string name="medication" description="Name of the medication the patient is taking"/> │
│ <string name="response" description="How the patient is responding to the medication"/> │
│ </object> │
│ </list> │
│ </output> │
│ │
│ │
│ ONLY return a valid JSON object (no other text is necessary), where the key of the field in JSON is the `name` │
│ attribute of the corresponding XML, and the value is of the type specified by the corresponding XML's tag. The │
│ JSON MUST conform to the XML format, including any types and format requests e.g. requests for lists, objects │
│ and specific types. Be correct and concise. If you are unsure anywhere, enter `null`. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭───────────────────────────────────────────────── Instructions ──────────────────────────────────────────────────╮
│ You are a helpful assistant, able to express yourself purely through JSON, strictly and precisely adhering to │
│ the provided XML schemas. │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭──────────────────────────────────────────────── Message History ────────────────────────────────────────────────╮
│ No message history. │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭──────────────────────────────────────────────── Raw LLM Output ─────────────────────────────────────────────────╮
│ { │
│ "gender": "Male", │
│ "age": 49, │
│ "symptoms": [ │
│ { │
│ "symptom": "chronic macular rash", │
│ "affected_area": "head" │
│ }, │
│ { │
│ "symptom": "itchy", │
│ "affected_area": "null" │
│ }, │
│ { │
│ "symptom": "itchy", │
│ "affected_area": "null" │
│ }, │
│ { │
│ "symptom": "itchy", │
│ "affected_area": "null" │
│ }, │
│ { │
│ "symptom": "flaky", │
│ "affected_area": "null" │
│ }, │
│ { │
│ "symptom": "slightly scaly", │
│ "affected_area": "null" │
│ } │
│ ], │
│ "current_meds": [ │
│ { │
│ "medication": "OTC steroid cream", │
│ "response": "Moderate" │
│ } │
│ ] │
│ } │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────── Validated Output ────────────────────────────────────────────────╮
│ { │
│ 'gender': 'Male', │
│ 'age': 49, │
│ 'symptoms': [ │
│ {'symptom': 'chronic macular rash', 'affected_area': 'head'}, │
│ {'symptom': 'itchy', 'affected_area': None}, │
│ {'symptom': 'itchy', 'affected_area': None}, │
│ {'symptom': 'itchy', 'affected_area': None}, │
│ {'symptom': 'flaky', 'affected_area': None}, │
│ {'symptom': 'slightly scaly', 'affected_area': None} │
│ ], │
│ 'current_meds': [ │
│ {'medication': 'OTC steroid cream', 'response': 'Moderate'} │
│ ] │
│ } │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

The prompt to the LLM was fully constructed by Guardrails. Additionally, the prompt only contained the previously invalid values for the affected_area field.

On receiving the corrected LLM output, Guardrails merges it with the previously invalid values to create a valid output. It then reruns the validation step to ensure that the output is valid, and returns the output to the user.

Bonus: See the full history of calls

Guardrails also logs the full history of calls made to the LLM. This can be useful for debugging purposes.

print(guard.history.last.tree)
Logs
├── ╭────────────────────────────────────────────────── Step 0 ───────────────────────────────────────────────────╮
│ │ ╭──────────────────────────────────────────────── Prompt ─────────────────────────────────────────────────╮
│ │ │ │
│ │ │ Given the following doctor's notes about a patient, please extract a dictionary that contains the │
│ │ │ patient's information. │
│ │ │ │
│ │ │ 49 y/o Male with chronic macular rash to face & hair, worse in beard, eyebrows & nares. │
│ │ │ Itchy, flaky, slightly scaly. Moderate response to OTC steroid cream │
│ │ │ │
│ │ │ │
│ │ │ Given below is XML that describes the information to extract from this document and the tags to extract │
│ │ │ it into. │
│ │ │ │
│ │ │ <output> │
│ │ │ <string name="gender" description="Patient's gender"/> │
│ │ │ <integer name="age" format="valid-range: min=0 max=100"/> │
│ │ │ <list name="symptoms" description="Symptoms that the patient is currently experiencing. Each │
│ │ │ symptom should be classified into a separate item in the list."> │
│ │ │ <object> │
│ │ │ <string name="symptom" description="Symptom that a patient is experiencing"/> │
│ │ │ <string name="affected_area" description="What part of the body the symptom is affecting" │
│ │ │ format="valid-choices: choices=['head', 'neck', 'chest']"/> │
│ │ │ </object> │
│ │ │ </list> │
│ │ │ <list name="current_meds" description="Medications the patient is currently taking and their │
│ │ │ response"> │
│ │ │ <object> │
│ │ │ <string name="medication" description="Name of the medication the patient is taking"/> │
│ │ │ <string name="response" description="How the patient is responding to the medication"/> │
│ │ │ </object> │
│ │ │ </list> │
│ │ │ </output> │
│ │ │ │
│ │ │ │
│ │ │ ONLY return a valid JSON object (no other text is necessary), where the key of the field in JSON is the │
│ │ │ `name` attribute of the corresponding XML, and the value is of the type specified by the corresponding │
│ │ │ XML's tag. The JSON MUST conform to the XML format, including any types and format requests e.g. │
│ │ │ requests for lists, objects and specific types. Be correct and concise. │
│ │ │ │
│ │ │ Here are examples of simple (XML, JSON) pairs that show the expected behavior: │
│ │ │ - `<string name='foo' format='two-words lower-case' />` => `{'foo': 'example one'}` │
│ │ │ - `<list name='bar'><string format='upper-case' /></list>` => `{"bar": ['STRING ONE', 'STRING TWO', │
│ │ │ etc.]}` │
│ │ │ - `<object name='baz'><string name="foo" format="capitalize two-words" /><integer name="index" │
│ │ │ format="1-indexed" /></object>` => `{'baz': {'foo': 'Some String', 'index': 1}}` │
│ │ │ │
│ │ │ │
│ │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯
│ │ ╭───────────────────────────────────────────── Instructions ──────────────────────────────────────────────╮
│ │ │ You are a helpful assistant, able to express yourself purely through JSON, strictly and precisely │
│ │ │ adhering to the provided XML schemas. │
│ │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯
│ │ ╭──────────────────────────────────────────── Message History ────────────────────────────────────────────╮
│ │ │ No message history. │
│ │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯
│ │ ╭──────────────────────────────────────────── Raw LLM Output ─────────────────────────────────────────────╮
│ │ │ { │
│ │ │ "gender": "Male", │
│ │ │ "age": 49, │
│ │ │ "symptoms": [ │
│ │ │ { │
│ │ │ "symptom": "chronic macular rash", │
│ │ │ "affected_area": "face" │
│ │ │ }, │
│ │ │ { │
│ │ │ "symptom": "itchy", │
│ │ │ "affected_area": "beard" │
│ │ │ }, │
│ │ │ { │
│ │ │ "symptom": "itchy", │
│ │ │ "affected_area": "eyebrows" │
│ │ │ }, │
│ │ │ { │
│ │ │ "symptom": "itchy", │
│ │ │ "affected_area": "nares" │
│ │ │ }, │
│ │ │ { │
│ │ │ "symptom": "flaky", │
│ │ │ "affected_area": "face" │
│ │ │ }, │
│ │ │ { │
│ │ │ "symptom": "slightly scaly", │
│ │ │ "affected_area": "face" │
│ │ │ } │
│ │ │ ], │
│ │ │ "current_meds": [ │
│ │ │ { │
│ │ │ "medication": "OTC steroid cream", │
│ │ │ "response": "Moderate" │
│ │ │ } │
│ │ │ ] │
│ │ │ } │
│ │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯
│ │ ╭─────────────────────────────────────────── Validated Output ────────────────────────────────────────────╮
│ │ │ { │
│ │ │ 'gender': 'Male', │
│ │ │ 'age': 49, │
│ │ │ 'symptoms': [ │
│ │ │ { │
│ │ │ 'symptom': 'chronic macular rash', │
│ │ │ 'affected_area': FieldReAsk( │
│ │ │ incorrect_value='face', │
│ │ │ fail_results=[ │
│ │ │ FailResult( │
│ │ │ outcome='fail', │
│ │ │ metadata=None, │
│ │ │ error_message="Value face is not in choices ['head', 'neck', 'chest'].", │
│ │ │ fix_value=None │
│ │ │ ) │
│ │ │ ], │
│ │ │ path=['symptoms', 0, 'affected_area'] │
│ │ │ ) │
│ │ │ }, │
│ │ │ { │
│ │ │ 'symptom': 'itchy', │
│ │ │ 'affected_area': FieldReAsk( │
│ │ │ incorrect_value='beard', │
│ │ │ fail_results=[ │
│ │ │ FailResult( │
│ │ │ outcome='fail', │
│ │ │ metadata=None, │
│ │ │ error_message="Value beard is not in choices ['head', 'neck', 'chest'].", │
│ │ │ fix_value=None │
│ │ │ ) │
│ │ │ ], │
│ │ │ path=['symptoms', 1, 'affected_area'] │
│ │ │ ) │
│ │ │ }, │
│ │ │ { │
│ │ │ 'symptom': 'itchy', │
│ │ │ 'affected_area': FieldReAsk( │
│ │ │ incorrect_value='eyebrows', │
│ │ │ fail_results=[ │
│ │ │ FailResult( │
│ │ │ outcome='fail', │
│ │ │ metadata=None, │
│ │ │ error_message="Value eyebrows is not in choices ['head', 'neck', 'chest'].", │
│ │ │ fix_value=None │
│ │ │ ) │
│ │ │ ], │
│ │ │ path=['symptoms', 2, 'affected_area'] │
│ │ │ ) │
│ │ │ }, │
│ │ │ { │
│ │ │ 'symptom': 'itchy', │
│ │ │ 'affected_area': FieldReAsk( │
│ │ │ incorrect_value='nares', │
│ │ │ fail_results=[ │
│ │ │ FailResult( │
│ │ │ outcome='fail', │
│ │ │ metadata=None, │
│ │ │ error_message="Value nares is not in choices ['head', 'neck', 'chest'].", │
│ │ │ fix_value=None │
│ │ │ ) │
│ │ │ ], │
│ │ │ path=['symptoms', 3, 'affected_area'] │
│ │ │ ) │
│ │ │ }, │
│ │ │ { │
│ │ │ 'symptom': 'flaky', │
│ │ │ 'affected_area': FieldReAsk( │
│ │ │ incorrect_value='face', │
│ │ │ fail_results=[ │
│ │ │ FailResult( │
│ │ │ outcome='fail', │
│ │ │ metadata=None, │
│ │ │ error_message="Value face is not in choices ['head', 'neck', 'chest'].", │
│ │ │ fix_value=None │
│ │ │ ) │
│ │ │ ], │
│ │ │ path=['symptoms', 4, 'affected_area'] │
│ │ │ ) │
│ │ │ }, │
│ │ │ { │
│ │ │ 'symptom': 'slightly scaly', │
│ │ │ 'affected_area': FieldReAsk( │
│ │ │ incorrect_value='face', │
│ │ │ fail_results=[ │
│ │ │ FailResult( │
│ │ │ outcome='fail', │
│ │ │ metadata=None, │
│ │ │ error_message="Value face is not in choices ['head', 'neck', 'chest'].", │
│ │ │ fix_value=None │
│ │ │ ) │
│ │ │ ], │
│ │ │ path=['symptoms', 5, 'affected_area'] │
│ │ │ ) │
│ │ │ } │
│ │ │ ], │
│ │ │ 'current_meds': [ │
│ │ │ {'medication': 'OTC steroid cream', 'response': 'Moderate'} │
│ │ │ ] │
│ │ │ } │
│ │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
└── ╭────────────────────────────────────────────────── Step 1 ───────────────────────────────────────────────────╮
╭──────────────────────────────────────────────── Prompt ─────────────────────────────────────────────────╮
│ │
│ I was given the following JSON response, which had problems due to incorrect values. │
│ │
│ { │
│ "gender": "Male", │
│ "age": 49, │
│ "symptoms": [ │
│ { │
│ "symptom": "chronic macular rash", │
│ "affected_area": { │
│ "incorrect_value": "face", │
│ "error_messages": [ │
│ "Value face is not in choices ['head', 'neck', 'chest']." │
│ ] │
│ } │
│ }, │
│ { │
│ "symptom": "itchy", │
│ "affected_area": { │
│ "incorrect_value": "beard", │
│ "error_messages": [ │
│ "Value beard is not in choices ['head', 'neck', 'chest']." │
│ ] │
│ } │
│ }, │
│ { │
│ "symptom": "itchy", │
│ "affected_area": { │
│ "incorrect_value": "eyebrows", │
│ "error_messages": [ │
│ "Value eyebrows is not in choices ['head', 'neck', 'chest']." │
│ ] │
│ } │
│ }, │
│ { │
│ "symptom": "itchy", │
│ "affected_area": { │
│ "incorrect_value": "nares", │
│ "error_messages": [ │
│ "Value nares is not in choices ['head', 'neck', 'chest']." │
│ ] │
│ } │
│ }, │
│ { │
│ "symptom": "flaky", │
│ "affected_area": { │
│ "incorrect_value": "face", │
│ "error_messages": [ │
│ "Value face is not in choices ['head', 'neck', 'chest']." │
│ ] │
│ } │
│ }, │
│ { │
│ "symptom": "slightly scaly", │
│ "affected_area": { │
│ "incorrect_value": "face", │
│ "error_messages": [ │
│ "Value face is not in choices ['head', 'neck', 'chest']." │
│ ] │
│ } │
│ } │
│ ], │
│ "current_meds": [ │
│ { │
│ "medication": "OTC steroid cream", │
│ "response": "Moderate" │
│ } │
│ ] │
│ } │
│ │
│ Help me correct the incorrect values based on the given error messages. │
│ │
│ Given below is XML that describes the information to extract from this document and the tags to extract │
│ it into. │
│ │
│ <output> │
│ <string name="gender" description="Patient's gender"/> │
│ <integer name="age" format="valid-range: min=0 max=100"/> │
│ <list name="symptoms" description="Symptoms that the patient is currently experiencing. Each │
│ symptom should be classified into a separate item in the list."> │
│ <object> │
│ <string name="symptom" description="Symptom that a patient is experiencing"/> │
│ <string name="affected_area" description="What part of the body the symptom is affecting" │
│ format="valid-choices: choices=['head', 'neck', 'chest']"/> │
│ </object> │
│ </list> │
│ <list name="current_meds" description="Medications the patient is currently taking and their │
│ response"> │
│ <object> │
│ <string name="medication" description="Name of the medication the patient is taking"/> │
│ <string name="response" description="How the patient is responding to the medication"/> │
│ </object> │
│ </list> │
│ </output> │
│ │
│ │
│ ONLY return a valid JSON object (no other text is necessary), where the key of the field in JSON is the │
│ `name` attribute of the corresponding XML, and the value is of the type specified by the corresponding │
│ XML's tag. The JSON MUST conform to the XML format, including any types and format requests e.g. │
│ requests for lists, objects and specific types. Be correct and concise. If you are unsure anywhere, │
│ enter `null`. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭───────────────────────────────────────────── Instructions ──────────────────────────────────────────────╮
│ You are a helpful assistant, able to express yourself purely through JSON, strictly and precisely │
│ adhering to the provided XML schemas. │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭──────────────────────────────────────────── Message History ────────────────────────────────────────────╮
│ No message history. │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭──────────────────────────────────────────── Raw LLM Output ─────────────────────────────────────────────╮
│ { │
│ "gender": "Male", │
│ "age": 49, │
│ "symptoms": [ │
│ { │
│ "symptom": "chronic macular rash", │
│ "affected_area": "head" │
│ }, │
│ { │
│ "symptom": "itchy", │
│ "affected_area": "null" │
│ }, │
│ { │
│ "symptom": "itchy", │
│ "affected_area": "null" │
│ }, │
│ { │
│ "symptom": "itchy", │
│ "affected_area": "null" │
│ }, │
│ { │
│ "symptom": "flaky", │
│ "affected_area": "null" │
│ }, │
│ { │
│ "symptom": "slightly scaly", │
│ "affected_area": "null" │
│ } │
│ ], │
│ "current_meds": [ │
│ { │
│ "medication": "OTC steroid cream", │
│ "response": "Moderate" │
│ } │
│ ] │
│ } │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────── Validated Output ────────────────────────────────────────────╮
│ { │
│ 'gender': 'Male', │
│ 'age': 49, │
│ 'symptoms': [ │
│ {'symptom': 'chronic macular rash', 'affected_area': 'head'}, │
│ {'symptom': 'itchy', 'affected_area': None}, │
│ {'symptom': 'itchy', 'affected_area': None}, │
│ {'symptom': 'itchy', 'affected_area': None}, │
│ {'symptom': 'flaky', 'affected_area': None}, │
│ {'symptom': 'slightly scaly', 'affected_area': None} │
│ ], │
│ 'current_meds': [ │
│ {'medication': 'OTC steroid cream', 'response': 'Moderate'} │
│ ] │
│ } │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────╯