Stream-validate LLM responses
# Few imports and global variables
from rich import print
import guardrails as gd
import openai
from IPython.display import clear_output
import time
Setup
Install the necessary validators from Guardrails hub in your CLI.
!guardrails hub install hub://guardrails/valid_range
!guardrails hub install hub://guardrails/uppercase
!guardrails hub install hub://guardrails/lowercase
!guardrails hub install hub://guardrails/one_line
1. For structured JSON output
Define the prompt and output schema
from pydantic import BaseModel, Field
from guardrails.hub import LowerCase, UpperCase, ValidRange, OneLine
from typing import List
prompt = """
Given the following doctor's notes about a patient, please extract a dictionary that contains the patient's information.
${doctors_notes}
${gr.complete_json_suffix_v2}
"""
doctors_notes = """152 y/o female with chronic macular rash to face and hair, worse in beard, eyebrows and nares.
The rash is itchy, flaky and slightly scaly. Moderate response to OTC steroid cream. Patient has been using cream for 2 weeks and also suffers from diabetes."""
class Symptom(BaseModel):
symptom: str = Field(description="Symptom that a patient is experiencing")
affected_area: str = Field(
description="What part of the body the symptom is affecting",
validators=[
LowerCase(on_fail="fix"),
],
)
class Medication(BaseModel):
medication: str = Field(
description="Name of the medication the patient is taking",
validators=[UpperCase(on_fail="fix")],
)
response: str = Field(description="How the patient is responding to the medication")
class PatientInfo(BaseModel):
gender: str = Field(description="Patient's gender")
age: int = Field(
description="Patient's age",
validators=[ValidRange(min=0, max=100, on_fail="fix")],
)
symptoms: List[Symptom] = Field(
description="Symptoms that the patient is currently experiencing. Each symptom should be classified into separate item in the list."
)
current_meds: List[Medication] = Field(
description="Medications the patient is currently taking and their response"
)
miscellaneous: str = Field(
description="Any other information that is relevant to the patient's health; something that doesn't fit into the other categories.",
validators=[LowerCase(on_fail="fix"), OneLine(on_fail="fix")],
)
/Users/zaydsimjee/workspace/guardrails/guardrails/validators/__init__.py:50: FutureWarning:
Importing validators from `guardrails.validators` is deprecated.
All validators are now available in the Guardrails Hub. Please install
and import them from the hub instead. All validators will be
removed from this module in the next major release.
Install with: `guardrails hub install hub://<namespace>/<validator_name>`
Import as: from guardrails.hub import `ValidatorName`
warn(
/Users/zaydsimjee/workspace/guardrails/guardrails/validator_base.py:397: FutureWarning: Accessing `LowerCase` using
`from guardrails.validators import LowerCase` is deprecated and
support will be removed after version 0.5.x. Please switch to the Guardrails Hub syntax:
`from guardrails.hub import LowerCase` for future updates and support.
For additional details, please visit: https://hub.guardrailsai.com/validator/guardrails/lowercase.
warn(
/Users/zaydsimjee/workspace/guardrails/guardrails/validator_base.py:397: FutureWarning: Accessing `UpperCase` using
`from guardrails.validators import UpperCase` is deprecated and
support will be removed after version 0.5.x. Please switch to the Guardrails Hub syntax:
`from guardrails.hub import UpperCase` for future updates and support.
For additional details, please visit: https://hub.guardrailsai.com/validator/guardrails/uppercase.
warn(
/Users/zaydsimjee/workspace/guardrails/guardrails/validator_base.py:397: FutureWarning: Accessing `ValidRange` using
`from guardrails.validators import ValidRange` is deprecated and
support will be removed after version 0.5.x. Please switch to the Guardrails Hub syntax:
`from guardrails.hub import ValidRange` for future updates and support.
For additional details, please visit: https://hub.guardrailsai.com/validator/guardrails/valid_range.
warn(
/Users/zaydsimjee/workspace/guardrails/guardrails/validator_base.py:397: FutureWarning: Accessing `OneLine` using
`from guardrails.validators import OneLine` is deprecated and
support will be removed after version 0.5.x. Please switch to the Guardrails Hub syntax:
`from guardrails.hub import OneLine` for future updates and support.
For additional details, please visit: https://hub.guardrailsai.com/validator/guardrails/one_line.
warn(
Create the Guard object
guard = gd.Guard.from_pydantic(output_class=PatientInfo, prompt=prompt)
Example 1: No streaming
By default, the stream parameter is set to False
# Wrap the OpenAI API call with the `guard` object
raw_llm_output, validated_output, *rest = guard(
openai.chat.completions.create,
prompt_params={"doctors_notes": doctors_notes},
max_tokens=1024,
temperature=0.3,
)
# Print the validated output from the LLM
print(validated_output)
HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
{
'gender': 'female',
'age': 100,
'symptoms': [
{'symptom': 'chronic macular rash', 'affected_area': 'face and hair, worse in beard, eyebrows and nares'},
{'symptom': 'itchy', 'affected_area': 'whole body'},
{'symptom': 'flaky', 'affected_area': 'whole body'},
{'symptom': 'slightly scaly', 'affected_area': 'whole body'}
],
'current_meds': [{'medication': 'OTC STEROID CREAM', 'response': 'moderate response'}],
'miscellaneous': 'patient has been using cream for 2 weeks and also suffers from diabetes'
}
# Let's see the logs
print(guard.history.last.tree)
Logs
└── ╭────────────────────────────────────────────────── Step 0 ───────────────────────────────────────────────────╮
│ ╭──────────────────────────────────────────────── Prompt ─────────────────────────────────────────────────╮ │
│ │ │ │
│ │ Given the following doctor's notes about a patient, please extract a dictionary that contains the │ │
│ │ patient's information. │ │
│ │ │ │
│ │ 152 y/o female with chronic macular rash to face and hair, worse in beard, eyebrows and nares. │ │
│ │ The rash is itchy, flaky and slightly scaly. Moderate response to OTC steroid cream. Patient has been │ │
│ │ using cream for 2 weeks and also suffers from diabetes. │ │
│ │ │ │
│ │ │ │
│ │ Given below is XML that describes the information to extract from this document and the tags to extract │ │
│ │ it into. │ │
│ │ │ │
│ │ <output> │ │
│ │ <string name="gender" description="Patient's gender"/> │ │
│ │ <integer name="age" description="Patient's age" format="valid-range: min=0 max=100"/> │ │
│ │ <list name="symptoms" description="Symptoms that the patient is currently experiencing. Each │ │
│ │ symptom should be classified into separate item in the list."> │ │
│ │ <object> │ │
│ │ <string name="symptom" description="Symptom that a patient is experiencing"/> │ │
│ │ <string name="affected_area" description="What part of the body the symptom is affecting" │ │
│ │ format="lower-case"/> │ │
│ │ </object> │ │
│ │ </list> │ │
│ │ <list name="current_meds" description="Medications the patient is currently taking and their │ │
│ │ response"> │ │
│ │ <object> │ │
│ │ <string name="medication" description="Name of the medication the patient is taking" │ │
│ │ format="upper-case"/> │ │
│ │ <string name="response" description="How the patient is responding to the medication"/> │ │
│ │ </object> │ │
│ │ </list> │ │
│ │ <string name="miscellaneous" description="Any other information that is relevant to the patient's │ │
│ │ health; something that doesn't fit into the other categories." format="lower-case; one-line"/> │ │
│ │ </output> │ │
│ │ │ │
│ │ │ │
│ │ ONLY return a valid JSON object (no other text is necessary), where the key of the field in JSON is the │ │
│ │ `name` attribute of the corresponding XML, and the value is of the type specified by the corresponding │ │
│ │ XML's tag. The JSON MUST conform to the XML format, including any types and format requests e.g. │ │
│ │ requests for lists, objects and specific types. Be correct and concise. │ │
│ │ │ │
│ │ Here are examples of simple (XML, JSON) pairs that show the expected behavior: │ │
│ │ - `<string name='foo' format='two-words lower-case' />` => `{'foo': 'example one'}` │ │
│ │ - `<list name='bar'><string format='upper-case' /></list>` => `{"bar": ['STRING ONE', 'STRING TWO', │ │
│ │ etc.]}` │ │
│ │ - `<object name='baz'><string name="foo" format="capitalize two-words" /><integer name="index" │ │
│ │ format="1-indexed" /></object>` => `{'baz': {'foo': 'Some String', 'index': 1}}` │ │
│ │ │ │
│ │ │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ ╭───────────────────────────────────────────── Instructions ──────────────────────────────────────────────╮ │
│ │ You are a helpful assistant, able to express yourself purely through JSON, strictly and precisely │ │
│ │ adhering to the provided XML schemas. │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ ╭──────────────────────────────────────────── Message History ────────────────────────────────────────────╮ │
│ │ No message history. │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ ╭──────────────────────────────────────────── Raw LLM Output ─────────────────────────────────────────────╮ │
│ │ {"gender":"female","age":152,"symptoms":[{"symptom":"chronic macular rash","affected_area":"face and │ │
│ │ hair, worse in beard, eyebrows and nares"},{"symptom":"itchy","affected_area":"whole │ │
│ │ body"},{"symptom":"flaky","affected_area":"whole body"},{"symptom":"slightly │ │
│ │ scaly","affected_area":"whole body"}],"current_meds":[{"medication":"OTC steroid │ │
│ │ cream","response":"moderate response"}],"miscellaneous":"patient has been using cream for 2 weeks and │ │
│ │ also suffers from diabetes"} │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ ╭─────────────────────────────────────────── Validated Output ────────────────────────────────────────────╮ │
│ │ { │ │
│ │ 'gender': 'female', │ │
│ │ 'age': 100, │ │
│ │ 'symptoms': [ │ │
│ │ { │ │
│ │ 'symptom': 'chronic macular rash', │ │
│ │ 'affected_area': 'face and hair, worse in beard, eyebrows and nares' │ │
│ │ }, │ │
│ │ {'symptom': 'itchy', 'affected_area': 'whole body'}, │ │
│ │ {'symptom': 'flaky', 'affected_area': 'whole body'}, │ │
│ │ {'symptom': 'slightly scaly', 'affected_area': 'whole body'} │ │
│ │ ], │ │
│ │ 'current_meds': [ │ │
│ │ {'medication': 'OTC STEROID CREAM', 'response': 'moderate response'} │ │
│ │ ], │ │
│ │ 'miscellaneous': 'patient has been using cream for 2 weeks and also suffers from diabetes' │ │
│ │ } │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Example 2: Streaming
Set the stream parameter to True
# Wrap the OpenAI API call with the `guard` object
fragment_generator = guard(
openai.chat.completions.create,
prompt_params={"doctors_notes": doctors_notes},
max_tokens=1024,
temperature=0,
stream=True,
)
for op in fragment_generator:
clear_output(wait=True)
print(op)
time.sleep(0.5)
ValidationOutcome(
raw_llm_output='{ "gender": "female", "age": 52, "symptoms": [ {
"symptom": "chronic macular rash", "affected_area": "face" }, {
"symptom": "chronic macular rash", "affected_area": "hair" }, {
"symptom": "itchy rash", "affected_area": "beard" }, { "symptom":
"itchy rash", "affected_area": "eyebrows" }, { "symptom": "itchy
rash", "affected_area": "nares" }, { "symptom": "flaky rash",
"affected_area": "face" }, { "symptom": "flaky rash", "affected_area":
"hair" }, { "symptom": "slightly scaly rash", "affected_area": "face"
}, { "symptom": "slightly scaly rash", "affected_area": "hair" }
], "current_meds": [ { "medication": "OTC STEROID CREAM", "response":
"moderate response" } ], "miscellaneous": "patient also suffers from diabetes" }',
validated_output={
'gender': 'female',
'age': 52,
'symptoms': [
{'symptom': 'chronic macular rash', 'affected_area': 'face'},
{'symptom': 'chronic macular rash', 'affected_area': 'hair'},
{'symptom': 'itchy rash', 'affected_area': 'beard'},
{'symptom': 'itchy rash', 'affected_area': 'eyebrows'},
{'symptom': 'itchy rash', 'affected_area': 'nares'},
{'symptom': 'flaky rash', 'affected_area': 'face'},
{'symptom': 'flaky rash', 'affected_area': 'hair'},
{'symptom': 'slightly scaly rash', 'affected_area': 'face'},
{'symptom': 'slightly scaly rash', 'affected_area': 'hair'}
],
'current_meds': [{'medication': 'OTC STEROID CREAM', 'response': 'moderate response'}],
'miscellaneous': 'patient also suffers from diabetes'
},
reask=None,
validation_passed=True,
error=None
)
# Let's see the logs
print(guard.history.last.tree)
Logs
└── ╭────────────────────────────────────────────────── Step 0 ───────────────────────────────────────────────────╮
│ ╭──────────────────────────────────────────────── Prompt ─────────────────────────────────────────────────╮ │
│ │ │ │
│ │ Given the following doctor's notes about a patient, please extract a dictionary that contains the │ │
│ │ patient's information. │ │
│ │ │ │
│ │ 152 y/o female with chronic macular rash to face and hair, worse in beard, eyebrows and nares. │ │
│ │ The rash is itchy, flaky and slightly scaly. Moderate response to OTC steroid cream. Patient has been │ │
│ │ using cream for 2 weeks and also suffers from diabetes. │ │
│ │ │ │
│ │ │ │
│ │ Given below is XML that describes the information to extract from this document and the tags to extract │ │
│ │ it into. │ │
│ │ │ │
│ │ <output> │ │
│ │ <string name="gender" description="Patient's gender"/> │ │
│ │ <integer name="age" description="Patient's age" format="valid-range: min=0 max=100"/> │ │
│ │ <list name="symptoms" description="Symptoms that the patient is currently experiencing. Each │ │
│ │ symptom should be classified into separate item in the list."> │ │
│ │ <object> │ │
│ │ <string name="symptom" description="Symptom that a patient is experiencing"/> │ │
│ │ <string name="affected_area" description="What part of the body the symptom is affecting" │ │
│ │ format="lower-case"/> │ │
│ │ </object> │ │
│ │ </list> │ │
│ │ <list name="current_meds" description="Medications the patient is currently taking and their │ │
│ │ response"> │ │
│ │ <object> │ │
│ │ <string name="medication" description="Name of the medication the patient is taking" │ │
│ │ format="upper-case"/> │ │
│ │ <string name="response" description="How the patient is responding to the medication"/> │ │
│ │ </object> │ │
│ │ </list> │ │
│ │ <string name="miscellaneous" description="Any other information that is relevant to the patient's │ │
│ │ health; something that doesn't fit into the other categories." format="lower-case; one-line"/> │ │
│ │ </output> │ │
│ │ │ │
│ │ │ │
│ │ ONLY return a valid JSON object (no other text is necessary), where the key of the field in JSON is the │ │
│ │ `name` attribute of the corresponding XML, and the value is of the type specified by the corresponding │ │
│ │ XML's tag. The JSON MUST conform to the XML format, including any types and format requests e.g. │ │
│ │ requests for lists, objects and specific types. Be correct and concise. │ │
│ │ │ │
│ │ Here are examples of simple (XML, JSON) pairs that show the expected behavior: │ │
│ │ - `<string name='foo' format='two-words lower-case' />` => `{'foo': 'example one'}` │ │
│ │ - `<list name='bar'><string format='upper-case' /></list>` => `{"bar": ['STRING ONE', 'STRING TWO', │ │
│ │ etc.]}` │ │
│ │ - `<object name='baz'><string name="foo" format="capitalize two-words" /><integer name="index" │ │
│ │ format="1-indexed" /></object>` => `{'baz': {'foo': 'Some String', 'index': 1}}` │ │
│ │ │ │
│ │ │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ ╭───────────────────────────────────────────── Instructions ──────────────────────────────────────────────╮ │
│ │ You are a helpful assistant, able to express yourself purely through JSON, strictly and precisely │ │
│ │ adhering to the provided XML schemas. │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ ╭──────────────────────────────────────────── Message History ────────────────────────────────────────────╮ │
│ │ No message history. │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ ╭──────────────────────────────────────────── Raw LLM Output ─────────────────────────────────────────────╮ │
│ │ { │ │
│ │ "gender": "female", │ │
│ │ "age": 52, │ │
│ │ "symptoms": [ │ │
│ │ { │ │
│ │ "symptom": "chronic macular rash", │ │
│ │ "affected_area": "face" │ │
│ │ }, │ │
│ │ { │ │
│ │ "symptom": "chronic macular rash", │ │
│ │ "affected_area": "hair" │ │
│ │ }, │ │
│ │ { │ │
│ │ "symptom": "itchy rash", │ │
│ │ "affected_area": "beard" │ │
│ │ }, │ │
│ │ { │ │
│ │ "symptom": "itchy rash", │ │
│ │ "affected_area": "eyebrows" │ │
│ │ }, │ │
│ │ { │ │
│ │ "symptom": "itchy rash", │ │
│ │ "affected_area": "nares" │ │
│ │ }, │ │
│ │ { │ │
│ │ "symptom": "flaky rash", │ │
│ │ "affected_area": "face" │ │
│ │ }, │ │
│ │ { │ │
│ │ "symptom": "flaky rash", │ │
│ │ "affected_area": "hair" │ │
│ │ }, │ │
│ │ { │ │
│ │ "symptom": "slightly scaly rash", │ │
│ │ "affected_area": "face" │ │
│ │ }, │ │
│ │ { │ │
│ │ "symptom": "slightly scaly rash", │ │
│ │ "affected_area": "hair" │ │
│ │ } │ │
│ │ ], │ │
│ │ "current_meds": [ │ │
│ │ { │ │
│ │ "medication": "OTC STEROID CREAM", │ │
│ │ "response": "moderate response" │ │
│ │ } │ │
│ │ ], │ │
│ │ "miscellaneous": "patient also suffers from diabetes" │ │
│ │ } │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ ╭─────────────────────────────────────────── Validated Output ────────────────────────────────────────────╮ │
│ │ { │ │
│ │ 'gender': 'female', │ │
│ │ 'age': 52, │ │
│ │ 'symptoms': [ │ │
│ │ {'symptom': 'chronic macular rash', 'affected_area': 'face'}, │ │
│ │ {'symptom': 'chronic macular rash', 'affected_area': 'hair'}, │ │
│ │ {'symptom': 'itchy rash', 'affected_area': 'beard'}, │ │
│ │ {'symptom': 'itchy rash', 'affected_area': 'eyebrows'}, │ │
│ │ {'symptom': 'itchy rash', 'affected_area': 'nares'}, │ │
│ │ {'symptom': 'flaky rash', 'affected_area': 'face'}, │ │
│ │ {'symptom': 'flaky rash', 'affected_area': 'hair'}, │ │
│ │ {'symptom': 'slightly scaly rash', 'affected_area': 'face'}, │ │
│ │ {'symptom': 'slightly scaly rash', 'affected_area': 'hair'} │ │
│ │ ], │ │
│ │ 'current_meds': [ │ │
│ │ {'medication': 'OTC STEROID CREAM', 'response': 'moderate response'} │ │
│ │ ], │ │
│ │ 'miscellaneous': 'patient also suffers from diabetes' │ │
│ │ } │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
As you can see here, the outputs in both examples match. The only difference is that, in the streaming example, the outputs are returned as soon as they are received and validated by Guardrails. In the non-streaming example, the outputs are returned only after the entire request has been processed by the API. In other words, when streaming is enabled, the API returns the outputs as soon as they are ready, rather than waiting for the entire request to be processed.
2. For unstructured text output
Define the prompt and Guard object with validators
from guardrails.hub import UpperCase, OneLine
prompt = """
Generate a short description of large language models. Each new sentence should be on another line.
"""
guard = gd.Guard.from_string(
validators=[
UpperCase(on_fail="fix"),
OneLine(on_fail="fix"),
],
description="testmeout",
prompt=prompt,
)
/Users/zaydsimjee/workspace/guardrails/guardrails/validator_base.py:397: FutureWarning: Accessing `UpperCase` using
`from guardrails.validators import UpperCase` is deprecated and
support will be removed after version 0.5.x. Please switch to the Guardrails Hub syntax:
`from guardrails.hub import UpperCase` for future updates and support.
For additional details, please visit: https://hub.guardrailsai.com/validator/guardrails/uppercase.
warn(
/Users/zaydsimjee/workspace/guardrails/guardrails/validator_base.py:397: FutureWarning: Accessing `OneLine` using
`from guardrails.validators import OneLine` is deprecated and
support will be removed after version 0.5.x. Please switch to the Guardrails Hub syntax:
`from guardrails.hub import OneLine` for future updates and support.
For additional details, please visit: https://hub.guardrailsai.com/validator/guardrails/one_line.
warn(
Example 1: No streaming
By default, the stream parameter is set to False
# Wrap the OpenAI API call with the `guard` object
raw, validated, *rest = guard(
openai.chat.completions.create,
max_tokens=50,
temperature=0.1,
)
# Print the raw and validated outputs
print(f"Raw output:\n{raw}")
print(f"Validated output:\n{validated}")
HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
```
Raw output:
Large language models are powerful artificial intelligence systems that can generate human-like text.
They are trained on vast amounts of data to understand and generate language.
These models have the ability to complete sentences, write stories, and even engage in conversations.
```
Validated output:
LARGE LANGUAGE MODELS ARE POWERFUL ARTIFICIAL INTELLIGENCE SYSTEMS THAT CAN GENERATE HUMAN-LIKE TEXT.
Example 2: With streaming
Set the stream parameter to True
# Wrap the OpenAI API call with the `guard` object
fragment_generator = guard(
openai.chat.completions.create,
max_tokens=50,
temperature=0.1,
stream=True,
)
for op in fragment_generator:
clear_output(wait=True)
print(op)
time.sleep(0.1)
ValidationOutcome(
raw_llm_output='Large language models are advanced artificial intelligence systems that can generate human-like
text. They are trained on vast amounts of data to understand and produce language. These models have the
ability to complete sentences, write stories, and even engage in conversations. ',
validated_output='LARGE LANGUAGE MODELS ARE ADVANCED ARTIFICIAL INTELLIGENCE SYSTEMS THAT CAN GENERATE
HUMAN-LIKE TEXT. ',
reask=None,
validation_passed=True,
error=None
)
# See guard history
print(guard.history.last.tree)
Logs
└── ╭────────────────────────────────────────────────── Step 0 ───────────────────────────────────────────────────╮
│ ╭──────────────────────────────────────────────── Prompt ─────────────────────────────────────────────────╮ │
│ │ │ │
│ │ Generate a short description of large language models. Each new sentence should be on another line. │ │
│ │ │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ ╭───────────────────────────────────────────── Instructions ──────────────────────────────────────────────╮ │
│ │ You are a helpful assistant, expressing yourself through a string. │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ ╭──────────────────────────────────────────── Message History ────────────────────────────────────────────╮ │
│ │ No message history. │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ ╭──────────────────────────────────────────── Raw LLM Output ─────────────────────────────────────────────╮ │
│ │ Large language models are advanced artificial intelligence systems that can generate human-like text. │ │
│ │ They are trained on vast amounts of data to understand and produce language. │ │
│ │ These models have the ability to complete sentences, write stories, and even engage in conversations. │ │
│ │ │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ ╭─────────────────────────────────────────── Validated Output ────────────────────────────────────────────╮ │
│ │ 'LARGE LANGUAGE MODELS ARE ADVANCED ARTIFICIAL INTELLIGENCE SYSTEMS THAT CAN GENERATE HUMAN-LIKE TEXT. │ │
│ │ ' │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
As you can see, the outputs in both examples match. The only difference is that, in the streaming example, the outputs are returned as soon as they are received and validated by Guardrails. In the non-streaming example, the outputs are returned only after the entire request has been processed by the API. In other words, when streaming is enabled, the API returns the outputs as soon as they are ready, rather than waiting for the entire request to be processed.
This also works with openai's chat completion API.