Skip to main content

2 posts tagged with "tutorial"

View All Tags

· 9 min read
Safeer Mohiuddin


Hallucination can be a serious issue when generating data using AI. In this article, we’ll show how you can use the Guardrails AI provenance validators to reduce hallucinations for more accurate results.

The hallucination problem in AI

In our last article, we showed how to use Guardrails AI to generate more realistic synthetic structured data using a Large Language Model (LLM). Structured data is great when we need machines - e.g., different services in an application service fabric - to communicate with one another. For data displayed to humans, we generally want to generate unstructured data.

AI engines such as ChatGPT can generate unstructured text with high fidelity to real human conversations. However, when an AI engine doesn't know something, or it will sometimes generate incorrect information that seems/look like a fact due to the nature of the generated language. This can happen in both structured and unstructured data.

These hallucinations can have negative real-world consequences. Just ask the attorneys who filed court documents that included fictitious citations generated by ChatGPT.

Common strategies to address hallucinations

Solutions to the hallucination problem are still evolving. Common ways to do so today are providing more context in prompts, fine tuning, retrieval augmented generation (RAG) and even building your own models.

Retrieval augmented generation (RAG) enables LLM applications to leverage vector embeddings from source documents. It is quickly becoming the most common way to reduce hallucinations but many still face challenges despite implementing this. Guardrails AI provenance validators leverage embeeddings from RAG systems to even further improve accuracy and reduce hallucinations.

Using Guardrails AI provenance validators to automate hallucination reduction

Another way to reduce hallucinations is to use Guardrails AI. Guardrails AI provides a rich library of validators you can leverage programmatically to test that AI output meets a specific set of criteria.

Guardrails AI’s provenance validators detect and limit hallucinations from LLMs that depart from the material in the supplied source documents. We provide two provenance validators, both of which can operate on either the whole text or on each sentence of the text in turn.

In the following example, we’ll show how to use these validators, provenance v0 and provenance v1, to operate over a text selection on a sentence-by-sentence basis in Python. In other words, the validators will inspect each sentence for potential hallucinations and remove the parts that have no foundation in the source text.


To run the code below, you’ll need to do the following:

Install Python prerequisites

You can run this from Jupyter or your Python command line:

pip install -U guardrails-ai cohere openai numpy nltk -q

Obtain Cohere API key (Provenance v0 example)

Guardrails AI can operate against any LLM in the marketplace. In this article, we'll use both Cohere and OpenAI.

You can obtain a Cohere API key for free by signing up for a Cohere account. Once registered, from your Cohere Dashboard, go to Settings -> API Keys to request a new key.

Cohere Signup

Obtain OpenAI key (Provenance v1 example)

To obtain a free OpenAI key, create an account. When asked to choose between using ChatGPT and API, select API.

OpenAI Signup

From there, select your account icon in the upper right corner and then select View API keys. Select Create new secret key to generate an API key.

Make sure to copy and save the value somewhere, as you will not be able to copy it again after this. (You can always generate a new API key if you lose the old one.)

Using provenance v0 to confirm a response is supported by the source text

First, let's see how to use provenance v0 to run sentence-by-sentence confirmation to ensure a response from an LLM is supported by the source text. In this example, we'll use Cohere's Embed model, which supports many Natural Language Processing (NLP) use cases, to compare a set of sources against previous output from another LLM.

First, we'll start with our source text, which is advice on how to litter train a cat. It's a lot of text to include in the article, so we've defined it in a GitHub Gist here. Import this into your Jupyter notebook and run it.

Next, we'll create an embed function for Cohere. This Python function will take our source above and embed it into Cohere. Embedding renders our sentences as numerical values whose values are close to one another if they are similar but far apart if they are dissimilar. They enable finding semantically similar sentences. Using embeddings supplements Cohere's training with your own data so you don't need to build your own Large Language Model from scratch.

(Make sure to replace the COHERE_API_KEY value below with your own API key. If committing source code, replace this value with a reference to a secure form of storage, such as a secrets vault.)

# Create an embedding function that uses a cohere model to embed the text
from rich import print
import cohere
import numpy as np
from guardrails import Guard
from guardrails.validators import ProvenanceV0
from typing import List, Union

# Create a cohere client
cohere_client = cohere.Client(api_key=<COHERE_API_KEY>)

def embed_function(text: Union[str, List[str]]) -> np.ndarray:
"""Embed the text using cohere's small model."""
if isinstance(text, str): # If text is a string, wrap it in a list
text = [text]

response = cohere_client.embed(
embeddings_list = response.embeddings
return np.array(embeddings_list)

Note that we’re only defining the function here. Before calling it, we will initialize a Guard object in Guardrails and pass it a provenance v0 validator.

# Initialize the guard object
guard = Guard.from_string(
ProvenanceV0(threshold=0.3, validation_method="sentence", on_fail="fix")

For the validator, we set three values:

  • threshold: How sensitive we want the validator to be in detecting variance or hallucinations. In this case, we set it to 0.3, which makes it highly sensitive. 
  • validation_method: Set to sentence to examine each sentence of the checked text for accuracy.
  • on_fail: What to do if Guardrails AI determines the LLM output is hallucinated. fix means Guardrails AI will remove the hallucinated sentences.

Finally, we call our Guard with the LLM output we want to check against our sources. In this case, we're supplying a chunk of text on what another LLM thinks the best way to litter train a cat is.

Guardrails AI will take this text and chunk it down into smaller units before embedding it into Cohere. Then, it will use Cohere's embeddings of our sources plus its own internal logic to determine which parts of the text are accurate and which might be hallucinated.

llm_output="To retrain a cat to use the litter box, put its litter box in a low traffic area with at least 2 exits so the cat doesn’t feel cornered. Find the right litterbox for your cat, as well as the right litter. Place the litterbox in a good spot, away from heavily trafficked and noisy areas. Keep the litterbox very clean, and do not punish the cat or confine her to just one room. Once training is complete, it is important to clean the litter box regularly and to provide a safe and healthy environment for the cat.",
metadata={"embed_function": embed_function, "sources": sources},

# See the guard logs

In this example, a good chunk of the LLM output corresponded with our source text. However, as you can see below, Guardrails AI deemed one sentence as suspect and stripped it out.


Using provenance v1 to perform LLM self-checks

The provenance v1 validator checks whether the LLM output is hallucinated by prompting a second LLM. This work similarly to the case above but with a few minor differences.

First, we initialize a provenance v1 evaluator. The llm_callable parameter here can take two types of arguments:

  • An OpenAI ChatCompletion model, such as gpt-3.5-turbo or gpt-4.
  • A Python callable function that calls back to an LLM of your choice.\ In the example below, we use gpt-3.5-turbo. We use a top_k value of 3 to determine how many of the most statistically relevant chunks we should retrieve. A low value of 3 restricts output to the most relevant chunks. The other parameters are the same as the ones we used for provenance v0.
# Initialize the guard object
from guardrails.validators import ProvenanceV1

guard_1 = Guard.from_string(
validation_method="sentence", # can be "sentence" or "full"
llm_callable="gpt-3.5-turbo", # as explained above
top_k=3, # number of chunks to retrieve

Next, we'll call Guardrails AI's parse function as we did in the example above, passing our sources as additional context to the AI engine.

# Use .parse directly when we already have the LLM output
llm_output="To retrain a cat to use the litter box, put its litter box in a low traffic area with at least 2 exits so the cat doesn’t feel cornered. Find the right litterbox for your cat, as well as the right litter. Place the litterbox in a good spot, away from heavily trafficked and noisy areas. Keep the litterbox very clean, and do not punish the cat or confine her to just one room. Once training is complete, it is important to clean the litter box regularly and to provide a safe and healthy environment for the cat.",
metadata={"embed_function": embed_function, "sources": sources},
api_key=<OPENAI_API_KEY>, # OpenAI API key if using an OpenAI model for 'llm_callable', OR using one of the above options

Note that we also pass an api_key method with our OpenAI key. You can omit this hard-coding by setting the environment variable OPENAI_API_KEY:

os.environ["OPENAI_API_KEY"] = "<OpenAI_API_KEY>"

Finally, we print out the result of the call:

# See the guard logs

In this example, our statements about litter training cats comport both with our source data and OpenAI’s own knowledge. So the response is little changed from the other LLM’s output.


But let’s see what happens when we pass in input that’s not quite as accurate:

guardrail_output = guard_1.parse(
llm_output="Cats love lollipops and other sweet and sour foods. Cats can be retrained to use a litter box. Reward them when they do with lots of candy.",
metadata={"embed_function": embed_function, "sources": sources},

Here, whatever LLM we used previously asserts that cats can be bribed with candy. Guardrails AI flags this obvious falsehood and whittles the response down to the one statement that appears to be factual (if not very useful):



Hallucinations in AI output undermine user’s trust in AI-powered solutions. Using Guardrails AI’s provenance validators, you can leverage original sources and the power of other LLM engines to reduce inaccuracies and improve output quality.

· 10 min read
Safeer Mohiuddin


Generating synthetic structured data is critical for training your AI models. But getting AI engines to produce structured data isn’t always easy. This article will show how you can easily use Cohere and Guardrails to produce synthetic structured data.

What is synthetic structured data?

Structured data is any data put into a format that machines can easily parse, manage, and analyze.

Structured data abounds in traditional applications. Database tables, XML files, and JSON files are common examples of structured data software developers use daily. Structured data is also a good way for models in a complex AI-based application to exchange data.

Synthetic data is fictional data that is statistically or mathematically similar to real data. Companies, particularly in fields such as finance and healthcare, are increasingly using synthetic data to train their AI models. Recent research shows that, on top of being cheaper to produce, synthetic data may create AI models that are just as good, if not better, than models trained on real-world data.

Generating synthetic structured data with Cohere and Guardrails AI

Generating synthetic structured data using today’s AI engines can be a challenge. By default, a Large Language Model (LLM) outputs unstructured text. So, how do you coach it to return properly structured synthetic text?

Using Cohere and Guardrails together, you can generate synthetic structured text with high realism and accuracy. Cohere’s Command model provide generation capabilities for structured data. Guardrails AI adds structural and quality assurances that refine Cohere’s output, resulting in more accurate, realistic data.

Let’s discuss how the Cohere and Guardrails AI portions work, then see how they work even better together.

How Cohere works

Cohere offers access to cutting-edge LLMs through a simple API. It provides a variety of API endpoints to use depending on your use case, including Chat, Generate, Embed, Rerank, Semantic Search, Rerank, and Classify.

Cohere’s models power a variety of use cases, including running interactive chatbots, generating text for product descriptions or blog articles, moderating content, and recognizing intent. Companies can leverage Cohere’s LLMs in their apps without needing to train their own AI models from the ground up.

How Guardrails AI works

Guardrails AI is a Python package you can use to enhance the outputs of LLMs by adding structural, type, and quality assurance checks.

Guardrails AI leverages the pydantic format, one of the industry's most widely used data validation libraries. Guardrails checks for defects such as bias in generated text and bugs in generated code. Guardrails also enforces structural and type guarantees (e.g., returning proper JSON formatting) and takes corrective actions, such as prompt submission retries, when validation fails.

Creating a Guardrail for LLM output is a three-step process: Guardrails Spec

  1. Create a data structure spec. You can create a spec either using a Pydantic model or using RAIL. RAIL (Reliable AI Markup Language) is a language-agnostic, human-readable XML dialect for defining the expected structure and type from the LLM, as well as any validators and corrective actions. For the example below, we will use Pydantic.  

  2. Create a guard from the spec. The Python gd.Guard object serves as the basic executable unit for calls to the LLM.

  3. Wrap the LLM call with the guard. The guard combines the spec and the call to the LLM in order to validate, structure, and correct its outputs.

Walkthrough: Generating structured data with Cohere and Guardrails AI

Now, let’s see how to use these two technologies to create highly realistic synthetic structured data.


  • Python 3 installed on a dev machine with the latest version of Pip 
  • Cohere account and a Cohere API key

Generating data with Cohere

First, to get started with Cohere, sign up and generate an API key.

Cohere Signup

We'll use Cohere's Generate endpoint (co.generate) to generate realistic text conditioned on a given input. Cohere supports a REST API that developers can call from any programming language. In this walkthrough, we'll use Cohere's official Python SDK.

Start by installing the Python library for Cohere on your dev box:

pip install cohere

Next, write a simple Cohere application to generate structured data in JSON format:

import cohere

co = cohere.Client(api_key='<API_KEY>')

response = co.generate(
prompt='Generate different structured data and render it in JSON format',


Let's step through this line by line to understand what's going on. 

  1. import cohere: Standard Python import call.

  2. co = cohere.Client(api_key='<API KEY>'): Create a client object named co to interact with the Cohere API. It uses the provided API key to authenticate the requests. The API key is essential for accessing the Cohere services.

    (Note: Remember never to check API secrets directly into source code or leave them in Notebooks! Use a secrets vault, such as AWS Secrets Manager, for secure storage and retrieval of secrets.)

  3. response = co.generate(...): This line sends a text generation request to the Cohere API using the generate method. It provides several parameters as input for the text generation task:

    • model='command': Specifies the type of language model to use for text generation. In this case, it uses the "command" model.
    • prompt='Generate different structured data and render it in JSON format': Contains the input text prompt that serves as a starting point for text generation. The language model will generate text based on this prompt.
    • max_tokens=300: Sets the maximum number of tokens (words or subwords) the generated text should contain. This is used to limit the length of the generated response. Longer responses take more computational power to process, which increases application costs. 
    • temperature=0.9: Controls the randomness of the generated text. Higher values (e.g., 1.0) make the output more diverse, while lower values (e.g., 0.2) make it more deterministic.
    • k=0: The number of top-k candidates to sample from during text generation. Setting it to 0 means it will consider all candidates.
    • stop_sequences=[]: A list of strings that will stop the text generation if encountered. However, the list is empty, so the generation will continue until it reaches the max_tokens limit.
    • return_likelihoods='NONE': Specifies whether to return the likelihoods of each generated candidate. In this case, it is set to 'NONE', meaning it won't return likelihoods.
  4. print(response): This line prints the entire response object returned by the Cohere API after text generation. The response may contain various information, such as the generated text, the likelihoods, and other metadata.

  5. print('Prediction: {}'.format(response.generations[0].text)): This line prints the generated text obtained from the response. The response.generations attribute is a list of generated text candidates, and response.generations[0].text retrieves the first candidate's text. The format function includes the generated text in the output string, labeled as "Prediction."

Cohere Code

Adding guards with Guardrails AI

Now, let’s improve the quality of Cohere’s response by adding a guard. Install Guardrails AI and other required dependencies locally using pip:

pip install guardrails-ai pydantic typing openai rich

Create a new Jupyter Notebook entry that imports the Guardrails AI library:

import guardrails as gd

We will define structured data for an online order that has the following attributes: 

  1. Each user should have a first and a last name. 
  2. Each user should have between 0 and 50 orders. 
  3. The dataset should contain exactly 10 rows.
  4. The output should be in JSON format.

To accomplish this, we’ll create a spec as a Pydantic model:

from pydantic import BaseModel, Field
from guardrails.validators import ValidLength, TwoWords, ValidRange
from typing import List
prompt = """
Generate a dataset of fake user orders. Each row of the dataset should be valid. The format should not be a list, it should be a JSON object.
an example of output may look like this:
"user_orders": [{ │ │
"user_id": 1,
"user_name": "John Mcdonald",
"num_orders": 6
class Order(BaseModel):
user_id: int = Field(description="The user's id.", validators=[("1-indexed", "noop")])
user_name: str = Field(
description="The user's first name and last name",
num_orders: int = Field(
description="The number of orders the user has placed",
validators=[ValidRange(0, 50)]

class Orders(BaseModel):
user_orders: List[Order] = Field(
description="Generate a list of users and how many orders they have placed in the past.",
validators=[ValidLength(10, 10, on_fail="noop")]

The Pydantic file above defines two models: an Order model that defines the format of each order; and an Orders model that holds a list of Order objects. The validators parameters for each property define the parameter format that the output from the LLM must satisfy for Guardrails to accept it.

Now, we can create a Guard from our Pydantic model:

guard = gd.Guard.from_pydantic(output_class=Orders, prompt=prompt)

Guardrails will generate a full prompt based on our Pydantic model. Note that it compiles an XML specification for the output and makes it part of the prompt.

Finally, let’s wrap our call to the LLM in Cohere with our guard:

raw_llm_response, validated_response = guard(

Once again, let's break this down line by line: 

  1. raw_llm_response: The raw response object returned by the GPT-3 model. It will contain various information, such as the generated text, the likelihoods, and other metadata.
  2. validated_response: The validated or processed version of the raw response.
  3. co.generate: The method to call on Cohere to generate our synthetic structured data.
  4. model="command": Specifies we could use Cohere's command module. 
  5. max_tokens=1024: Again, we use max_tokens to limit response length and cap computing resources.
  6. temperature=0: The degree of randomness. Since this is structured text, we use 0.3 to specify we want the result to be mostly deterministic. 

The result in validated_response is the JSON data generated by Guardrails:

'user_orders': [
{'user_id': 1, 'user_name': 'John Smith', 'num_orders': 10},
{'user_id': 2, 'user_name': 'Jane Doe', 'num_orders': 20},
{'user_id': 3, 'user_name': 'Bob Jones', 'num_orders': 30},
{'user_id': 4, 'user_name': 'Alice Smith', 'num_orders': 40},
{'user_id': 5, 'user_name': 'John Doe', 'num_orders': 50},
{'user_id': 6, 'user_name': 'Jane Jones', 'num_orders': 0},
{'user_id': 7, 'user_name': 'Bob Smith', 'num_orders': 10},
{'user_id': 8, 'user_name': 'Alice Doe', 'num_orders': 20},
{'user_id': 9, 'user_name': 'John Jones', 'num_orders': 30},
{'user_id': 10, 'user_name': 'Jane Smith', 'num_orders': 40}

Guardrails logs the full history of calls it makes to the LLM. You can see this history in Python by running:


You can use this information for: 

  • Debugging and pinpointing issues. Use the full sequence of LLM calls to identify the source of errors or unexpected behavior in the generated output.
  • Understanding model behavior. By reviewing the full history, developers can better understand how the LLM interprets and responds to various input types. 
  • Fine-tuning and parameter optimization. The history of calls provides insights into how different prompts and parameters affect the model's responses. This information can be invaluable when fine-tuning the model or optimizing parameters to achieve desired outcomes.
  • Version control and collaboration. Developers can track changes and experiment with different prompt variations over time. This is essential for version control and collaboration among team members on the same project.
  • Context preservation. Examining past calls preserves the context of previous interactions with the model. This context is vital when dealing with conversations or dialogue-based systems, where the model's responses depend on preceding prompts.


Cohere combined with Guardrails AI is a novel and groundbreaking approach to data generation. In this article, we’ve introduced both technologies and shown how you can leverage them with a simple Python script to create your own synthetic structured data. By harnessing the power of Large Language Models, you can effortlessly generate diverse and contextually relevant structured data with just a few lines of code.