Generating Bug Free Leetcode Solutions

note

To download this tutorial as a Jupyter notebook, click here.

In this example, we want to solve String Manipulation leetcode problems such that the code is bug free.

We make the assumption that:

1. We don't need any external libraries that are not already installed in the environment.
2. We are able to execute the code in the environment.

Objective

We want to generate bug-free code for solving leetcode problems. In this example, we don't account for semantic bugs, only for syntactic bugs.

In short, we want to make sure that the code can be executed without any errors.

Step 1: Install validators from the hub

First, we install the validators and packages we need to make sure generated python is valid.

guardrails hub install hub://reflex/valid_python --quiet

    Installing hub://reflex/valid_python...
    ✅Successfully installed reflex/valid_python!
    
    

Step 2: Create a Guard object

The Guard object contains the validations we aim to check the generated code against. This object also takes corrective action to fix the code if it doesn't pass the validations. As configured here, it will reask the LLM to correct the code.

from guardrails.hub import ValidPython
from guardrails import Guard

guard = Guard().use(ValidPython(on_fail="reask"))

Step 3: Use the Guard to make and validate the LLM API call

# Add your OPENAI_API_KEY as an environment variable if it's not already set
# import os
# os.environ["OPENAI_API_KEY"] = "YOUR_API_KEY"

prompt = """
Given the following high level leetcode problem description, write a short Python code snippet that solves the problem.
Do not include any markdown in the response.

Problem Description:
${leetcode_problem}
"""

leetcode_problem = """
Given a string s, find the longest palindromic substring in s. You may assume that the maximum length of s is 1000.
"""

response = guard(
    model="gpt-4o",
    messages=[{
        "role": "user",
        "content": prompt
    }],
    # prompt=prompt,
    prompt_params={"leetcode_problem": leetcode_problem},
    temperature=0
)

print(response)

ValidationOutcome(
    call_id='12105167920',
    raw_llm_output='def longest_palindromic_substring(s):
    n = len(s)
    if n == 0:
        return ""
    

    start, max_length = 0, 1
    
    for i in range(1, n):
        low, high = i - 1, i
        while low >=
0 and high < n and s[low] == s[high]:
            if high - low + 1 > max_length:
                start = low
  
max_length = high - low + 1
            low -= 1
            high += 1
        
        low, high = i - 1, i + 
1
        while low >= 0 and high < n and s[low] == s[high]:
            if high - low + 1 > max_length:
       
start = low
                max_length = high - low + 1
            low -= 1
            high += 1
    
    
return s[start:start + max_length]

# Example usage:
s = "babad"
print(longest_palindromic_substring(s))  # 
Output: "bab" or "aba"',
    validated_output='def longest_palindromic_substring(s):
    n = len(s)
    if n == 0:
        return ""
   

    start, max_length = 0, 1
    
    for i in range(1, n):
        low, high = i - 1, i
        while low >=
0 and high < n and s[low] == s[high]:
            if high - low + 1 > max_length:
                start = low
  
max_length = high - low + 1
            low -= 1
            high += 1
        
        low, high = i - 1, i + 
1
        while low >= 0 and high < n and s[low] == s[high]:
            if high - low + 1 > max_length:
       
start = low
                max_length = high - low + 1
            low -= 1
            high += 1
    
    
return s[start:start + max_length]

# Example usage:
s = "babad"
print(longest_palindromic_substring(s))  # 
Output: "bab" or "aba"',
    reask=None,
    validation_passed=True,
    error=None
)

The response above shows a brief summary of what the Guard did. We can destructure it to show the final output:

print(response.validated_output)

def longestPalindrome(s: str) -> str:
    if len(s) == 0:
        return ""

    start, end = 0, 0

    for i in range(len(s)):
        len1 = expandAroundCenter(s, i, i)
        len2 = expandAroundCenter(s, i, i + 1)
        max_len = max(len1, len2)

        if max_len > end - start:
            start = i - (max_len - 1) // 2
            end = i + max_len // 2

    return s

def expandAroundCenter(s: str, left: int, right: int) -> int:
    while left >= 0 and right < len(s) and s == s:
        left -= 1
        right += 1
    return right - left - 1

# Example usage:
s = "babad"
print(longestPalindrome(s))  # Output: "bab" or "aba"

We can confirm that the code is bug free by executing the code in the environment.

try:
    exec(response.validated_output)
    print("Success!")
except Exception as e:
    print("Failed!")

aba

```

Success!