Skip to content

Latest commit

 

History

History
189 lines (181 loc) · 5.49 KB

140.md

File metadata and controls

189 lines (181 loc) · 5.49 KB
Raw

Results of Code LLama 34B on HumanEval/140

Date: 01 September, 2023
Model: codellama-34b-instruct
Temperature: 0.2
Passing runs: 0
Total runs: 10
Success rate: 0%

Prompt

def fix_spaces(text):
    """
    Given a string text, replace all spaces in it with underscores, 
    and if a string has more than 2 consecutive spaces, 
    then replace all consecutive spaces with - 
    
    fix_spaces("Example") == "Example"
    fix_spaces("Example 1") == "Example_1"
    fix_spaces(" Example 2") == "_Example_2"
    fix_spaces(" Example   3") == "_Example-3"
    """

The above is an incomplete Python code fragment. Return the complete and correct code with no additional text.

Correct solution

def fix_spaces(text):
    """
    Given a string text, replace all spaces in it with underscores, 
    and if a string has more than 2 consecutive spaces, 
    then replace all consecutive spaces with - 
    
    fix_spaces("Example") == "Example"
    fix_spaces("Example 1") == "Example_1"
    fix_spaces(" Example 2") == "_Example_2"
    fix_spaces(" Example   3") == "_Example-3"
    """new_text = ""
    i = 0
    start, end = 0, 0
    while i < len(text):
        if text[i] == " ":
            end += 1
        else:
            if end - start > 2:
                new_text += "-"+text[i]
            elif end - start > 0:
                new_text += "_"*(end - start)+text[i]
            else:
                new_text += text[i]
            start, end = i+1, i+1
        i+=1
    if end - start > 2:
        new_text += "-"
    elif end - start > 0:
        new_text += "_"
    return new_text

Unit tests

def check(candidate):

    # Check some simple cases
    assert candidate("Example") == "Example", "This prints if this assert fails 1 (good for debugging!)"
    assert candidate("Mudasir Hanif ") == "Mudasir_Hanif_", "This prints if this assert fails 2 (good for debugging!)"
    assert candidate("Yellow Yellow  Dirty  Fellow") == "Yellow_Yellow__Dirty__Fellow", "This prints if this assert fails 3 (good for debugging!)"
    
    # Check some edge cases that are easy to work out by hand.
    assert candidate("Exa   mple") == "Exa-mple", "This prints if this assert fails 4 (good for debugging!)"
    assert candidate("   Exa 1 2 2 mple") == "-Exa_1_2_2_mple", "This prints if this assert fails 4 (good for debugging!)"

Runs

❌ Run 1

Start time: 2023-09-01 02:46:18
Duration: 3.1s
Result: Fail

def fix_spaces(text):
    return text.replace(" ", "-").replace(" ", "_")

❌ Run 2

Start time: 2023-09-01 02:49:15
Duration: 3.4s
Result: Fail

def fix_spaces(text):
    return text.replace(" ", "-").replace(" ", "_")

❌ Run 3

Start time: 2023-09-01 02:52:11
Duration: 2.61s
Result: Fail

def fix_spaces(text):
    return text.replace(" ", "-").replace(" ", "_")

❌ Run 4

Start time: 2023-09-01 02:55:05
Duration: 7.22s
Result: Fail

def fix_spaces(text):
    """
    Given a string text, replace all spaces in it with underscores, 
    and if a string has more than 2 consecutive spaces, 
    then replace all consecutive spaces with - 
    
    fix_spaces("Example") == "Example"
    fix_spaces("Example 1") == "Example_1"
    fix_spaces(" Example 2") == "_Example_2"
    fix_spaces(" Example   3") == "_Example-3"
    """
    return text.replace(" ", "-").replace(" ", "_")

❌ Run 5

Start time: 2023-09-01 02:58:04
Duration: 3.29s
Result: Fail

def fix_spaces(text):
    return text.replace(" ", "-").replace(" ", "_")

❌ Run 6

Start time: 2023-09-01 03:01:08
Duration: 6.57s
Result: Fail

def fix_spaces(text):
    """
    Given a string text, replace all spaces in it with underscores, 
    and if a string has more than 2 consecutive spaces, 
    then replace all consecutive spaces with - 
    
    fix_spaces("Example") == "Example"
    fix_spaces("Example 1") == "Example_1"
    fix_spaces(" Example 2") == "_Example_2"
    fix_spaces(" Example   3") == "_Example-3"
    """
    return text.replace(" ", "-")

❌ Run 7

Start time: 2023-09-01 03:04:06
Duration: 2.54s
Result: Fail

def fix_spaces(text):
    return text.replace(" ", "-").replace(" ", "_")

❌ Run 8

Start time: 2023-09-01 03:07:10
Duration: 6.62s
Result: Fail

def fix_spaces(text):
    """
    Given a string text, replace all spaces in it with underscores, 
    and if a string has more than 2 consecutive spaces, 
    then replace all consecutive spaces with - 
    
    fix_spaces("Example") == "Example"
    fix_spaces("Example 1") == "Example_1"
    fix_spaces(" Example 2") == "_Example_2"
    fix_spaces(" Example   3") == "_Example-3"
    """
    return text.replace(" ", "-").replace(" ", "_")

❌ Run 9

Start time: 2023-09-01 03:10:11
Duration: 6.83s
Result: Fail

def fix_spaces(text):
    """
    Given a string text, replace all spaces in it with underscores, 
    and if a string has more than 2 consecutive spaces, 
    then replace all consecutive spaces with - 
    
    fix_spaces("Example") == "Example"
    fix_spaces("Example 1") == "Example_1"
    fix_spaces(" Example 2") == "_Example_2"
    fix_spaces(" Example   3") == "_Example-3"
    """
    return text.replace(" ", "-").replace(" ", "_")

❌ Run 10

Start time: 2023-09-01 03:13:10
Duration: 2.67s
Result: Fail

def fix_spaces(text):
    return text.replace(" ", "-").replace(" ", "_")