Skip to content

Latest commit

 

History

History
201 lines (199 loc) · 5.14 KB

File metadata and controls

201 lines (199 loc) · 5.14 KB

Results of Code LLama 34B on HumanEval/83

Date: 01 September, 2023
Model: codellama-34b-instruct
Temperature: 0.2
Passing runs: 10
Total runs: 10
Success rate: 100%

Prompt

def starts_one_ends(n):
    """
    Given a positive integer n, return the count of the numbers of n-digit
    positive integers that start or end with 1.
    """

The above is an incomplete Python code fragment. Return the complete and correct code with no additional text.

Correct solution

def starts_one_ends(n):
    """
    Given a positive integer n, return the count of the numbers of n-digit
    positive integers that start or end with 1.
    """if n == 1: return 1
    return 18 * (10 ** (n - 2))

Unit tests

def check(candidate):

    # Check some simple cases
    assert True, "This prints if this assert fails 1 (good for debugging!)"
    assert candidate(1) == 1
    assert candidate(2) == 18
    assert candidate(3) == 180
    assert candidate(4) == 1800
    assert candidate(5) == 18000

    # Check some edge cases that are easy to work out by hand.
    assert True, "This prints if this assert fails 2 (also good for debugging!)"

Runs

✅ Run 1

Start time: 2023-09-01 02:47:19
Duration: 6.59s
Result: Pass

def starts_one_ends(n):
    """
    Given a positive integer n, return the count of the numbers of n-digit
    positive integers that start or end with 1.
    """
    count = 0
    for i in range(10**(n-1), 10**n):
        if str(i)[0] == '1' or str(i)[-1] == '1':
            count += 1
    return count

✅ Run 2

Start time: 2023-09-01 02:50:19
Duration: 5.76s
Result: Pass

def starts_one_ends(n):
    """
    Given a positive integer n, return the count of the numbers of n-digit
    positive integers that start or end with 1.
    """
    count = 0
    for i in range(10**(n-1), 10**n):
        if str(i)[0] == '1' or str(i)[-1] == '1':
            count += 1
    return count

✅ Run 3

Start time: 2023-09-01 02:53:15
Duration: 5.91s
Result: Pass

def starts_one_ends(n):
    """
    Given a positive integer n, return the count of the numbers of n-digit
    positive integers that start or end with 1.
    """
    count = 0
    for i in range(10**(n-1), 10**n):
        if str(i)[0] == '1' or str(i)[-1] == '1':
            count += 1
    return count

✅ Run 4

Start time: 2023-09-01 02:56:13
Duration: 5.98s
Result: Pass

def starts_one_ends(n):
    """
    Given a positive integer n, return the count of the numbers of n-digit
    positive integers that start or end with 1.
    """
    count = 0
    for i in range(10**(n-1), 10**n):
        if str(i)[0] == '1' or str(i)[-1] == '1':
            count += 1
    return count

✅ Run 5

Start time: 2023-09-01 02:59:06
Duration: 6.1s
Result: Pass

def starts_one_ends(n):
    """
    Given a positive integer n, return the count of the numbers of n-digit
    positive integers that start or end with 1.
    """
    count = 0
    for i in range(10**(n-1), 10**n):
        if str(i)[0] == '1' or str(i)[-1] == '1':
            count += 1
    return count

✅ Run 6

Start time: 2023-09-01 03:02:12
Duration: 5.52s
Result: Pass

def starts_one_ends(n):
    """
    Given a positive integer n, return the count of the numbers of n-digit
    positive integers that start or end with 1.
    """
    count = 0
    for i in range(10**(n-1), 10**n):
        if str(i)[0] == '1' or str(i)[-1] == '1':
            count += 1
    return count

✅ Run 7

Start time: 2023-09-01 03:05:13
Duration: 7.1s
Result: Pass

def starts_one_ends(n):
    """
    Given a positive integer n, return the count of the numbers of n-digit
    positive integers that start or end with 1.
    """
    count = 0
    for i in range(10**(n-1), 10**n):
        if str(i)[0] == '1' or str(i)[-1] == '1':
            count += 1
    return count

✅ Run 8

Start time: 2023-09-01 03:08:16
Duration: 5.93s
Result: Pass

def starts_one_ends(n):
    """
    Given a positive integer n, return the count of the numbers of n-digit
    positive integers that start or end with 1.
    """
    count = 0
    for i in range(10**(n-1), 10**n):
        if str(i)[0] == '1' or str(i)[-1] == '1':
            count += 1
    return count

✅ Run 9

Start time: 2023-09-01 03:11:14
Duration: 6.58s
Result: Pass

def starts_one_ends(n):
    """
    Given a positive integer n, return the count of the numbers of n-digit
    positive integers that start or end with 1.
    """
    count = 0
    for i in range(10**(n-1), 10**n):
        if str(i)[0] == '1' or str(i)[-1] == '1':
            count += 1
    return count

✅ Run 10

Start time: 2023-09-01 03:14:14
Duration: 5.85s
Result: Pass

def starts_one_ends(n):
    """
    Given a positive integer n, return the count of the numbers of n-digit
    positive integers that start or end with 1.
    """
    count = 0
    for i in range(10**(n-1), 10**n):
        if str(i)[0] == '1' or str(i)[-1] == '1':
            count += 1
    return count