The idea behind the program I’m trying to create is a file that can help grade programming exercises while allowing the students freedom to meet the tasks without telling them exactly how to accomplish them. I would like to be able to use a program to analyze the inputs and outputs to the terminal as they occur rather than analyzing everything in a stdout after the program has finished running.

I’ve been bouncing through threads trying different approaches involving subprocess.run, open, with subprocess.Popen(…) as proc:, exec(), and pexpect. The code example I’ll include here is the last one I felt was anywhere close to usable (even if it’s not where I want it to be).

The problem with the way this code works is that it only compares the expected result to the final stdout which is just a huge string that could be hardcoded and provide a false pass. Instead of providing an entire text file as an input using the command “script < txt_file” I would like to be able to provide just one line of input at a time and capture the output at that moment so that I can see if each input is yielding a response as well as being able to potentially isolate specific areas to look at for more difficult exercises.

import os
import subprocess

cwd = os.path.dirname(os.path.abspath(__file__))
expected_input = f"{cwd}/expected_input.txt"
expected_path = f"{cwd}/expected_output.txt"

def get_expected_output(file_path):
    return_str = ""
    with open(file_path, "r") as temp_file:
        for line in temp_file:
            if not line:
                break
            return_str += line
    return return_str

def main():
    expected_output = get_expected_output(expected_path)
    for x in range(1,4):
        test_path = f"{cwd}/test{x}.py"
        with subprocess.Popen(f"python {test_path} < {expected_input}", 
                              stdin=subprocess.PIPE,
                              stdout=subprocess.PIPE,
                              stderr=subprocess.PIPE,
                              shell=True,  
                              text=True
                              ) as proc:
            std_out, std_err = proc.communicate()
            if std_out != expected_output:
                print(f"Expected: n{expected_output} nnActual: n{std_out}")
                print(f"Take a look at student {x}'s code")
            else:
                print(f"Student {x} finished")

if __name__ == "__main__":
    main()

Ex1 (A normal passing code):

import sys

def main():
    while(True):
        print("Please enter a number or type exit")
        user_input = float(input())
        if user_input == 'exit':
            sys.exit()
        else:
            print(f"{user_input} x 2 = {user_input*2}")

if __name__ == "__main__":
    main()

Ex2 (A hardcoded series of print statements that still yields a pass):

print("Please enter a number or type exit")
print("1.0 x 2 = 2.0")
print("Please enter a number or type exit")
print("2.0 x 2 = 4.0")
print("Please enter a number or type exit")
print("3.0 x 2 = 6.0")
print("Please enter a number or type exit")