Forums

I'm an instructor looking to evaluate student code programmatically. Although PA for Educators lets me become any student user and run their code as that user, I want to do this programmatically, to validate their code solutions using my automated evaluation software.

I'd also like to consider the possibility of doing this for a non-student, that is someone who is using my web application to learn Python and has submitted code for automated evaluation.

In other words, I want to run untrusted code in a sandbox. I would like the code to have access to a limited collection of files as well.

I see the pynbox project (https://github.com/dsagal/pynbox) and this discussion (https://wiki.python.org/moin/SandboxedPython) but not sure if PA has its own solution for this.

thanks

David

Hi David, when you have a student, you can from your own account access their files at /home/studentusername/. So you could then say run some test file from your own account to validate their code?

For the web app submission part, that sounds a bit unsafe to let just anyone submit code to you and for you to run it- eg: what happens if they just run python code to delete your webapp? It would not be possible to sandbox that easily.

Hi Conrad,

thank you - I suppose that could work although if I run a student's code without looking at it myself, couldn't it also do damage to my web space since I am running their code as myself?

I guess what I'd like to be able to do is run the student's code as the student, or an untrusted user's code as the untrusted user (if they had a PythonAnywhere account)...

David

Yes- that could potentially trample your space.

Perhaps you could do something like put a submit.py into all your students' homedirs, which the students can run themselves. This submit.py would take input data, generates results, and save it in their directories. You could then collect those results and just mark/grade on whether those results are correct or not.

In fact, if you want to give more feedback for your students, have a tests.py with a couple simple cases that lets your students check their solution code, so they know if they are on the right track or not, and if their formatting is correct etc. Then your final scoring/marks can be an extended version of the tests.py code.

What about allowing me to sudo as the student? This way I could run their code as them without additional steps for the student to follow. This would be similar functionality to what PA has enabled in the GUI (where I can use the dropdown to see the student's account).

Test cases make a lot of sense but when I've tried them in the past I've had difficulty having beginners follow my instructions properly or understand the results...

Adding some way to easily sudo to the student is on our to-do list, but it's a pretty big job -- although PythonAnywhere looks like a standard Linux environment, there's a lot of different stuff happening under the hood, so we can't just use the standard sudo command.

I guess you could use the UI to start a Bash console as each student, and then literally run their code while switched to using their accounts (which of course would not allow their code any access to their accounts) but I appreciate that would be a lot of clicking around, and wouldn't scale awfully well...