Forums

When I run doctest in a script, it doesn't work on the first run, but does on the second run.

To reproduce the issue:

1. Make a script with a doctest and save it. Here's a minimal example which should pass:

   """
   >>> 1
   1
   """
   import doctest
   print(doctest.testmod())  # Print results
   

2. Press Ctrl+R to run, and you get results as if there are no docstrings:

   TestResults(failed=0, attempted=0)
   

3. Press Ctrl+R to run again, and you get the correct results:

   TestResults(failed=0, attempted=1)
   

4. This issue also affects changes to an existing docstring. Change the code to this, which should fail:

   """
   >>> 1
   2
   """
   import doctest
   print(doctest.testmod())  # Print results
   

5. Press Ctrl+R, and it succeeds:

   TestResults(failed=0, attempted=1)
   

6. Press Ctrl+R again, and it fails as it should:

   **********************************************************************
   File "/.../test.py", line 2, in __main__
   Failed example:
       1
   Expected:
       2
   Got:
       1
   **********************************************************************
   1 items had failures:
      1 of   1 in __main__
   ***Test Failed*** 1 failures.
   TestResults(failed=1, attempted=1)
   

It's almost like it's getting the docstrings from the last run, but I have no idea why. FWIW, I've checked and made sure that __doc__ is working properly, and the doctest works properly if you run the script from a shell.

Also I just noticed that if you remove a module-level docstring, __doc__ is not reset no matter how many times you rerun the script. To remove it you have manually reset it (__doc__ = None) or restart the console.

Hmm, that's interesting -- it sounds like it might be a bug in our system that runs Python files. We keep the same console running all of the time when we can, because consoles take some time to load and we don't want you to have to wait for one to start up every time you hit Ctrl-R, and when you run your code we clear everything out of the existing Python session and then run the script. What you describe suggests that the "clear everything out" step of that process is missing some stuff.

To double-check that I'm right there, could you try doing the same thing, but this time explicitly running exit() in the console before you hit Ctrl-R each time? If that fixes it, then we'll know what the bug is and can start looking into a fix.

If I close the console with exit() or Ctrl+D then Ctrl+R again, it reproduces the issue like in step 2 of the instructions I posted:

TestResults(failed=0, attempted=0)

Thanks, that's interesting (and the opposite way round to what I thought it was). Definitely a bug in our save and run code, so I've logged it as such so that we can work out how to fix it.

Here's a workaround for the meantime -- close the Python console using Ctrl-D, then refresh the page -- at the bottom you should now have two buttons where the console was, one of them being "Bash console here". Click that to start a Bash console, and there you can run (for example)

python3.8 yourscriptname.py

...to run your test. Then as you code, you can hit the normal save button instead of the run one, and then use up-arrow, return to run the tests. It's not as good as being able to use the run button, but hopefully it's a useful workaround.