Robot Py pytest Test Runner in Visual Studio Code

[surbach]

Hello, I was wondering if anyone has been able to use the Test Explorer in Visual Studio Code Python Extension from Microsoft, to run Unit Tests for a robot.py file on Windows instead of using the command for **py -3 robot.py test **?

I am wondering if it is possible to debug the unit tests inside the test runner. From what I have seen, it appears there are many fixtures needed to be setup to get the tests to run on wpilib.RobotBase. I have not traced through the code yet to see what happens when the ‘test’ argument is used when running the robot.py script.

Reference:
Python testing in Visual Studio Code
The Python extension supports testing with Python’s built-in unittest framework as well as pytest.

[virtuald]

The underlying test command uses pytest with a custom plugin (https://github.com/robotpy/pyfrc/blob/master/lib/pyfrc/mains/cli_test.py#L115). It would be cool to get this sort of thing working in vscode (and documented) if you want to dig into it. If there are things we can change to make it easier then pull requests are welcome too.

[surbach]

Hello Dustin,
Thank you for your response and the link to the PyFrcPlugin location.
I agree that it would be cool to have Unit Testing work in VS Code. It seems like there needs to be a way to feed the pytest_plugin.PyFrcPlugin location to PyTest when it is being started by the Test Explorer, and make sure that PyTest accesses the fixtures needed.

retv = pytest.main( pytest_args,
plugins=[ pytest_plugin.PyFrcPlugin(robot_class, robot_file, robot_path) ], )

• control - the wpilib.internal module

• fake_time - the FAKETIME object that controls time
  

• robot - This is whatever is returned from the run function in robot.py
  

• robot_file - The filename of robot.py
  

• robot_path - The directory name that robot.py is located in
  

• wpilib - This is the wpilib module
  

I am not certain yet what is required to be configured in VS Code to get this setup to run outside of the main.py and PyFrcTest class methods.
“main.py” appears to setup the variables for the robot_class, the PyTest arguments, and has the references to the classes needed, which may not be imported in the Unit Test classes (if the example tests are followed).

It would be nice to be able to call pytest from the command line and run the unit test file, but it is failing for me.

It also seems like main.py is not passing along any additional arguments entered after “test” when calling “py -3 robot.py test”, but there is framework for it. It may be nice to feed the pytest module call with additional pytest arguments ; “main.py” may need to be modified for this to happen, allowing arguments to pass through to the pytest.main() method call. I am thinking of commands along the lines of:

py -3 -m pytest -v robot_test.py -p pytest_plugin
py -3 robot.py test -v --fixtures --version
py -3 robot.py test -v --maxfail=2 TestDirectory/
py -3 robot.py test -rA test_mod.py
py -3 robot.py test -v -k “MyClass and not method”

May need to use “PythonPath/Lib/site-packages/pyfrc/mains/cli_test.py” in the argument to get to the FRC plugin.

It also might be nice to have it drop to the Debugger if it fails a test

pytest -x --pdb
py -3 robot.py test -x --pdb

Debugger

I was able to get the debugger to run with the automated tests by adding some commands to the Workspace launch.json file in the.vscode directory. I added “test” for the “args” key in the “configurations” dictionary to get the unit tests to run in the debugger, and I set the KVP “justMyCode” to false to get the debugger to run in the WPIlib class files.

I also was able to launch the simulator from the debugger by using a “args”: [“sim”] KVP. I have the “program” key set to the current file, so the “robot.py” file has to be open and selected for the debugger to launch correctly. It can also work by hard coding the path to the robot.py file instead of using “${file}”.

It is basically calling

cd ‘MyPath\Seth’; ${env:PYTHONIOENCODING}=‘UTF-8’; ${env:PYTHONUNBUFFERED}=‘1’; & ‘PYTHONPath\python.exe’ 'VSCodePath.vscode\extensions\ms-python.python-2019.11.50794\pythonFiles\ptvsd_launcher.py’ ‘–default’ ‘–client’ ‘–host’ ‘localhost’ ‘–port’ ‘56841’ ‘SourceCodePath\robot.py’ ‘test’

to launch the debugger with the equivalent of the command “py -3 robot.py test” .

This has allowed me to do debugging in the unit tests to find out what is happening when an assert fails. It does not have the nice presentation that Test Explorer has, and the functionality to pick and choose a subset of tests; it runs through all tests discovered (that adhere to the directory and filename patterns).

[surbach]

Screenshot of my launch.json file that works with the Debugger:

[auscompgeek]

This is pretty similar to what I wrote last season:

Add VS Code config files

[virtuald]

Hey @surbach I expect the RobotPy test stuff to change quite a bit this year, but I haven’t really worked through it yet. If you’d like to help change it, stop by gitter on New Years, I expect I’ll be there most of the day.

[surbach]

Hello @auscompgeek,
Thank you very much for your reply! Your config files for VS Code are quite helpful in showing how to use a Python module instead of just the current file. It also looks great that you have a config for Attaching to the RoboRio. I am curious to try that out.

[surbach]

Hello @virtuald, Thank you for the invitation. I am interested in helping out if I can, with the RobotPy test modules. I am unfamiliar with “gitter”; I will have to look that up.

[virtuald]

https://gitter.im/robotpy/robotpy-wpilib is what I’m referring to