Ability to run single test methods or classes

[giampaolo]

This is a common use case, supported both by unittest and pytest. E.g. in unittest I can run an individual test method with:

python3 -m unittest somelib.tests.test_process.TestProcess.test_kill

With pytest:

python3 -m pytest mylib/tests/test_process.py::TestProcess::test_kill

Apparently with UnitTesting it's possible to specify a single test module to run, but not a test class or a test method.
Right now I am running individual test modules by using this build config:

    "build_systems": [
        {
            "name": "Sublime tests:",
            "target": "unit_testing",
            "package": "mypackage",
            "variants": [
               {"name": "test_create_snippet_from_sel.py",  "pattern": "test_create_snippet_from_sel.py"},
            ]
         }

I would like to be able to specify something like (mimicking pytest style):

 {"name": "test_kill",  "pattern": "test_create_snippet_from_sel.py::TestProcess::test_kill"},

To give even more context about my specific use case: I have a dynamic build system able to run individual test methods based on the cursor position (see ST forum post). I did this for both pytest and unittest. I would like to do the same by using UnitTesting.

@randy3k if you like the idea I can try working on a PR.

[randy3k]

UnitTesting uses TestLoader.discover to discover tests. I beleive you might be also to switch to use loadTestsFromModule or loadTestsFromNames like waht unittest does.

[giampaolo]

Thanks for your response. loadTestsFromName indeed looks like the way to go. It seems difficult to integrate it into the current code though. Not without breaking the existing configurations out there at least. The 2 approaches seems just incompatible.

Perhaps UnitTesting could accept a new dot_pattern parameter which takes precedence over pattern. If specified, tests are loaded via loadTestsFromName, else via TestLoader.discover.

I put this together:

diff --git a/unittesting/package.py b/unittesting/package.py
index 6259772..cffda42 100644
--- a/unittesting/package.py
+++ b/unittesting/package.py
@@ -82,10 +82,14 @@ class UnitTestingCommand(sublime_plugin.ApplicationCommand, UnitTestingMixin):
                 # use custom loader which supports reloading modules
                 self.remove_test_modules(package, settings["tests_dir"])
                 loader = TestLoader(settings["deferred"])
-                if os.path.exists(os.path.join(start_dir, "__init__.py")):
-                    tests = loader.discover(start_dir, settings["pattern"], top_level_dir=package_dir)
+                if "dot_pattern" in settings:
+                    tests = loader.loadTestsFromName(settings["dot_pattern"])
                 else:
-                    tests = loader.discover(start_dir, settings["pattern"])
+                    if os.path.exists(os.path.join(start_dir, "__init__.py")):
+                        tests = loader.discover(start_dir, settings["pattern"], top_level_dir=package_dir)
+                    else:
+                        tests = loader.discover(start_dir, settings["pattern"])
+
                 # use deferred test runner or default test runner
                 if settings["deferred"]:
                     if settings["legacy_runner"]:

With the above change I was able to run a single test method with the following build config:

    "build_systems": [
        {
            "name": "Sublime tests:",
            "target": "unit_testing",
            "package": "User",
            "variants": [
               {"name": "test_copy_pyobj_path",  "dot_pattern": "User.tests.test_pypaths.TestCopyPathsCommand.test_copy_pyobj_path"},
            ]
       }
    ]

If you think this is a reasonable approach I can make a PR which also updates the README.

[randy3k]

I think the current setup only works for plugins under User because User is always loaded. We might need to handle the loading of the module for plugins under Pacakges/MyPackage.

[giampaolo]

We might need to handle the loading of the module for plugins under Packages/MyPackage.

Mmm... unittest's dotted notation should represent the absolute path of the python object, so as long as MyPackage can be found in sys.path, loadTestsFromName should be able to find it, regardless of whether it's loaded. E.g., with the above patch, I am able to run an individual test of the Package/UnitTesting package by specifying its absolute python object location.

    "build_systems": [
        {
            "name": "Sublime tests:",
            "target": "unit_testing",
            "package": "User",
            "variants": [
               {"name": "single UnitTesting test", "dot_pattern": "UnitTesting.tests.test_await_worker.TestAwaitingWorkerInDeferredTestCase.test_await_worker"}
           ]
        },
    ]

Panel output:

test_await_worker (UnitTesting.tests.test_await_worker.TestAwaitingWorkerInDeferredTestCase) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.508s

OK

Basically this is the same dotted notation you can also use via the import statement. E.g. in a ST console I can do:

>>> from UnitTesting.tests.test_await_worker import TestAwaitingWorkerInDeferredTestCase
>>> TestAwaitingWorkerInDeferredTestCase
<class 'UnitTesting.tests.test_await_worker.TestAwaitingWorkerInDeferredTestCase'>

[randy3k]

Thanks for the investigation. I will take a look soon.

[giampaolo]

Thank you Randy. For the record, I wanted a way to quickly run a single test method while I am writing it, and ended up with a different (more dynamic) solution by monkey patching the TestLoader, which is a solution customized for my specific setup (so not really generic):

from unittest.mock import patch

import sublime
from unittesting.core import TestLoader
from unittesting.package import UnitTestingCommand

from .pypaths import pyobj_dotted_path


class CursorTestLoader(TestLoader):
    def discover(self, *args, **kwargs):
        pypath = pyobj_dotted_path(sublime.active_window().active_view())
        pypath = pypath.lstrip("home..config.sublime-text.Packages.")
        return self.loadTestsFromName(pypath)


class UnitTestingAtCursorCommand(UnitTestingCommand):
    """A variant which runs a test method/class/module given the current
    cursor position."""

    def unit_testing(self, stream, package, settings, cleanup_hooks=[]):
        with patch(
            "unittesting.package.TestLoader",
            side_effect=CursorTestLoader,
            create=True,
        ):
            return super().unit_testing(
                stream=stream,
                package=package,
                settings=settings,
                cleanup_hooks=cleanup_hooks,
            )

...so personally I'm "covered" for this specific problem. =)
With that said, thank you for this package. I think it's crucial for the quality of the ST plugins ecosystem. I have some other ideas in mind actually, so I may come up with some other proposal/discussion soon, if you don't mind.

[randy3k]

I'm glad that you have found a way to get what you need.
Just 1 comment, it may be more robust if you use arg and kwarg, e.g.

    def unit_testing(self, *args, **kwargs):
        with patch(
            "unittesting.package.TestLoader",
            side_effect=CursorTestLoader,
            create=True,
        ):
            return super().unit_testing(*args, **kwargs)

[giampaolo]

Definitively. Thanks for the suggestion.