python 从Nose2插件跳过单元测试

lzfw57am  于 2023-03-21  发布在  Python
关注(0)|答案(2)|浏览(78)

我在实际跳过Nose2插件的单元测试时遇到了麻烦。我可以标记跳过的测试并在最终结果中查看原因,但测试仍然运行。只要插件处于活动状态,此示例代码基本上应该跳过任何测试。

from nose2.events import Plugin

class SkipAllTests(Plugin):
    def startTest(self, event):
        event.result.addSkip(event.test, 'skip it')
        event.handled = True

如果我调用event.test.skipTest('reason'),它实际上像它应该的那样引发了SkipTest异常,只是这个异常没有被测试运行器捕获,它只是在我的startTest钩子方法内部引发。

yzuktlbb

yzuktlbb1#

我不认为你真的可以用startTest钩子阻止测试运行。nose2文档建议使用matchPathgetTestCaseNames来做到这一点。下面是一个使用matchPath的工作示例:

from nose2.events import Plugin

class SkipAllTests(Plugin):
    configSection = "skipper"
    commandLineSwitch = (None, 'skipper', "Skip all tests")

    def matchPath(self, event):
        event.handled = True
        return False

matchPath文档实际上明确解释了如何使用它来停止测试运行:
插件可以使用此钩子来阻止测试加载器加载python模块,或者强制测试加载器加载它们。将event.handled设置为True并返回False,以使加载器跳过该模块。
使用这个方法可以防止测试用例被加载。如果你想让测试在列表中显示为被跳过,而不是根本不显示在测试列表中,你可以对StartTestEvent做一些修改:

def dummy(*args, **kwargs):
    pass

class SkipAllTests(Plugin):
    configSection = "skipper"
    commandLineSwitch = (None, 'skipper', "Skip all tests")
    def startTest(self, event):
        event.test._testFunc = dummy
        event.result.addSkip(event.test, 'skip it')
        event.handled = True

在这里,我们用一个什么都不做的伪函数来替换测试将要运行的实际函数。这样,当测试执行时,它没有操作,然后报告它被跳过了。

3bygqnnd

3bygqnnd2#

我遇到了同样的问题,我的解决方案是:

class SkipMe :
    """
    Use this module together with nose2 and preferably with the 'with such.A() ...' construct.

    Synopsis          :
        import nose2.utils.such
        import unittest

        with such.A( 'thingy') as it :                  # Create a SkipMe object.
            skipme = SkipMe( 'my_id' )                  # You can give this section/chapter a name
            skipme = SkipMe()                           # or just leave it as 'Default'.

            @it.has_setup                               # Integrate to test setup to skip all tests if you like.
            def setup() :
                skipme.skip_all()                       # To skip every test in here.
                skipme.skip_all( 'for some reason' )    # Same but give a reason in verbose mode.
                ...

            @it.has_test_setup                          # Recommended to integrate into setup for every test.
            def test_setup() :
                skipme.skip_if_reason()                 # Skip test if an overall skip reason was set.
                ...

            @it.should( "do something basic")
            @skipme.skip_reg( skip_all_on_failed=True ) # Register this test. If it fails, skip all consecutive tests.
            def test_basic_things() :
                ...

            @it.should( "test something")
            @skipme.skip_reg()                          # Register this test. It will be marked as passed or failed.
            def test_one_thing() :
                ...

            @it.should( "test another thing")           # Skip this test if previous 'test_one_thing' test failed.
            @skipme.skip_reg( func_list=['test_one_thing'] )
            def test_another_thing() :                  # 'skipme.helper' is a unittest.TestCase object.
                skipme.helper.assertIs( onething, otherthing, 'MESSAGE' )
                ...

            it.createTests( globals() )

    Purpose :   Have a convenient way of skipping tests in a nose2 'whith such.A() ...' construct.
                As a bonus you have with the 'object.helper' method a 'unittest.TestCase' object.
                You can use it to have all the fine assert functions at hand, like assertRaisesRegexp.

    Initialize object with:
        :param  chapter :   A string as identifier for a chapter/section.

    Defaults:
        chapter :   'Default'

    Prerequisites:  import nose2.tools.such as such, unittest, inspect, functools

    Description:
        Basically this class has an internal dict that sets and browses for values.
        The dict looks like:
            {
                'IDENTIFYER_1' : {                          # This is the value of 'chapter'.
                    '__skip_all__'  :   BOOLEAN|FUNC_NAME,  # Skip all consecutive tests if this is True or a string.
                    'func1' :   BOOLEAN,                    # Register functions as True (passed) or False (failed).
                    'func2' :   BOOLEAN,                    # Also skipped tests are marked as failed.
                    ...
                },
                'IDENTIFYER_1' : { ... },                   # This is the domain of another SkipMe object with
                ...                                         # a different value for 'chapter'
            }

        It provides a decorator 'object.skip_reg' to decorate test functions to register them to the class,
        meaning updating the internal dict accordingly.
        Skipped tests are marked as failed in this context.

        Skipping all tests of a 'with such.A()' construct:
            Integrate it into the setup and the test setup like so:

                with such.A( 'thingy') as it :
                    skipme = SkipMe()

                    @it.has_setup
                    def setup() :
                        skipme.skip_all()

                    @it.has_test_setup
                    def test_setup() :
                        skipme.skip_if_reason()

        If you intend to skip all tests or all consecutive tests after a special test failed,
        you need only the '@it.has_test_setup' part.

        Register tests with the 'skip_reg' method:

            Decorate the test functions with the 'object.skip_reg' method under the @it.should decorator.
                Example:

                with such.A( 'thingy') as it :
                    skipme = SkipMe()
                    # Same setup as above
                    ...

                    @it.should( "Do something")
                    @skipme.skip_reg()                                  # Just register this function
                    @skipme.skip_reg( func_list=[TEST_FUNCTION_NAMES] ) # Skip test if one function in the list failed.
                    @skipme.skip_reg( skip_all_on_failed=True )         # Skip all consecutive tests if this fails
                    @skipme.skip_reg( func_list=[LIST_OF_TEST_FUNCTIONS], skip_all_on_failed=True ) # Or both.

    Example:

        with such.A( 'thingy' ) as it :
            skipme = SkipMe()

            @it.has_test_setup
            def test_setup() :
                skipme.skip_if_reason()

            @it.should( "Do something" )
            @skipme.skip_reg()
            def test_one():
                raise

            @it.should( "Do another thing" )
            @skipme.skip_reg( func_list=[ 'test_one' ] )
            def test_two():
                pass

            it.createTests( globals() )

        # Then run:
        nose2 --layer-reporter --plugin=nose2.plugins.layers
        # Prints:
        A thingy
          should Do something ... ERROR
          should Do another thing ... skipped because of failed: 'test_one'

          ...
    """
    chapter_of = {}

    def __init__( self, chapter=None ) :
        """
        Initialize a SkipMe object.
        :param chapter: If set, must be a string, else it's 'Default'.
        """
        func_name = inspect.stack()[ 0 ][ 3 ]  # This function Name
        if chapter is None :
            chapter = 'Default'                                 # Set default chapter for convenience

        if not isinstance( chapter, str ) :
            wrong_type = type( chapter )
            raise ValueError( "{0} {1}.{2}: Invalid input for 'chapter': '{3}'\n"
                              .format( 'ERROR', 'SkipMe', func_name, str( chapter ) )
                              + "{0} Must be string, but was: {1}".format( 'INFO', wrong_type.__name__ ) )

        self.chapter = chapter
        self.helper = self.SkipMeHelper()                       # Set unittest.TestCase object as helper

    @classmethod
    def set_chapter( cls, chapter=None, func=None, value=None ):
        """
        Mark a function of a chapter as passed (True) or failed (False) in class variable 'chapter_of'.
        Expands 'chapter_of' by chapter name, function and passed/failed value.
        :param chapter:     Chapter the function belongs to
        :param func:        Function name
        :param value:       Boolean
        :return:            None
        """
        func_name = inspect.stack()[ 0 ][ 3 ]  # This function Name
        if chapter is None :
            chapter = 'Default'                                 # Set default chapter for convenience

        if not isinstance( chapter, str ) :
            wrong_type = type( chapter )
            raise ValueError( "{0} {1}.{2}: Invalid input for 'chapter': '{3}'\n"
                              .format( 'ERROR', 'SkipMe', func_name, str( chapter ) )
                              + "{0} Must be string, but was: {1}".format( 'INFO', wrong_type.__name__ ) )

        if func is None :
            raise ValueError( "{0} {1}.{2}: No input for 'func'".format( 'ERROR', 'SkipMe', func_name ) )

        if not isinstance( func, str ) :
            wrong_type = type( func )
            raise ValueError( "{0} {1}.{2}: Invalid input for 'func': '{3}'\n"
                              .format( 'ERROR', 'SkipMe', func_name, str( func ) )
                              + "{0} Must be string, but was: {1}".format( 'INFO', wrong_type.__name__ ) )

        if not isinstance( value, bool ) :
            raise ValueError( "{0} {1}.{2}: No or invalid input for 'value'".format( 'ERROR', 'SkipMe', func_name ) )

        if chapter not in cls.chapter_of :                      # If we have this chapter not yet,
            cls.chapter_of[ chapter ] = {}                      # add it and set skip all to false.
            cls.chapter_of[ chapter ][ '__skip_all__' ] = False

        if func not in cls.chapter_of[ chapter ] :              # If we don't have the function yet, add it with value.
            cls.chapter_of[ chapter ][ func ] = value

    @classmethod
    def get_func_state( cls, chapter=None, func_list=None ):
        """
        Return function names  out of function list that previously failed
        :param chapter:     The chapter to search for functions
        :param func_list:   Browse for these function names
        :return:            List with failed functions. If none found, an empty list.
        """
        func_name = inspect.stack()[ 0 ][ 3 ]  # This function Name
        if chapter is None :
            chapter = 'Default'                                 # Set default chapter for convenience

        if not isinstance( chapter, str ) :
            wrong_type = type( chapter )
            raise ValueError( "{0} {1}.{2}: Invalid input for 'chapter': '{3}'\n"
                              .format( 'ERROR', 'SkipMe', func_name, str( chapter ) )
                              + "{0} Must be string, but was: {1}".format( 'INFO', wrong_type.__name__ ) )

        if func_list is None :
            raise ValueError( "{0} {1}.{2}: No input for 'func_list'".format( 'ERROR', 'SkipMe', func_name ) )

        #-------------------------
        # Function candidates to check.
        # Collect those candidates, that previously returned as failed or skipped.
        # Otherwise, return empty list.
        if isinstance( func_list, list ) :
            func_candidates = func_list
        elif isinstance( func_list, str ) :
            func_candidates = [ x.strip() for x in func_list.split( ',' ) ]
        else:
            wrong_type = type( func_list )
            raise ValueError( "{0} {1}: Invalid input for 'func_list': '{2}'\n"
                              .format( 'ERROR', func_name, str( func_list ) )
                              + "{0} Must be list or comma separated string, but was: '{1}'"
                              .format( 'INFO', wrong_type.__name__ ) )

        to_return = []                                          # List of failed functions
        if chapter not in cls.chapter_of :                      # If chapter not found, just return empty list
            return to_return

        for func in func_candidates :                           # Otherwise look for each candidate
            if func not in cls.chapter_of[ chapter ] :          # if it's in the chapter, skip if not.
                continue

            if not cls.chapter_of[ chapter ][ func ] :          # If it's value is False, append it.
                to_return.append( func )

        return to_return

    @classmethod
    def mark_chapter_as_skipped( cls, chapter=None, func=None ):
        """
        Mark chapter as skipped. Maybe because of a failed function
        :param chapter:     Which chapter to mark as skipped
        :param func:        Maybe the failed function that causes this decision
        :return:            None
        """
        func_name = inspect.stack()[ 0 ][ 3 ]                   # This function Name
        if chapter is None :
            chapter = 'Default'                                 # Set default chapter for convenience

        if not isinstance( chapter, str ) :
            wrong_type = type( chapter )
            raise ValueError( "{0} {1}.{2}: Invalid input for 'chapter': '{3}'\n"
                              .format( 'ERROR', 'SkipMe', func_name, str( chapter ) )
                              + "{0} Must be string, but was: {1}".format( 'INFO', wrong_type.__name__ ) )

        # Either func is a name or True.
        if func :
            if not isinstance( func, str ) :
                wrong_type = type( chapter )
                raise ValueError( "{0} {1}.{2}: Invalid input for 'func': '{3}'\n"
                                  .format( 'ERROR', 'SkipMe', func_name, str( func ) )
                                  + "{0} Must be string, but was: {1}".format( 'INFO', wrong_type.__name__ ) )
        else :
            func = True

        if chapter not in cls.chapter_of :                      # If we have this chapter not yet,
            cls.chapter_of[ chapter ] = {}                      # add it and set skip all to false.
            
        cls.chapter_of[ chapter ][ '__skip_all__' ] = func

    @classmethod
    def chapter_marked_skipped( cls, chapter=None ):
        """
        Check if a chapter is marked to skip.
        :param chapter:     The chapter to check
        :return:    False   :   Chapter is not marked to be skipped
                    True    :   Chapter was intentionally skipped
                    String  :   This function was marked with 'skip_all_on_failed=True' and failed.
        """
        func_name = inspect.stack()[ 0 ][ 3 ]                   # This function Name
        if chapter is None :
            chapter = 'Default'                                 # Set default chapter for convenience

        if not isinstance( chapter, str ) :
            wrong_type = type( chapter )
            raise ValueError( "{0} {1}.{2}: Invalid input for 'chapter': '{3}'\n"
                              .format( 'ERROR', 'SkipMe', func_name, str( chapter ) )
                              + "{0} Must be string, but was: {1}".format( 'INFO', wrong_type.__name__ ) )

        to_return = False
        if chapter not in cls.chapter_of :
            return to_return

        to_return = cls.chapter_of[ chapter ].get( '__skip_all__', False )
        return to_return

    def skip_reg( self, func_list=None, skip_all_on_failed=None ) :
        """
        Synopsis          :
            skipme = SkipMe( 'my_id' )
            @skipme.skip_reg()
            def some_test_func() :
                ...

            @skipme.skip_reg( func_list=[ LIST_OF_FUNCTIONS_TO_SKIP_IF_FAILED ], skip_all_on_failed=BOOLEAN )
            def some_other_test_func():
                ...

        Purpose           : Decorator to register functions in a SkipMe object and skip tests if necessary

        Incoming values :
            :param func_list    :   List or comma separated string with function names.
                                    Skip this test if one of these functions in the list failed or were skipped.
            :param chapter      :   Identifier string that can be used to control skipping more generally.
                                    Default name for chapter is 'Default'.
            :param skip_all_on_failed   :   Boolean. If this test fails, mark the current chapter
                                            to skip the rest of tests.
        Outgoing results  :
            :return     :   Updated class attribute 'chapter_of', maybe skipped test.
        Defaults          :
                chapter :   'Default'
        Prerequisites     :
            inspect
        Description:
            Register functions by decorating the functions.
            It returns a dict with the function name as key and the function
            reference as value.
        """
        func_name = inspect.stack()[ 0 ][ 3 ]  # This function Name
        chapter = self.chapter

        if isinstance( func_list, list ) :
            func_candidates = func_list
        elif isinstance( func_list, str ) :
            func_candidates = [ x.strip() for x in func_list.split( ',' ) ]
        elif func_list is None:
            func_candidates = []
        else :
            wrong_type = type( func_list )
            raise ValueError( "{0} {1}: Invalid input for 'func_list': '{2}'\n"
                              .format( 'ERROR', func_name, str( func_list ) )
                              + "{0} Must be list or comma separated string, but was: '{1}'"
                              .format( 'INFO', wrong_type.__name__ ) )

        def inner_skip_reg( func ) :
            @functools.wraps( func )
            def skip_reg_wrapper( *args, **kwargs ) :
                #-------------------------
                # First check if the whole chapter was marked as skipped.
                # The function either returns:
                #   True            :   Means 'skip_all' was set in the beginning (e.g. with @has_setup)
                #   False           :   No, chapter is not marked as to be skipped
                #   Function name   :   This function was marked with 'skip_all_on_failed' and failed.
                skip_reason = self.get_skip_reason()
                if skip_reason :
                    if isinstance( skip_reason, bool ) :
                        self.helper.skipTest( "chapter '{0}' because it's marked to be skipped".format( chapter ) )
                    else :
                        self.helper.skipTest( "chapter '{0}' because of: '{1}'"
                                       .format( chapter, skip_reason ) )

                #-------------------------
                # Now see if one of our functions we depend on failed.
                # If so, mark our func as failed and skip it.
                if func_candidates :
                    found_failed = self.__class__.get_func_state( chapter=chapter, func_list=func_candidates )
                    if found_failed :
                        self.__class__.set_chapter( chapter=chapter, func=func.__name__, value=False )
                        self.helper.skipTest( "because of failed: '{0}'".format( ', '.join( found_failed ) ) )

                #-------------------------
                # Now run the test.
                # If it fails (assertion error), mark as failed (False), else mark as passed (True)
                # If it fails and was marked as 'skip_all_on_failed', mark chapter as to skip all further tests.
                try :
                    result = func( *args, **kwargs )
                    self.__class__.set_chapter( chapter=chapter, func=func.__name__, value=True )
                except Exception as error :
                    self.__class__.set_chapter( chapter=chapter, func=func.__name__, value=False )
                    if skip_all_on_failed :
                        self.__class__.mark_chapter_as_skipped( chapter=chapter, func=func.__name__ )
                    raise error

                return result
            return skip_reg_wrapper
        return inner_skip_reg

    def get_skip_reason( self ):
        chapter = self.chapter
        skip_reason = self.__class__.chapter_marked_skipped( chapter=chapter )
        return skip_reason

    def skip_all( self, reason=None ):
        func_name = inspect.stack()[ 0 ][ 3 ]  # This function Name
        if reason is not None :
            if not isinstance( reason, str ) :
                wrong_type = type( reason )
                raise ValueError( "{0} {1}: Invalid input for 'reason': '{2}'\n"
                                  .format( 'ERROR', func_name, str( reason ) )
                                  + "{0} Must be string, but was: '{1}'".format( 'INFO', wrong_type.__name__ ) )

        self.__class__.mark_chapter_as_skipped(chapter=self.chapter, func=reason )

    def skip_if_reason( self ):
        skip_reason = self.get_skip_reason()
        if skip_reason:
            if skip_reason is True :
                reason = "it's marked to be skipped."
            else :      # Either function or other text.
                if skip_reason in self.__class__.chapter_of[ self.chapter ].keys() :
                    reason = "'{0}' failed.".format( skip_reason )
                else :
                    reason = "'{0}'.".format( skip_reason )
            self.helper.skipTest( "chapter '{0}' because: {1}".format( self.chapter, reason ) )

    class SkipMeHelper( unittest.TestCase ):
        def runTest(self):
            pass

    SkipMeHelper.maxDiff = None         # No limitation in depth while comparing

相关问题