书籍来源：房荔枝 梁丽丽《pytest框架与自动化测试应用》

一边学习一边整理老师的课程内容及实验笔记，并与大家分享，侵权即删，谢谢支持！

附上汇总贴：pytest框架进阶自学系列 | 汇总_热爱编程的通信人的博客-CSDN博客

在测试过程中有时会遇到这种情况，在执行某个用例后找出了用例中的Bug，开发人员会修改，但最近提交的版本中这个Bug并未被修改，开发人员可能在下个版本才会修改，这时我们知道即使执行这个用例也会失败。

这时可以使用@pytest.mark.xfail标记用例，表示期望这个用例执行失败。用例会正常执行，只是失败时不再显示堆栈信息，最终的结果有两个：用例执行失败（XFAIL：符合预期的失败）、用例执行成功（XPASS：不符合预期的成功）。也就是说执行成功反倒不正确了。

在用例执行中可以使用pytest.xfail实现这种效果。

如何取消设置的标识呢？通过下面3节内容进行说明。

@pytest.mark.xfail标记用例

首先看一下@pytest.mark.xfail()的用法，xfail有几个不同的参数，参数的不同组合会有不同的结果。condition是条件，reason是原因，raises是引起异常，run参数表明是否执行，strict是失败的用例是否显示为FAILED的开关，然后看一下源码，再通过具体实例体会不同参数的组合及执行的结果标记。

源码如下：

(1)预期失败的用例在执行失败时显示XFAIL标记，包含reason参数。

condition为位置参数，默认值为None。reason为关键字参数，默认值为None，可以加字符串写清原因。

代码如下：

import pytest

class Test_pytest():
    @pytest.mark.xfail(reason='该功能有Bug')
    def test_one(self):
        print("------start------")
        print("test_one方法执行")
        assert 2 == 1
    def test_two(self):
        print("test_two方法执行")
        assert "o" in "love"

    def test_three(self):
        print("test_three方法执行")
        assert 3 - 2 == 1

if __name__ == '__main__':
    pytest.main(['-sr', 'test_xfail.py'])

执行的结果如下，2个passed通过，1个xfailed是预期失败，原因是断言失败。

import pytest

class Test_pytest():
    @pytest.mark.xfail(reason='该功能有Bug')
    def test_one(self):
        print("------start------")
        print("test_one方法执行")
        assert 2 == 1
    def test_two(self):
        print("test_two方法执行")
        assert "o" in "love"

    def test_three(self):
        print("test_three方法执行")
        assert 3 - 2 == 1

if __name__ == '__main__':
    pytest.main(['-sr', 'test_xfail.py'])

(2)预期失败的用例在执行通过时显示XPASS标记。

如果将assert 2==1改成assert 1==1，则该功能是成功的，也就是说本来功能是有Bug的，但断言成功了，说明断言错误或功能有问题，这时显示的标记为XPASS。

执行的结果如下：2 passed，1 xpassed。

D:SynologyDriveCodeLearningWINpytest-bookvenvScriptspython.exe "C:/Program Files/JetBrains/PyCharm Community Edition 2022.3.2/plugins/python-ce/helpers/pycharm/_jb_pytest_runner.py" --path D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2	est_xfail.py 
Testing started at 14:34 ...
Launching pytest with arguments D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2	est_xfail.py in D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2

============================= test session starts =============================
platform win32 -- Python 3.7.7, pytest-5.4.1, py-1.11.0, pluggy-0.13.1 -- D:SynologyDriveCodeLearningWINpytest-bookvenvScriptspython.exe
cachedir: .pytest_cache
rootdir: D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2, inifile: pytest.ini
plugins: allure-pytest-2.13.2, collect-formatter2-0.1.3
collecting ... collected 3 items

test_xfail.py::Test_pytest::test_one XPASS                               [ 33%]
------start------
test_one方法执行

test_xfail.py::Test_pytest::test_two PASSED                              [ 66%]test_two方法执行

test_xfail.py::Test_pytest::test_three PASSED                            [100%]test_three方法执行


======================== 2 passed, 1 xpassed in 0.08s =========================

Process finished with exit code 0

(3)带条件的标记用例XFAIL，包含condition参数。

和@pytest.mark.skipif一样，它也可以接收一个Python表达式，表明只有满足条件时才标记用例。例如，只在pytest 3.6及以上版本标记用例，如果用例返回失败，则显示xfail。如果用例返回正确，则显示xpass。

代码如下：

import sys
import pytest


@pytest.mark.xfail(sys.version_info>=(3,6), reason="Python 3.6 API变更了")
def test_function():
    print("不同版本下执行不同测试用例")
    assert 2 == 1

执行结果一，断言失败显示XFAIL：

D:SynologyDriveCodeLearningWINpytest-bookvenvScriptspython.exe "C:/Program Files/JetBrains/PyCharm Community Edition 2022.3.2/plugins/python-ce/helpers/pycharm/_jb_pytest_runner.py" --path D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2	est_xfail_raise.py 
Testing started at 14:51 ...
Launching pytest with arguments D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2	est_xfail_raise.py in D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2

============================= test session starts =============================
platform win32 -- Python 3.7.7, pytest-5.4.1, py-1.11.0, pluggy-0.13.1 -- D:SynologyDriveCodeLearningWINpytest-bookvenvScriptspython.exe
cachedir: .pytest_cache
rootdir: D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2, inifile: pytest.ini
plugins: allure-pytest-2.13.2, collect-formatter2-0.1.3
collecting ... collected 1 item

test_xfail_raise.py::test_function XFAIL                                 [100%]不同版本下执行不同测试用例

@pytest.mark.xfail(sys.version_info>=(3,6), reason="Python 3.6 API变更了")
    def test_function():
        print("不同版本下执行不同测试用例")
>       assert 2 == 1
E       assert 2 == 1
E         +2
E         -1

test_xfail_raise.py:9: AssertionError


============================= 1 xfailed in 0.09s ==============================

Process finished with exit code 0

执行结果二，断言成功显示XPASS：

D:SynologyDriveCodeLearningWINpytest-bookvenvScriptspython.exe "C:/Program Files/JetBrains/PyCharm Community Edition 2022.3.2/plugins/python-ce/helpers/pycharm/_jb_pytest_runner.py" --path D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2	est_xfail_raise.py 
Testing started at 14:51 ...
Launching pytest with arguments D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2	est_xfail_raise.py in D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2

============================= test session starts =============================
platform win32 -- Python 3.7.7, pytest-5.4.1, py-1.11.0, pluggy-0.13.1 -- D:SynologyDriveCodeLearningWINpytest-bookvenvScriptspython.exe
cachedir: .pytest_cache
rootdir: D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2, inifile: pytest.ini
plugins: allure-pytest-2.13.2, collect-formatter2-0.1.3
collecting ... collected 1 item

test_xfail_raise.py::test_function XPASS                                 [100%]不同版本下执行不同测试用例


============================= 1 xpassed in 0.06s ==============================

Process finished with exit code 0

(4)将不符合预期的成功XPASS标记为失败，使用strict参数。

strict为关键字参数，默认值为False。

当strict=False时，如果用例执行失败，则结果标记为XFAIL，表示符合预期的失败。如果用例执行成功，则结果标记为XPASS，表示不符合预期的成功。

当strict=True时，如果用例执行成功，则结果将标记为FAILED，而不再标记为XPASS。

我们也可以在pytest.ini文件中配置，代码如下：

[pytest]
xfail_strict = ture

如果用例的失败不是因为所期望的异常导致的，pytest将会把测试结果标记为FAILED。

上面的代码将让assert 1==1断言成功，执行结果如下：

D:SynologyDriveCodeLearningWINpytest-bookvenvScriptspython.exe "C:/Program Files/JetBrains/PyCharm Community Edition 2022.3.2/plugins/python-ce/helpers/pycharm/_jb_pytest_runner.py" --path D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2	est_xfail_raise.py 
Testing started at 14:54 ...
Launching pytest with arguments D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2	est_xfail_raise.py in D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2

============================= test session starts =============================
platform win32 -- Python 3.7.7, pytest-5.4.1, py-1.11.0, pluggy-0.13.1 -- D:SynologyDriveCodeLearningWINpytest-bookvenvScriptspython.exe
cachedir: .pytest_cache
rootdir: D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2, inifile: pytest.ini
plugins: allure-pytest-2.13.2, collect-formatter2-0.1.3
collecting ... collected 1 item

test_xfail_raise.py::test_function FAILED                                [100%]不同版本下执行不同测试用例

test_xfail_raise.py:5 (test_function)
[XPASS(strict)] Python 3.6 API变更了


================================== FAILURES ===================================
________________________________ test_function ________________________________
[XPASS(strict)] Python 3.6 API变更了
---------------------------- Captured stdout call -----------------------------
不同版本下执行不同测试用例
=========================== short test summary info ===========================
FAILED test_xfail_raise.py::test_function
============================== 1 failed in 0.03s ==============================

Process finished with exit code 1

(5)标记为XFAIL的用例不再执行，包含run参数。

run为关键字参数，默认值为True。

当run=False时，pytest不会再执行测试用例，而直接将结果标记为XFAIL。

执行结果如下，有个xx，Results为1 xfailed。也就是一个被执行，而另一个未被执行。

使用PyCharm中的pytest运行工具执行，其结果有两个xfailed，并且显示其中一个是跳过执行NOTRUN。

D:SynologyDriveCodeLearningWINpytest-bookvenvScriptspython.exe "C:/Program Files/JetBrains/PyCharm Community Edition 2022.3.2/plugins/python-ce/helpers/pycharm/_jb_pytest_runner.py" --path D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2	est_xfail_raise.py 
Testing started at 15:23 ...
Launching pytest with arguments D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2	est_xfail_raise.py in D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2

============================= test session starts =============================
platform win32 -- Python 3.7.7, pytest-5.4.1, py-1.11.0, pluggy-0.13.1 -- D:SynologyDriveCodeLearningWINpytest-bookvenvScriptspython.exe
cachedir: .pytest_cache
rootdir: D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2, inifile: pytest.ini
plugins: allure-pytest-2.13.2, collect-formatter2-0.1.3
collecting ... collected 2 items

test_xfail_raise.py::test_xfail XFAIL                                    [ 50%]
cls = 
func = . at 0x000002AEDE117E58>
when = 'setup'
reraise = (, )

    @classmethod
    def from_call(cls, func, when, reraise=None) -> "CallInfo":
        #: context of invocation: one of "setup", "call",
        #: "teardown", "memocollect"
        start = time()
        excinfo = None
        try:
>           result = func()

....venvlibsite-packages_pytestrunner.py:244: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
....venvlibsite-packages_pytestrunner.py:217: in 
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
....venvlibsite-packagespluggyhooks.py:286: in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
....venvlibsite-packagespluggymanager.py:93: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
....venvlibsite-packagespluggymanager.py:87: in 
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
....venvlibsite-packages_pytestskipping.py:93: in pytest_runtest_setup
    check_xfail_no_run(item)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

item = 

    def check_xfail_no_run(item):
        """check xfail(run=False)"""
        if not item.config.option.runxfail:
            evalxfail = item._store[evalxfail_key]
            if evalxfail.istrue():
                if not evalxfail.get("run", True):
>                   xfail("[NOTRUN] " + evalxfail.getexplanation())
E                   _pytest.outcomes.XFailed: [NOTRUN]

....venvlibsite-packages_pytestskipping.py:111: XFailed

test_xfail_raise.py::test_function XFAIL                                 [100%]不同版本下执行不同测试用例

@pytest.mark.xfail(sys.version_info>=(3,6), reason="Python 3.6 API变更了")
    def test_function():
        print("不同版本下执行不同测试用例")
>       assert 2 == 1
E       assert 2 == 1
E         +2
E         -1

test_xfail_raise.py:14: AssertionError


============================= 2 xfailed in 0.17s ==============================

Process finished with exit code 0

(6)引起正确异常的失败，包含raises参数。

raises为关键字参数，默认值为None。可以指定为一个异常类或者多个异常类的元组，表明我们期望用例上报指定的异常。pytest.mark.xfail()也可以接收一个raises参数，用来判断用例是否因为一个具体的异常而导致失败，代码如下：

import sys
import pytest

@pytest.mark.xfail(raises=Exception)
def test_xfail():
    print(broken_fixture())

def broken_fixture():
    raise Exception("Sorry, it's中断异常")

执行结果如下，如果test_xfail()触发一个Exception异常，则用例标记为xfailed。如果没有，则正常执行test_xfail()。

执行结果一，引起的异常相同。

D:SynologyDriveCodeLearningWINpytest-bookvenvScriptspython.exe "C:/Program Files/JetBrains/PyCharm Community Edition 2022.3.2/plugins/python-ce/helpers/pycharm/_jb_pytest_runner.py" --path D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2	est_xfail_raise.py 
Testing started at 15:17 ...
Launching pytest with arguments D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2	est_xfail_raise.py in D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2

============================= test session starts =============================
platform win32 -- Python 3.7.7, pytest-5.4.1, py-1.11.0, pluggy-0.13.1 -- D:SynologyDriveCodeLearningWINpytest-bookvenvScriptspython.exe
cachedir: .pytest_cache
rootdir: D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2, inifile: pytest.ini
plugins: allure-pytest-2.13.2, collect-formatter2-0.1.3
collecting ... collected 1 item

test_xfail_raise.py::test_xfail XFAIL                                    [100%]
@pytest.mark.xfail(raises=Exception)
    def test_xfail():
>       print(broken_fixture())

test_xfail_raise.py:6: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def broken_fixture():
>       raise Exception("Sorry, it's中断异常")
E       Exception: Sorry, it's中断异常

test_xfail_raise.py:9: Exception


============================= 1 xfailed in 0.22s ==============================

Process finished with exit code 0

修改raise=IndexError后再执行，正常执行，显示XPASS，由于设置了strict为true，这种直接结果为失败，即1 failed，与预期引起的异常不同。

D:SynologyDriveCodeLearningWINpytest-bookvenvScriptspython.exe "C:/Program Files/JetBrains/PyCharm Community Edition 2022.3.2/plugins/python-ce/helpers/pycharm/_jb_pytest_runner.py" --path D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2	est_xfail_raise.py 
Testing started at 15:18 ...
Launching pytest with arguments D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2	est_xfail_raise.py in D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2

============================= test session starts =============================
platform win32 -- Python 3.7.7, pytest-5.4.1, py-1.11.0, pluggy-0.13.1 -- D:SynologyDriveCodeLearningWINpytest-bookvenvScriptspython.exe
cachedir: .pytest_cache
rootdir: D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2, inifile: pytest.ini
plugins: allure-pytest-2.13.2, collect-formatter2-0.1.3
collecting ... collected 1 item

test_xfail_raise.py::test_xfail FAILED                                   [100%]
test_xfail_raise.py:3 (test_xfail)
@pytest.mark.xfail(raises=IndexError)
    def test_xfail():
>       print(broken_fixture())

test_xfail_raise.py:6: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def broken_fixture():
>       raise Exception("Sorry, it's中断异常")
E       Exception: Sorry, it's中断异常

test_xfail_raise.py:9: Exception


================================== FAILURES ===================================
_________________________________ test_xfail __________________________________

    @pytest.mark.xfail(raises=IndexError)
    def test_xfail():
>       print(broken_fixture())

test_xfail_raise.py:6: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def broken_fixture():
>       raise Exception("Sorry, it's中断异常")
E       Exception: Sorry, it's中断异常

test_xfail_raise.py:9: Exception
=========================== short test summary info ===========================
FAILED test_xfail_raise.py::test_xfail - Exception: Sorry, it's中断异常
============================== 1 failed in 0.15s ==============================

Process finished with exit code 1

注意：如果test_xfail()测试成功，则用例的结果是xpass，而不是pass。pytest.raises适用于检查由代码故意引发的异常，而@pytest.mark.xfail()更适用于记录一些未修复的Bug。

(7)小结。

使用pytest.xfail标记用例

可以通过pytest.xfail方法在用例执行过程中直接将用例结果标记为XFAIL，并跳过剩余的部分，也就是该用例中的后续代码不会被执行。

应用的场景是功能未完成、已知有问题。除此之外，用例的执行需要前置条件或操作，如果前置条件或操作失败，那么我们就可以直接将该用例设为失败，也就是xfail。

同样可以为pytest.xfail指定一个reason参数，表明原因。

(1)功能未完成，已知有Bug未修改，执行预期失败。

(2)无效配置时直接显示预期失败，不进行后续测试。

在代码中增加下面测试方法，为大家展示无效配置时显示失败的效果。

xfail标记如何失效

(1)通过--runxfail参数让xfail标记失效。

通过命令行选项pytest --runxfail让xfail标记失效，使这些用例变成正常执行的用例，仿佛没有被标记过一样。同样，pytest.xfail()方法也将失效。

执行命令pytest--runxfail test_xfail.py会有以下结果，显示2个通过，1个失败。

执行结果如下：

PS D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2> pytest --runxfail .	est_xfail.py
========================================================================================================= test session starts =========================================================================================================
platform win32 -- Python 3.7.7, pytest-5.4.1, py-1.11.0, pluggy-0.13.1
rootdir: D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2, inifile: pytest.ini
plugins: allure-pytest-2.13.2
collected 3 items                                                                                                                                                                                                                       

test_xfail.py F..                                                                                                                                                                                                                [100%] 

============================================================================================================== FAILURES =============================================================================================================== 
________________________________________________________________________________________________________ Test_pytest.test_one _________________________________________________________________________________________________________
[XPASS(strict)] 该功能有Bug
-------------------------------------------------------------------------------------------------------- Captured stdout call --------------------------------------------------------------------------------------------------------- 

------start------
test_one方法执行
======================================================================================================= short test summary info ======================================================================================================= 
FAILED test_xfail.py::Test_pytest::test_one
===================================================================================================== 1 failed, 2 passed in 0.02s ===================================================================================================== 
PS D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2>  

(2)pytest.param用于将参数化标记为失败。

这部分知识与第4章的知识有交叉，建议学习完第4章内容之后回头学习这部分知识会更好理解。

@pytest.mark.parametrize功能将在第4章详细讲解。

pytest.param方法可用于为@pytest.mark.parametrize或者参数化的fixture指定一个具体的实参，它有一个关键字参数marks，可以接收一个或一组标记，用于标记这轮测试的用例。

代码如下：

import pytest
import sys

@pytest.mark.parametrize(
    ('n', 'expected'),
    [(2,1),
     pytest.param(2, 1, marks=pytest.mark.xfail(), id='XPASS'),
     pytest.param(0, 1, marks=pytest.mark.xfail(raises=ZeroDivisionError), id='XFAIL'),
     pytest.param(1, 2, marks=pytest.mark.skip(reason='无效的参数，跳出执行')),
     pytest.param(1, 2, marks=pytest.mark.skipif(sys.version_info<= (3,8), reason='请使用3.8及以上版本的Python.'))])
def test_params(n, expected):
    assert 2/n == expected

执行pytest-rA test_xfail_param.py，结果如下：

PS D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2> pytest -rA test_xfail_param.py
========================================================================================================= test session starts =========================================================================================================
platform win32 -- Python 3.7.7, pytest-5.4.1, py-1.11.0, pluggy-0.13.1
rootdir: D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2, inifile: pytest.ini
plugins: allure-pytest-2.13.2
collected 5 items                                                                                                                                                                                                                      

test_xfail_param.py .Fxss                                                                                                                                                                                                        [100%]

============================================================================================================== FAILURES =============================================================================================================== 
_________________________________________________________________________________________________________ test_params[XPASS] __________________________________________________________________________________________________________ 
[XPASS(strict)]
=============================================================================================================== PASSES ================================================================================================================ 
======================================================================================================= short test summary info =======================================================================================================
PASSED test_xfail_param.py::test_params[2-1]
SKIPPED [1] test_xfail_param.py:4: 无效的参数，跳出执行
SKIPPED [1] test_xfail_param.py:4: 请使用3.8及以上版本的Python.
XFAIL test_xfail_param.py::test_params[XFAIL]
FAILED test_xfail_param.py::test_params[XPASS]
========================================================================================== 1 failed, 1 passed, 2 skipped, 1 xfailed in 0.08s ========================================================================================== 
PS D:SynologyDriveCodeLearningWINpytest-booksrcchapter-2>