summaryrefslogtreecommitdiff
path: root/Documentation/dev-tools
diff options
context:
space:
mode:
authorLinus Torvalds <torvalds@linux-foundation.org>2022-03-21 14:13:25 -0700
committerLinus Torvalds <torvalds@linux-foundation.org>2022-03-21 14:13:25 -0700
commit346658a5e189f35b61206fed2f9f60d4bb34b881 (patch)
treeeff44d85cbd92095d1d17271489ca9d89a239824 /Documentation/dev-tools
parentd2eb5500f1d916c9b0815cdc63c48a6d615532cc (diff)
parent75c05fabb873367d9e64f063dda8a310c4c58826 (diff)
Merge tag 'docs-5.18' of git://git.lwn.net/linux
Pull documentation updates from Jonathan Corbet: "It has been a moderately busy cycle for documentation; some of the highlights are: - Numerous PDF-generation improvements - Kees's new document with guidelines for researchers studying the development community. - The ongoing stream of Chinese translations - Thorsten's new document on regression handling - A major reworking of the internal documentation for the kernel-doc script. Plus the usual stream of typo fixes and such" * tag 'docs-5.18' of git://git.lwn.net/linux: (80 commits) docs/kernel-parameters: update description of mem= docs/zh_CN: Add sched-nice-design Chinese translation docs: scheduler: Convert schedutil.txt to ReST Docs: ktap: add code-block type docs: serial: fix a reference file name in driver.rst docs: UML: Mention telnetd for port channel docs/zh_CN: add damon reclaim translation docs/zh_CN: add damon usage translation docs/zh_CN: add admin-guide damon start translation docs/zh_CN: add admin-guide damon index translation docs/zh_CN: Refactoring the admin-guide directory index zh_CN: Add translation for admin-guide/mm/index.rst zh_CN: Add translations for admin-guide/mm/ksm.rst Add Chinese translation for vm/ksm.rst docs/zh_CN: Add sched-stats Chinese translation docs/zh_CN: add devicetree of_unittest translation docs/zh_CN: add devicetree usage-model translation docs/zh_CN: add devicetree index translation Documentation: describe how to apply incremental stable patches docs/zh_CN: add peci subsystem translation ...
Diffstat (limited to 'Documentation/dev-tools')
-rw-r--r--Documentation/dev-tools/ktap.rst49
1 files changed, 32 insertions, 17 deletions
diff --git a/Documentation/dev-tools/ktap.rst b/Documentation/dev-tools/ktap.rst
index 878530cb9c27..5ee735c6687f 100644
--- a/Documentation/dev-tools/ktap.rst
+++ b/Documentation/dev-tools/ktap.rst
@@ -1,8 +1,8 @@
.. SPDX-License-Identifier: GPL-2.0
-========================================
-The Kernel Test Anything Protocol (KTAP)
-========================================
+===================================================
+The Kernel Test Anything Protocol (KTAP), version 1
+===================================================
TAP, or the Test Anything Protocol is a format for specifying test results used
by a number of projects. It's website and specification are found at this `link
@@ -68,7 +68,7 @@ Test case result lines
Test case result lines indicate the final status of a test.
They are required and must have the format:
-.. code-block::
+.. code-block:: none
<result> <number> [<description>][ # [<directive>] [<diagnostic data>]]
@@ -117,32 +117,32 @@ separator.
Example result lines include:
-.. code-block::
+.. code-block:: none
ok 1 test_case_name
The test "test_case_name" passed.
-.. code-block::
+.. code-block:: none
not ok 1 test_case_name
The test "test_case_name" failed.
-.. code-block::
+.. code-block:: none
ok 1 test # SKIP necessary dependency unavailable
The test "test" was SKIPPED with the diagnostic message "necessary dependency
unavailable".
-.. code-block::
+.. code-block:: none
not ok 1 test # TIMEOUT 30 seconds
The test "test" timed out, with diagnostic data "30 seconds".
-.. code-block::
+.. code-block:: none
ok 5 check return code # rcode=0
@@ -174,6 +174,13 @@ There may be lines within KTAP output that do not follow the format of one of
the four formats for lines described above. This is allowed, however, they will
not influence the status of the tests.
+This is an important difference from TAP. Kernel tests may print messages
+to the system console or a log file. Both of these destinations may contain
+messages either from unrelated kernel or userspace activity, or kernel
+messages from non-test code that is invoked by the test. The kernel code
+invoked by the test likely is not aware that a test is in progress and
+thus can not print the message as a diagnostic message.
+
Nested tests
------------
@@ -186,13 +193,16 @@ starting with another KTAP version line and test plan, and end with the overall
result. If one of the subtests fail, for example, the parent test should also
fail.
-Additionally, all result lines in a subtest should be indented. One level of
+Additionally, all lines in a subtest should be indented. One level of
indentation is two spaces: " ". The indentation should begin at the version
line and should end before the parent test's result line.
+"Unknown lines" are not considered to be lines in a subtest and thus are
+allowed to be either indented or not indented.
+
An example of a test with two nested subtests:
-.. code-block::
+.. code-block:: none
KTAP version 1
1..1
@@ -205,7 +215,7 @@ An example of a test with two nested subtests:
An example format with multiple levels of nested testing:
-.. code-block::
+.. code-block:: none
KTAP version 1
1..2
@@ -224,10 +234,15 @@ An example format with multiple levels of nested testing:
Major differences between TAP and KTAP
--------------------------------------
-Note the major differences between the TAP and KTAP specification:
-- yaml and json are not recommended in diagnostic messages
-- TODO directive not recognized
-- KTAP allows for an arbitrary number of tests to be nested
+================================================== ========= ===============
+Feature TAP KTAP
+================================================== ========= ===============
+yaml and json in diagnosic message ok not recommended
+TODO directive ok not recognized
+allows an arbitrary number of tests to be nested no yes
+"Unknown lines" are in category of "Anything else" yes no
+"Unknown lines" are incorrect allowed
+================================================== ========= ===============
The TAP14 specification does permit nested tests, but instead of using another
nested version line, uses a line of the form
@@ -235,7 +250,7 @@ nested version line, uses a line of the form
Example KTAP output
--------------------
-.. code-block::
+.. code-block:: none
KTAP version 1
1..1