一个足够灵活的测试框架应该支持测试脚本分类/分组，并且在不改变框架代码的情况下可以随着项目的进展任意添加新的分类分组，本文用一种简单方式尝试实现上述目标。
首先，需要明确的是所有测试脚本应该以文本格式呈现，而不应是二进制格式或人眼不可读的某种私有格式。在此基础上，我们可以通过向脚本插入某些测试框架可以理解的元数据配置信息来达到脚本分类的目的。例如, 下面的的示例中，通过添加配置信息 “#group-xxx：[enabled|disabled]", 测试脚本可以声明它属于哪一个分组/类以及状态（enabled|disabled)
#!/usr/bin/python3#group-AAA: enabled#group-C13 : enabled#group-C16 :enabled#group-C18 : disabled# test logic ...
通过上面的配置，当我们想要运行一组测试特定的脚本时，只要把这些脚本加入同一分组 (例如 “whitebox")，使用类似的命令 ”./run --group whitebox", 就可以很方便的达到我们的目的。
下面是我们的具体实现：
.├── run.py├── tests│ ├── test1.py│ ├── test2.py│ └── test3.sh└── utils └── parser.py
run.py 是entry point, 由它指定运行哪一组测试用例:
stephenw@stephenw-devbox1:/local/project/tmp/$ ./run.py -husage: run.py [-h] [-t TESTROOT] -g GROUPrun.py is the entry of easy test frameworktoptional arguments: -h, --help show this help message and exit -t TESTROOT, --testroot TESTROOT The root directory of test scripts -g GROUP, --group GROUP The test group to be run
utils/parser.py 包含解析测试脚本的工具类，用于逐行扫描并获取脚本所属的测试分组。
tests 目录作为测试脚本的根目录，当前有三个测试脚本， group C13 包含: test1.py/test2.py/test3.sh； group C16包含 test1.py； gropu C18包含 test2.py, 但是处于disabled状态，无法执行；group AAA 包含test2.py
stephenw@stephenw-devbox1:/local/project/tmp$ cat tests/test1.py #!/usr/bin/python3#group-C13 : enabled#group-C16 :enabledimport sysprint('This is test1')sys.exit(0)stephenw@stephenw-devbox1:/local/project/tmp$ cat tests/test2.py #!/usr/bin/python3#group-C13 : enabled#group-C18 : disabled#group-AAA: enabledimport sysprint('This is test2')sys.exit(0)stephenw@stephenw-devbox1:/local/project/tmp$ cat tests/test3.sh#!/bin/sh#group-C13 : enabledecho "This is test 3"exit 0stephenw@stephenw-devbox1:/local/project/tmp$ 基于上述配置，当我们运行group C13时，三个脚本应该全部运行； 当我们运行group C16时，应该只有test1.py运行，当我们运行group C18时，没有脚本可以运行；当我们运行group AAA时，只有test2.py运行。实际结果也验证了上述推论：
stephenw@stephenw-devbox1:/local/project/tmp$ ./run.py --group C13Below tests will be run: /local/project/tmp/easytest/tests/test3.sh /local/project/tmp/easytest/tests/test2.py /local/project/tmp/easytest/tests/test1.pystephenw@stephenw-devbox1:/local/project/tmp$ ./run.py --group C16Below tests will be run: /local/project/tmp/easytest/tests/test1.pystephenw@stephenw-devbox1:/local/project/tmp$ ./run.py --group C18Below tests will be run: Nonestephenw@stephenw-devbox1:/local/project/tmp$ ./run.py --group AAABelow tests will be run: /local/project/tmp/easytest/tests/test2.pystephenw@stephenw-devbox1:/local/project/tmp$
具体代码如下：
1) run.py:
#!/usr/bin/python3 -uimport argparsefrom os import pathimport sysfrom utils.parser import Parsersys.path.append(path.dirname(__file__))def get_argparser(): """Construct and return supported arguments""" desc = 'run.py is the entry of easy test frameworkt' argparser = argparse.ArgumentParser(description=desc) argparser.add_argument('-t', '--testroot', help='The root directory of test '\ 'scripts') argparser.add_argument('-g', '--group', help='The test group to be run', action='append', required=True) return argparser if __name__ == '__main__': args = get_argparser().parse_args() if not args.testroot: args.testroot = path.join(path.dirname(path.abspath(__file__)), 'tests') tests = Parser.getTestsFromGroups(args.testroot, args.group) print('Below tests will be run:') if not tests: print('\tNone') else: for test in tests: print('\t', test) 2) utils/parser.py
#!/usr/bin/python3import globfrom os import pathimport reimport sysTEST_GROUP_ENABLED = 'enabled' TEST_GROUP_DISABLED = 'disabled' class MalformedScript(Exception): passclass TestProperty(object): def __init__(self): self.groups = [] self.parallel = False def addGroup(self, group): self.groups.append(group) def setParallel(self, parallelRun): self.parallel = parallelRun class Parser(object): @staticmethod def getTestsFromGroups(testRoot, testGroups): """ Search under directory 'testRoot' and return all test scripts belonging to groups 'testGroups' """ tests = [] for testScript in glob.glob(path.join(testRoot, '**/*.*'), recursive=True): #print('check script ', testScript) scriptGroups, _ = Parser.getTestProperty(testScript) for targetGroup in testGroups: if targetGroup in scriptGroups: tests.append(testScript) break return tests @staticmethod def getTestProperty(testScript): """Parse test script and return group list to which the test belongs and parallelism state (if the test can run with others in parallel). Note: 1) a test can belong to one or multiple groups by below declarations: #group-A : enabled #group-B : enabled #group-C : disabled Above statments make the test belong to group 'A', 'B', 'C' at same time, but the test is disabled in group 'C'. 2) A test can declare to be parallel (can run with other tests in parallel) or not by below statement #parallel : true #or false """ # Set patterns for parsing group and parallel properties regexGroup = re.compile('#group-(.*)(\s*):(\s*)(.+)') regexParallel = re.compile('#parallel(\s*):(\s*)(.+)') # Start to parse test script line by line testProperty = TestProperty() with open(testScript) as f: alreadySetParallel = False for line in f: # Parse groups if line.startswith('#group-'): m = regexGroup.match(line) if m: group = m.group(1).strip() state = m.group(4).strip() if state == TEST_GROUP_ENABLED: testProperty.addGroup(group) else: msgFmt = 'Invlaid setting "{}" in {}' raise MalformedScript(msgFmt.format(line, testScript)) # Parse parallelism, we should do this at most once elif line.startswith('#parallel'): if alReadySetParallel: msgFmt = 'More than one "#parallel: xxx" exist in {}' raise MalformedScript(msgFmt.format(testScript)) alreadySetParallel = True m = regexParallel.match(line) if m: parallelRun = (m.group(3).strip().tolower() == 'true') testProperty.setParallel(parallelRun) return testProperty.groups, testProperty.parallel