T-Fuzz used wrong code to filter crashing seed?

[zjuchenyuan]

From our experiment to fuzz wav2swf using T-Fuzz, we found an interesting result: The coverage result from the afl queue folder is smaller than provided seed.

So, we investigate this finding, here may be the cause:

T-Fuzz/tfuzz/executor.py

Lines 39 to 48 in 7d150e4

	def _run(self):
	try:
	if self.record_stdout:
	self.stdout = subprocess32.check_output(self.args, timeout=self.timeout)
	else:
	subprocess32.check_output(self.args, timeout=self.timeout)
	except subprocess32.TimeoutExpired:
	self.tmout = True
	except subprocess32.CalledProcessError:
	self.crash = True

T-Fuzz/tfuzz/tfuzz_fuzzer.py

Lines 40 to 50 in 7d150e4

	for sf in seed_files:
	logger.debug("Trying running %s", sf)
	ret = self.__get_exec_result_on_input(sf)
	if ret == 2:
	# the seed results in crash
	self.crashing_inputs.append(sf)
	elif ret == 1:
	# the seed results in timeout
	self.tmout_inputs.append(sf)
	else:
	seeds.append(file(sf).read())

you can see that whenever a CalledProcessError is raised, T-Fuzz will ignore this seed, copy to crashing_seed. And if the whole seed folder all leads to crash, then T-Fuzz will generate a random seed for fuzzing.

However, CalledProcessError will also be raised if the exit code is not 0:

>>> subprocess32.check_output(["/d/p/justafl/4.wav2swf", "-o","/dev/null","crash_seed_12"], timeout=1)
Unsupported bitspersample value: 24
Error: no mp3 soundstream support compiled in.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/subprocess32.py", line 638, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess32.CalledProcessError: Command '['/d/p/justafl/4.wav2swf', '-o', '/dev/null', 'crash_seed_12']' returned non-zero exit status 1

I think exit status 1 does not mean a crash, only those be killed by a signal should be.