Thread Certification Fail for MTD

[WestChau]

Describe the bug SED.MED 3.5 Test inconclusive

To Reproduce Information to reproduce the behavior, including:
The cli cmd "ot test tmforiginfilter" depends on
OPENTHREAD_FTD & OPENTHREAD_CONFIG_REFERENCE_DEVICE_ENABLE
But our MTD lib don't want to enable OPENTHREAD_FTD

Expected behavior A clear and concise description of what you expected to happen.
The cli cmd "ot test tmforiginfilter" don't depends on OPENTHREAD_FTD

Console/log output If applicable, add console/log output to help explain your problem.
COM78 command: ot test tmforiginfilter disable
COM78 FUNC __executeCommand failed: Error 35: InvalidCommand
COM78 command: ot test tmforiginfilter disable
COM78 FUNC __executeCommand failed: Error 35: InvalidCommand
COM78 command: ot test tmforiginfilter disable

[abtink]

@WestChau , some questions:

Is the MTD the device-under-test (DUT)?

• If so, I would not recommend enabling and using OPENTHREAD_CONFIG_REFERENCE_DEVICE_ENABLE on it. The OPENTHREAD_CONFIG_REFERENCE_DEVICE_ENABLE is intended for reference devices. Reference devices can be used for cert tests as devices interacting with the DUT.

Why does the MTD need to use the tmforiginfilter during a cert test if acting as DUT?

---

Beyond the above, updating the code to allow these CLI commands on an MTD would be simple. I will submit a PR to allow this, but I would suggest understanding why this is needed or used in the first place, as it does not seem like the right thing to do to me, especially if the MTD is the DUT.

---

Edit: Submitted

• [cli] allow ccm, test, and tvcheck commands on MTD reference de… #11961

This change goes beyond a change in cli and requires updating all the corresponding public APIs to make them available for MTD. I closed the above PR. Unless there is a clear need for this, I would not recommend making this change.

[WestChau]

In Test Harness, send udp will disable tmforiginfilter firstly.

@API
def send_udp(
self,
interface,
destination,
port,
payload="12ABcd",
return_result=False,
spiff_udp_ip="",
):
"""payload hexstring"""
assert payload is not None, "payload should not be none"
assert interface == 0, "non-BR must send UDP to Thread interface"
self._udpOpen(spiff_udp_ip, port)
time.sleep(0.5)
**cmd = "test tmforiginfilter disable"**
self._executeCommand(cmd)
cmd = "udp send %s %s -x %s" % (destination, port, payload)
if not return_result:
result = self.__executeCommand(cmd)[-1] == "Done"
else:
result = self.__executeCommand(cmd)[0]
_cmd = "test tmforiginfilter enable"
self.__executeCommand(_cmd)
return result

[abtink]

@WestChau, you did not answer my question about whether the MTD is the DUT.

In a certification test, there is a DUT (whose behavior is being validated and tested) and other reference devices interacting with the DUT as part of the test.

Reference devices involved in a test must use an FTD build.

As I said, the DUT should not generally be built as a reference device with OPENTHREAD_CONFIG_REFERENCE_DEVICE_ENABLE, so the tmforigin and any other test (reference device-specific) commands should not be needed on the DUT in the first place.

Allowing or using such APIs/commands on the DUT is actually dangerous, since these APIs often modify the stack's behavior in ways that make it non-compliant with the Thread specification. Enabling and using them on the DUT would defeat the purpose of certification.

[WestChau]

HI @abtink
yes, MTD is DUT, but there is many certification test case is related to test in nordic/Slicon testharness THCI , even FTD is DUT

[jwhui]

Reference device firmware requires FTD build. See the build script to learn more about the reference device builds:
https://github.com/openthread/ot-reference-release/blob/main/script/make-firmware.bash