An "out-of-spec message" is a protocol message that is not valid according to the relevant protocol specification (e.g. RFC 2616 for HTTP).
Despite Postel's principle ("Be generous in what you accept"), we know that in practice network software often fails to be robust against unexpected messages. If the request is not designed to be an attack and has no malicious payload, the bad effects are normally limited to denial of service to other clients. Let's call a network component (origin server, proxy, middlebox) "fragile" if receiving an out-of-spec message causes it to fail to give correct service to other clients.
The policy question for this issue is whether Ooni-probe 1.0 should ship with net-tests that perform out-of-spec requests. (Whether Ooni-b can relay out-of-spec requests is also important, but not in the scope of this issue.)
Of the candidate tests in issue #89, the only one that sends out-of-spec messages according to its description seems to be "HTTP Invalid Request Line".
Here are the arguments I see against including such net-tests:
a) The operators of network components affected by out-of-spec messages may view them as attacks.
Note that many countries, including Western liberal democracies, have computer misuse laws that cover activities that are perceived to be exploiting bugs in network software. It may be difficult for someone suspected under such a law to defend themself, especially if the suspicion draws attention to their other activities. There have been cases where people were convicted under computer misuse laws for sending out-of-spec messages (or messages that were perceived to be out-of-spec), even when there is considerable doubt that they intended to perform an attack, e.g. http://www.theregister.co.uk/2005/10/11/tsunami_hacker_followup/. The risk in countries where the rule of law is less consistently applied is only likely to be worse.
It's also possible that other network users could be misidentified as having
originated the probe, and similarly be viewed as attackers.
b) Effects on fragile network components can deny service to other network users.
Suppose, for instance, that a censoring proxy fails because it receives an out-of-spec message. It may fail "open" (i.e. let through subsequent requests) or "closed" (i.e.
fail to correctly relay subsequent requests). If it fails closed, then the probe has
had the effect of making the censorship worse for other network users, at least in
the short term, which is obviously counterproductive.
This can happen whether or not the fragile component was part of a system
of network interference. The description of "HTTP Invalid Request Line" seems
to make the implicit assumption that only "interfering" network components are
likely to be fragile. This is wrong; transparent/caching HTTP proxies,
firewalls that are not intended to be interfering, and origin servers, can also
realistically be fragile.
c) Effects on fragile network components can result in misleading measurements.
In principle, any active network test can change the behaviour of the network.
In fact such changes are one of the things we want to measure! However, if the
intent is to measure network behaviour that could in principle have been
encountered by non-test clients in normal operation, that could result in
misleading overreporting of network interference.
Documenting these problems for specific tests only partially addresses point a), since a user can't really be expected to have enough information to determine their risk of being viewed as an attacker. It does not address points b) and c) at all.
If ooni-probe supports running such tests but the results are not stored by a given
collector (e.g. MLab's collectors), that would address point c) for that collector,
but not points a) and b).
(There are some other net-tests that send atypical messages that are clearly in-spec, but that seems to be much less of an issue; other client software will occasionally send messages that are atypical in the same way.)