IRTF Chair review of draft-irtf-hrpc-association-13

[mallory]

The HRPC chairs have requested that draft-irtf-hrpc-association-13 be published as an RFC on the IRTF stream. The IRTF publication process is described in RFC 5743, and comprises a review by the IRSG to ensure technical and editorial quality, followed by a check by the IESG to ensure the work does not conflict with IETF standards activities.

As IRTF Chair, I perform an initial review of all drafts submitted for publication on the IRTF stream before sending them for detailed review by the IRSG. This note provides my review comments, for discussion.

Authors, please can you also respond to this message to confirm that all necessary IPR disclosures, as described on https://irtf.org/policies/ipr, have been made?

Result:

Not ready

RFC 5743 compliance:

The draft does not follow the guidelines in RFC 5743

Comments:

This is an interesting draft, but I have some concerns about the way the work is presented and have identified some technical issues that should be considered. I sent these comments to the authors shortly after the San Francisco meeting, and Mallory and I have discussed on a number of occasions. I'm now sending them to the broader research group for input and discussion.

Overall, the draft is presented following the structure of a research paper, but the content does not follow a systematic research methodology. This makes it unsuitable for publication in its current form. However, restructuring the document to reflect more the form of an essay or a discussion piece would be straightforward and would play to the strengths of the material.

Considering presentation, the draft is structured as a research document that identifies a question to consider, conducts a literature review, develops an analysis, then draws some conclusions. This is reasonable structure for the draft, but I'm concerned that the methodology is not sufficiently rigorous to support the approach.

Firstly, while a literature review is developed, it is limited and highly selective, and the papers don't appear to have been chosen in a systematic manner such that they are representative of the literature and to avoid bias. The topic area is complex enough that I wouldn't expect the survey to be exhaustive, but I would expect an identified methodology for selecting papers to consider, along with a discussion of potential biases in the selection process and how they were mitigated when the selection of papers was made. The paper lacks such discussion, and following such a process would likely lead to the selection of a significantly different, and likely expanded, set of papers. I also have some concern that the discussion in this section adopts a less than rigorous style of argumentation in places that is not well suited to a research paper.

Secondly, the analysis of the protocols is similarly ad-hoc. There are many protocols that could be studied, but it's not clear why this particular set is chosen. As with the literature survey, there's no discussion of the protocol selection process and it's not obvious that the protocols and systems studied were chosen in a systematic way that provides a representative and unbiased sample.

Given these methodological shortcomings my view is that the draft is unsuitable for publication in its current form. And, while it could form the basis for a strong research contribution, I expect the effort needed to develop it into such would be significant, and the end result would be a very different draft.

With relatively minor changes, however, I believe the draft would make an excellent discussion piece: it's well written and makes a clear argument. Structuring it in such a manner would avoid the need for the systematisation that's required in a research contribution.

Regarding the technical issues, there are a number of errors and limitations in the discussion of protocols. The discussion of how WebRTC has the ability to function without a central server is not correct, for example, and the discussion of BGP peering and transit as a peer-to-peer protocol doesn't reflect the emergence of hypergiants and IXPs. The discussion of the technical issues is also highly selective at times and not representative of the range of protocols that exist. While the technical errors need to be corrected, the extent to which the discussion must consider the range of design choices depends on the chosen direction for the presentation.

Detailed Comments

Abstract

The first sentence is ungrammatical. Perhaps remove the word "whether"?

RFC 5743 requires that "There must be a statement in the abstract identifying it as the product of the RG". This is missing.

“It is therefore recommended that the potential impacts of Internet technologies should be assessed, reflecting recommendations of various UN bodies and international norms” – it's unclear who is recommended to make such an assessment, and an IRTF document cannot recommend the IETF does so. Perhaps "It is therefore recommended that further research be undertaken into the potential impact of Internet technologies…"? Alternatively, specify who the recommendation is aimed at.

Introduction

"the present document seeks to deepen the relationship between these human rights and Internet architecture, protocols, and standards" – seeks to deepen the relationship or seeks to deepen the understanding of the relationship? The latter is within scope for the IRTF; the former seems like an IETF activity.

Vocabulary Used

“Architecture The design of a structure” – this is a very generic definition. Would it be possible to relate it to network architecture more specifically?

“Autonomous Systems are the unit of routing policy in the modern world of exterior routing [RFC1930]” – I'm not sure this is quite right, although BGP routing allows ASes to reflect certain types of policy. Citing an RFC from 1996 as reflecting "the modern world" is problematic.

“Decentralization Implementation or deployment of standards, protocols or systems without one single point of control.” – this is a very restrictive definition of decentralisation that it's not clear is met by any deployed system. It may be more appropriate to discuss the extent to which a system has a single point of control since deployed systems have a range of designs that are neither fully centralised nor fully decentralised.

Research question

The research question is open ended and doesn’t pose a clear hypothesis that can be tested. This makes the later sections difficult to structure as a research paper. The question is highly appropriate as the basis for a discussion document or essay.

Methodology

As noted in the general comments, the methodology seems weak for a research document. That the literature review is not exhaustive is fine, given the size of the field, but I'd expect a clearly stated methodology for identifying the literature to survey that aims to find a representative sample and avoid biases. Similarly, the methodology for selecting protocols to consider seems arbitrary. If the draft was pitched as a discussion document to highlight particular concerns, then the approach taken seems reasonable. For a research document, the methodology should be more rigorous.

Literature survey

“The rights to peaceful assembly and the freedom of association are defined and guaranteed in national law and international treaties; however, in this document we limit ourselves to international treaties. “ - this restriction is significant and could be better justified.

The presentation, on page 12 especially but not only, does not adopt the tone and style of a research contribution. If the draft was presented as an essay or discussion piece that would not matter. If it's intended as a research contribution, then it should adopt a more rigorous, strongly justified, neutral point of view and a more systematic approach.

Section 5.3 ("Specific questions raised from the literature review") lists a set of questions to consider. While these may be reasonable questions, it’s not clear how the follow from the literature review. Again, as a research contribution, the draft should clearly evidence why these are appropriate questions.

Analysis

"To answer our research question regarding how Internet architecture enables and/or inhibits such human rights…" – that’s not the stated research question

This section includes a range of case studies, but doesn’t clearly motivate why these particular examples were chosen, or how they relate to the literature survey and the specific questions identified. This is not such a concern for a discursive essay that but a research contribution would need to follow a more systematic approach.

The discussion of spam could include significantly more technical depth. Many of the members of the former Anti-Spam Research Group are still active in the IETF and IRTF community and might be willing to provide input.

The inclusion of IRC as an example is surprising, given how infrequently it’s used these days. There are many more widely used messaging or social media services that could be used to illustrate the same points.

The discussion of WebRTC notes the “ability to function without a central server” as "a strong facilitator of freedom of association and assembly", but WebRTC requires a central server to operate (the media data is sometimes sent in a peer-to-peer manner, but there is always a central server involved to setup the communication).

The discussion of WebRTC notes the privacy risks of collecting IP addresses, but doesn't mention that the IETF considered this issue when it was identified and made recommendations on how to address it that have been widely implemented [RFC 8828]. Further, many of the other protocols identified have similar information leaks that are not mentioned.

The discussion of BGP as a peer to peer protocol, in Section 6.3.5, reflects a model of peering and transit that is of increasingly limited applicability in the Internet. Geoff Huston makes this argument clearly in his "Death of Transit" talk.

BitTorrent is an example of a peer to peer protocol, but there are other peer to peer protocols with quite different properties that aren’t mentioned. In particular, many BitTorrent systems use a central tracker while other designs may be more decentralised.

The discussion of internationalisation confuses internationalisation of the protocol elements and internationalisation of the user visible features. Both are important but combining them into a single discussion confuses rather than helps. When considering internationalisation of protocol elements, the document discusses the issue from a social perspective but neglects that there are strong technical arguments that such internationalisation hinders interoperability and evolution of the network. I might agree that the social arguments are often neglected by the protocol development community, and that this is problematic, but a draft that highlights the social issues but ignores the technical concerns is also problematic.