chapter 4. research methodology · chapter 4. research methodology 4.1 introduction in light of the...

21
83 Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship about net neutrality, this thesis seeks to make a substantive, positive contribution by leveraging empirical evidence about the behavior of network operators and telecommunications regulators since the mid-2000s. Network operators around the world have been making decisions about how to manage their networks while regulators have grappled with the question of whether their intervention is warranted to safeguard nondiscrimination. The choices that both sets of parties have made – what they did, and, more importantly, why they did it – offer a rich basis for understanding the merits of existing normative arguments and theoretical modeling assumptions. A close investigation of these decisions indicates that the outcomes do not necessarily align with what some of the most prevalent theories would have predicted, particularly as they pertain to the respective roles of competition and regulation as deterrents against discriminatory behavior. Given the limited number of countries in which national regulatory policy concerning broadband traffic management has received significant attention, I chose a cross-national comparative approach so as to draw out explanations for divergent regulatory and marketplace outcomes. This study makes use of the two prominent cases of the UK and the US, neither of which appears to fit the predicted paradigm. This thesis seeks explanations for these discrepancies using qualitative methods – elite interviews, documentary analysis, and participant observation – to gain an in-depth understanding of the traffic management decisions that large broadband providers have made and the ways in which regulators have responded in both countries. By exploring qualitative empirical evidence pertaining to specific decisions that broadband providers and regulators have made throughout the last decade, the first-order goal is to elucidate which factors have

Upload: others

Post on 22-Jul-2020

12 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

83

Chapter 4. Research Methodology

4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic

scholarship about net neutrality, this thesis seeks to make a substantive, positive contribution

by leveraging empirical evidence about the behavior of network operators and

telecommunications regulators since the mid-2000s. Network operators around the world

have been making decisions about how to manage their networks while regulators have

grappled with the question of whether their intervention is warranted to safeguard

nondiscrimination. The choices that both sets of parties have made – what they did, and, more

importantly, why they did it – offer a rich basis for understanding the merits of existing

normative arguments and theoretical modeling assumptions.

A close investigation of these decisions indicates that the outcomes do not necessarily align

with what some of the most prevalent theories would have predicted, particularly as they

pertain to the respective roles of competition and regulation as deterrents against

discriminatory behavior. Given the limited number of countries in which national regulatory

policy concerning broadband traffic management has received significant attention, I chose a

cross-national comparative approach so as to draw out explanations for divergent regulatory

and marketplace outcomes. This study makes use of the two prominent cases of the UK and

the US, neither of which appears to fit the predicted paradigm.

!This thesis seeks explanations for these discrepancies using qualitative methods – elite

interviews, documentary analysis, and participant observation – to gain an in-depth

understanding of the traffic management decisions that large broadband providers have made

and the ways in which regulators have responded in both countries. By exploring qualitative

empirical evidence pertaining to specific decisions that broadband providers and regulators

have made throughout the last decade, the first-order goal is to elucidate which factors have

Page 2: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

84

been most determinative of traffic management outcomes and how those factors square with

the existing body of regulatory and economic theory concerning net neutrality. More broadly,

this research seeks to identify the mix of institutional factors that shape the regulatory

landscape in cases where traditional principles of competition and regulation fall short.

4.2 Comparative Approach For a number of reasons, a qualitative comparative approach is a natural fit for achieving

these goals. The comparative method is a research strategy especially well suited to problems

that exhibit a small number of cases and many variables (Lijphart 1971). Thus cross-national

research often takes a comparative form given the limited number of nations in the world that

exhibit a particular set of traits or circumstances. Events or policies affecting entire nations

also tend to emerge from complex processes, fitting the “many variables” mold.

Both of these characteristics are exhibited in the case of regulatory policy pertaining to

Internet traffic management. The issues of discriminatory treatment of Internet traffic and

regulatory responses to it have garnered substantial consideration in a limited number of

countries. Where these issues have arisen, a diversity of actors, institutions, and events have

come together in a mixture of different ways to determine the ultimate market and policy

outcomes. Comparative research thrives when the subject of study requires deep knowledge

and familiarity with how the parts of each case contribute to the whole (Ragin 1994). The

complexity of factors contributing to traffic management outcomes warrants this kind of in-

depth analysis.

The comparative method is also highly suitable in cases where traditional controlled

experimentation is infeasible, as is often the case when studying policy issues on a national

scale. Instead, as Heidenheimer et al. (1990, 1) note, “the fact that different countries often

adopt alternative strategies for dealing with similar problems represents a kind of natural

experiment.” Studying multiple countries side-by-side helps to dissect causes and effects in a

way that is not possible from studying a single country in isolation, especially when one

Page 3: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

85

country chose a different course than the other (Rose 1973), as is the case with UK and US

net neutrality policy.

4.2.1 Case Selection Both scholarly debate and policymakers’ attention to net neutrality originated in the United

States in the early 2000s. A number of events conspired in the mid-2000s to elevate the

profile of the issue, sparking legislative and regulatory initiatives and resulting in extensive

grassroots mobilization and mass media coverage. Since then, regulators, policymakers, and

other stakeholders around the world have engaged with the issue to various degrees. Major

regulatory and legislative activities have taken place in a number of countries, including

Canada, Chile, Finland, France, Japan, the Netherlands, Norway, Peru, Singapore, Slovenia,

and South Korea. A number of other countries’ regulators have hosted consultations or

requested public input on the issue, including in Croatia, Germany, Sweden, and the United

Kingdom.

I selected the United States and the United Kingdom for this study so as to create a rich basis

for comparison. The choice of these two cases follows comparative approaches broadly based

on John Stuart Mill’s “Method of Difference” (Skocpol and Somers 1980). The Method of

Difference involves contrasting cases that are similar in some respects but where the

emphasis is on key differentiating characteristics, and where the phenomenon to be studied

exists in one case but not the other (Skocpol 1979). This structure is especially well suited to

research that takes one or more explanatory problems as its focus and seeks to draw out

causal relationships, as this thesis seeks to do.

As a general matter, the US and the UK are similar when it comes to broadband: both are

advanced economies that have had consistently above-average broadband penetration since

the mid-2000s compared to their OECD counterparts (OECD 2011). But as explained in

Chapter 1, the two countries are rather opposite extremes in three crucial respects relevant to

broadband traffic management: competition, discrimination, and regulation. Conducting the

Page 4: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

86

comparison allows for analysis of the effects of these important differences, while their

similarities with each other and with other advanced economies provides a basis to draw

generalized conclusions outside of the two countries’ national boundaries.

4.3 Data Collection One of the key contributions of this thesis is the synthesis of empirical evidence collected via

participant observation, elite interviews, and documentary analysis. Because of timing

constraints at the site where participant observation was conducted, participant observation

formed the first phase of data collection. This was followed by dedicated periods of

interviewing and documentary analysis in the US and the UK, respectively. Each of the latter

two phases began with some preliminary documentary analysis and the development of a list

of potential interview candidates. As candidates began to accept invitations to interview, I

developed individualized interview protocols, and as interviews were completed, I began to

transcribe them. Transcription and documentary analysis continued for a number of months

after the final interview was completed in each country.

4.3.1 Participant Observation For a period of five months in 2010, I spent several days per month at a UK ISP observing a

selection of meetings, conference calls, and interactions between members of a corporate

traffic management research team. Participant observation was arranged as part of my joint

work with the research team on a project at the company that sought to characterize the

effects of traffic management on network traffic and on company expenditures. I was

working with team members as they explored different traffic management options for the

company and observing meetings and interactions between the team members while on site at

the ISP. Thus my role was as participant-as-observer (Gold 1957) – I simultaneously

contributed to the project team while developing relationships with informants and observing

their activities and interactions.

Page 5: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

87

Participant observation is a powerful tool for investigating the way that individuals think,

feel, and act within a natural setting. It is particularly well suited in cases where the processes

and interactions of interest may be obscured from public view or are poorly understood

(Waddington 1994). For this study, participant observation provided an intimate perspective

of traffic management decisions as they were being made, complementing the more reflective

insights obtainable via the other two methods. It was also a valuable means of learning the

language of telecom company culture, a necessary step in comprehending the mental models

of study participants (Burgess 1995). Because participant observation comprised the first

phase of the research, the insights and vocabulary that I gained at the research site were used

to shape the framing and language used later on in developing interview protocols,

particularly the UK protocols since industry jargon can be highly country-specific.

My process of data collection involved taking detailed field notes at the conclusion of each

day at the research site. My observations focused on the people involved in researching,

discussing, and setting internal policy about traffic management; the processes involved in

deciding traffic management strategy from both technical and policy perspectives; the

substantive discussions of different traffic management strategies; and the feelings and

emotions of the informants. Immersion in the field for the purpose of observation is a

naturally inductive process (Waddington 1994); as this was the first data collection phase, I

aimed to gather a diversity of initial observations that could inform and refine my hypothesis

formulation and research direction as I moved into the later phases.

Because I was in the role of participant-as-observer, it was never my aim to act as a

completely passive observer of the team’s work. I contributed to the team by analyzing

existing data about network traffic in the presence of traffic management and creating models

to predict future traffic growth and its associated costs under a variety of hypothetical

scenarios in which the company’s traffic management solution was altered or removed. Thus,

my work for the team was primarily technical and descriptive, although it did involve

conceptualizing arguments for and against the reduction or removal of application-specific

Page 6: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

88

traffic management. While the primary consequences of this work for me as a researcher were

purely educational (particularly since I began the research process with a lot to learn about

traffic management and the UK broadband industry), it also allowed me to develop an

appreciation of the team’s opinions about traffic management, including that there may be

valid reasons to either continue or discontinue application-specific traffic management. I

recognized the potential for these opinions to bias my research going forward, and sought at

every turn thereafter in the data collection process to question whether newly collected data

contradicted or supported what I had observed and discussed with the research team. I also

interviewed five employees from the company as part of the interviewing phase, only one of

whom was also a member of the research team, so as to obtain an in-depth understanding of

the diversity of views about traffic management within the company.

My points of contact on the research team were fully aware that I was conducting participant

observation and provided their consent to be observed as part of my thesis work. They

understood the value of being able to contribute to a larger study of traffic management

industry-wide and thus were receptive to questions about how traffic management was

handled within the company. During the process of writing the thesis, I consulted with the

team leader numerous times to ensure that I was accurately reporting information that I had

learned from observation and to verify that I was not revealing any information that was

sensitive for the company. Members of the research team are not directly quoted in this

thesis; the terms “the observed research team” or “the observed network” are used to indicate

information reported based on participant observation.

I was cognizant of the potential impact that my own participation had on the people and

processes that I was studying. My aim was to remain as unobtrusive as possible while seeking

to connect with informants on a human level so as to be able to observe them and their work

environment in their natural states (Schwartz and Schwartz 1955). Furthermore, because

participant observation was conducted at only one ISP, in one country, and with access to a

handful of employees, the extent to which inferences and conclusions were drawn solely from

Page 7: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

89

the observation data was limited. Throughout the study I viewed participant observation as a

means to triangulate my findings and as a vivid way to understand the day-to-day tensions

and constraints that can contribute to traffic management decisions.

4.3.2 Elite Interviews I opted to use interviews as a primary method of data collection to elucidate the decision-

making processes of broadband providers and regulators. Interviews create the possibility of

“seeing through others’ eyes,” providing researchers with a window into the depth and

breadth of perspective of experts in the field (Bryman 2008, 465). Because traffic

management and regulatory decisions have been occurring over a number of years, I chose

interviews as an effective strategy for both reconstructing the circumstances leading to firms’

and regulators’ choices and illuminating each constituency’s perception of the other as they

have developed over time.

Interviews were vital to this research because all of the motivations and tensions behind

traffic management decision-making are not necessarily matters of public record. As Useem

(1995, 20) notes, “to answer some of the most vexing questions about business, we often have

little choice but to enter that world directly.” While public documentation provided a window

into the formal, packaged rationales and justifications used by corporations and governments

for their traffic management choices, interviews were used to elicit more raw, private, and

unvarnished accounts of how those choices were constructed.

I conducted 70 semi-structured interviews for this thesis. These were “elite” interviews in the

sense that participants were selected based on their expertise and standing in their

professional fields (Marshall and Rossman 2010). The participant pool was comprised

primarily of senior managers, corporate executives, and mid- or high-ranking government

officials. Participants were drawn from two populations: current and former employees of

broadband network operators, and current and former employees of regulatory or

policymaking bodies, with a small number (three) of current and former employees of

Page 8: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

90

network equipment vendors also included. Six of the interviewees had relevant career

experience both in industry and as a regulator or civil servant.

Any inquiry into organizational decision-making faces inherent limitations related to the

biases and potential hesitancy of participants to provide frank views. I employed a number of

mitigation strategies to help ensure that the data collected reflected an accurate depiction of

the pertinent events and the reasoning behind them. I spoke with multiple employees of

nearly every organization included in the study to ensure that different accounts of the same

events could be corroborated. By further triangulating my findings with documentary analysis

and participant observation, I sought to insulate my conclusions from the biases of any

particular individual or company.

All interviews were conducted for anonymous attribution – participants are attributed by their

field of occupation but neither their names nor the names of their employers are disclosed.

Every quote in this thesis that is given with anonymous attribution is a quote from an

interview. The guarantee of anonymous attribution was made explicit at the beginning of

every interview and, in some cases, reaffirmed throughout if a participant wanted to revisit

our agreement. While in a handful of cases participants were noticeably hesitant to engage

candidly, it was far more common for participants to openly and forthrightly discuss their

experiences and perspectives, often sharing details not available in the public record. In short,

I found that rather than needing to constantly seek ways to break through the company line, it

was more often the case that participants approached the interview as an opportunity to

provide their own unique perspectives of past events, which was perfectly suited to the goals

of the study.!

United States In the US, I invited 52 participants (all via email) and 37 agreed and were interviewed. My

initial list of potential interview candidates was developed based on my own prior experience

in the field and by gathering the names of key individuals that were publicly involved in net

Page 9: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

91

neutrality discussions either in the press or at the FCC. Because gaining access to elites can

be notoriously difficult (Thomas 1995), where appropriate I leveraged my own professional

ties or enlisted the help of former colleagues to make introductions to potential interviewees,

as other scholars have suggested (Hirsch 1995; Ostrander 1995; Thomas 1995). I also

formulated the invitations so as to draw in the potential participants by suggesting how their

unique perspectives would contribute to the study.

As the interviews progressed, I used snowball sampling to generate further potential

candidates. I asked every interviewee for recommendations for other individuals to interview,

often basing my query on the interviewee’s specific connections or knowledge (by asking

about co-workers at the same company or counterparts at other companies, for example).

Interview participants generally had expertise and experience with traffic management either

from an engineering perspective or from a policy or legal perspective. Only one interviewee

had a strictly business background, but a number of others, particularly at the senior executive

level, had substantial strategy or business management responsibilities. With limited

exceptions, interviewees had at least a decade of experience in broadband or

telecommunications and many had been in their line of work for several decades. Several

current and former board-level executives and high-ranking FCC officials participated in the

study.

Interviewees’ employment experience was split fairly evenly between cable operators,

telephone companies, and the FCC. Private sector employees came from six of the largest

broadband Internet service providers in the country. The employment experience and

expertise of the participants are shown in Figure 1, with overlaps represented (for example, a

participant with employment experience both at the FCC and at a cable operator is reflected

in both categories).

Page 10: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

92

Figure 1. Employment experience and expertise of the 37 US interview participants.

Of those who were invited but did not participate, seven never responded, four declined to

participate, and four agreed but were not able to accommodate an interview given scheduling

constraints. If a response was not received within one to two weeks after the initial email was

sent, a follow-up email was sent. In two cases, recipients of the follow-up email agreed to

participate.

Interviews were conducted in person whenever possible, usually at the participant’s

workplace, and by phone or Skype (audio only) otherwise. In two cases, multiple employees

of the same organization were interviewed together at the participants’ request and despite

appeals to interview individuals separately. In one case, a preliminary phone conversation

was conducted with a participant and later followed up with an in-person interview. Thus 35

interviews were performed in total with the 37 participants.

All interviewees agreed to have the interview audio recorded except for one participant for

whom detailed handwritten notes were used rather than audio recording. The recorded

interviews totaled more than 25 hours of audio, with an average interview duration of

approximately 44 minutes.

United Kingdom In the UK, 60 participants were invited to participate and 33 accepted and were interviewed.

My recruitment strategy was the same as in the US, but because I had fewer past professional

contacts to begin with and because sources of public discourse about net neutrality that

30% 38%

35%

Cable Telco FCC

0% 10% 20% 30% 40%

Percentage of US Interviewees

Employment Experience

51% 49%

8%

Policy/Legal Engineering

Business

0% 20% 40% 60%

Percentage of US Interviewees

Expertise

Page 11: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

93

identify individuals are fewer in the UK, I relied more heavily on snowball sampling to

develop my list of candidates.

Interviewees came from policy, engineering, and business backgrounds, with many of the

business experts also having significant technical experience or knowledge. As in the US,

most participants had worked a decade or more in the telecommunications industry and the

participant pool included current and former senior level executives.

Participants were drawn from industry, government, and Ofcom. While in the US a strict

focus on the regulator (the FCC) was used in choosing participants from the regulatory sector,

there were several reasons for including government interviewees in the UK. The regulatory

regime under which Ofcom operates is in large part determined by the broader EU regulatory

framework, which underwent significant revision between 2007 and 2009. Net neutrality

became a significant topic of discussion over the course of that regulatory review and has

continued to draw attention from EU policymakers. Both Ofcom and the UK government

participate actively in the EU policymaking process, and therefore it was important to

incorporate the views of government officials as well as those from Ofcom. Separate from the

EU activity, the UK government has been involved in coordinating self-regulation related to

Internet traffic management, providing further impetus to include government participants in

the study. Because responsibility for telecommunications policy has at various times resided

with the Department for Trade and Industry, the Department for Business, Innovation &

Skills, and the Department for Culture, Media & Sport, participants from all three

departments were included.

As with US participants, more UK interviewees had private sector experience than public

sector experience, with a fairly even split between those with engineering or business

backgrounds and those with policy or legal backgrounds. Private sector participants came

from six of the nation’s largest broadband Internet service providers plus one smaller

provider. The employment experience and expertise of the UK participants are shown in

Page 12: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

94

Figure 2, again with overlaps represented in cases where participants belong to more than one

category. Since there is at present only one large cable company in the UK, cable and telco

participants are not broken out separately.

Figure 2. Employment experience and expertise of the 33 UK interview participants.

Of those who were invited but did not participate, nine never responded, eight agreed to

participate but we were unable to resolve scheduling conflicts, five referred me to colleagues

of theirs who did participate, and five declined. Five participants did not respond to the initial

email but agreed to participate after receiving a follow-up email.

One interview was conducted via Skype (audio only), and one was conducted at a conference

hotel, but otherwise all of the interviews were conducted in person at or near the participant’s

workplace. All participants agreed to have their interview audio recorded except one for

whom detailed notes were taken instead. The average interview length was 60 minutes (33%

longer than the US average) and the total interview time was 34 hours.

Interviewing Process and Structure I conducted five pilot interviews (separate from the 70 core interviews) to become

accustomed to interviewing and to trial interview questions. Four of these took place during

the research design phase and one occurred in between the US and UK interview phases.

None of the data collected from the pilot interviews was incorporated into the study’s broader

analysis.

79%

30%

Industry

Public Sector

0% 50% 100%

Percentage of UK Interviewees

Employment Experience

52%

30%

24%

Policy/Legal Engineering

Business

0% 20% 40% 60%

Percentage of UK Interviewees

Expertise

Page 13: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

95

The process of interview protocol development was evolutionary. I began with a generic

interview questionnaire structure. For each participant, I altered, added, or removed

individual questions based on research conducted prior to the interview about the

participant’s background, professional endeavors, and expertise.

One participant requested the interview questionnaire in advance and asked that we only

discuss those specific questions. In all other cases, the interview protocols served merely as

guides. In many instances the protocol questions were put aside for a time in order to explore

in depth a particular experience or topic raised by the interviewee before returning to the

protocol to ensure that all key topics had been covered, as recommended by Hirsch (1995).

This flexibility contributed to further evolution in the interview protocols, where I would

incorporate questions originally asked spontaneously in one interview into future interview

protocols. In this way, I combined inductive and deductive approaches to data collection, at

times steering participants towards topics drawn from the literature and my hypotheses, and at

other times focusing on themes that emerged from the interviews themselves.

I drew heavily on the techniques of responsive and depth interviewing (Rubin and Rubin

2004). I strived as much as possible for the interviews to be conversational and grounded in

participants’ specific past experiences. I encouraged participants to tell their own stories

about processes and events in which they had participated (Rubin and Rubin 2004). I found

that participants often raised sensitive topics on their own when I inquired about related or

tangential topics. For example, network operators’ use of deep packet inspection (DPI) was

one potentially sensitive topic given some of the negative press attention that it had garnered.

I did not ask directly about DPI, but many participants brought it up of their own accord when

I asked about their interactions with network equipment vendors (some of whom sell DPI

equipment).

All interviews began with basic questions about the participant’s job title, general work

experience, and specific work experience relevant to Internet traffic management. For

Page 14: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

96

industry participants, I would then ask at a generic level about how traffic management

decisions are made at the company or how changes in the way that traffic is managed come

about. I started with these more basic and generic questions so as to ease into the conversation

and build rapport, as recommended by Rubin and Rubin (2004).

Depending on the response to the generic traffic management question, I would probe on the

influence of specific factors on the traffic management decision-making process. The entire

selection of question topics was based on the classification of technical, economic, social, and

legal influences provided by Lessig (1999; 1998) to describe the framework of formal and

informal constraints on individual and organizational behavior. Further refinements in the

technical, economic, and social realms suggested by van Schewick (2010) provided a

framework for structuring questions. These included an individually tailored selection of

questions about network usage, architecture, and engineering; the costs of different solution

approaches and their potential to help providers reduce costs; the effects of competing

providers’ approaches and the role of competitive differentiation; interactions with network

equipment vendors; and the intersection between traffic management and initiatives to police

copyright infringement.

I also asked participants about specific regulatory events or incidents that had received

attention in the press and the impact that those had on traffic management and internal

decision-making processes. For those that had experience engaging with regulators, I would

ask a series of questions to learn about the nature of the participant’s relationship with

regulatory bodies and his or her perceptions of the function and impact of the agencies’ work

on traffic management. I placed a particular focus on formal and informal constraints that

were most relevant given each interviewee’s professional experiences and how these

constraints shaped participants’ perceptions of the regulator as adversarial or consensual.

Interviews with public sector officials covered a similar selection of topics but were focused

more on interviewees’ perspectives on traffic management in the industry at large and on

Page 15: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

97

their experiences during particular regulatory events. I asked participants to reflect about why

regulatory agencies had pursued particular courses and the nature and frequency of

interaction with the broadband industry. We also discussed how a number of institutional

factors – the authority and style of the agency chair, the strength of the agency’s jurisdiction

to act, the ability for companies to challenge agency decisions in court, and formal and

informal agency accountability mechanisms – shaped regulatory outcomes, broadly pursuing

themes inspired by a variety of theories discussed in Chapter 3 related to institutional design,

external forces, and internal characteristics.

I concluded all of the interviews by asking about any topics not previously discussed (which

often elicited substantial further commentary from the participant), requesting suggestions for

others to interview, and thanking the participants for their time. I took time as soon as

possible after each interview to document my own reactions to the substance and flow of the

conversation in my interview journal.

My goal was to reach data saturation, and in each country I felt that was accomplished with

respect to the both the dominant themes that I planned ahead of time to investigate and many

of those that emerged during the process. By the end of each round of interviews, I was often

hearing similar answers and observations in response to similar questions.

Data Handling I transcribed all interview recordings using F5, a software package. Transcription was

primarily word-for-word, although speech fillers, verbal stumbles, and interview portions that

were far outside the scope of the study were not included in the transcripts. I approached

transcription as a vital part of the analytic process, a means to gain intimate knowledge of my

data, and a mechanism for early-stage recognition of emerging themes. I included my own

mental notes in the transcripts (delimited with square brackets) at points where participants

were particularly insightful or where I felt that I could draw a conclusion relevant to my

research questions or hypotheses.

Page 16: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

98

Since all quoted portions of transcripts were anonymously attributed, I did not seek to have

interviewees expend time and effort to review their own transcripts or approve the use of

individual quotes. The full interview recordings remained available to me throughout the

process of writing the thesis and I frequently revisited the recordings to obtain complete

assurance that quotes were being reported accurately. Some interviewees used profanity, and

I chose to report their words exactly as they were spoken. The interview recordings will be

deleted upon publication of this thesis.

4.3.3 Documentation !In this study documents served as an important source of data to help reconstruct the realities

of industry and public sector postures related to traffic management and to help understand

the nature of the relationship between regulators and industry. The documentary corpus was

composed primarily of official company and government documents. These documents are of

value not because they necessarily give objective accounts of the traffic management

landscape, but rather “precisely because of the biases they reveal” (Bryman 2008, 521). The

goal of combining documentary analysis with the other methods was to provide a basis for

interrogating the messages that companies and governments use in public forums against the

information they reveal in other ways. It is the aggregated insights drawn from all the data

sources that together allow for drawing explanatory conclusions.

United States The series of FCC proceedings that were relevant to traffic management and discrimination

provided a structured document set from which to sample documents for analysis in the US

portion of this study. I chose the three most relevant proceedings as the basis for document

sampling: the Broadband Industry Practices inquiry, the Comcast proceeding, and the Open

Internet proceeding. The Open Internet proceeding provided six separate opportunities for

public comment: comment and reply comment periods for the initial Notice of Proposed

Rulemaking, the regulatory reclassification of broadband, and a further inquiry into

Page 17: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

99

specialized services and mobile broadband. I sampled comments and reply comments from

the first and last of these, but not from the reclassification inquiry as it was primarily

concerned with interpretations of statutes and legal precedents.

For each proceeding, I included all of the comments and reply comments filed by the six

broadband companies whose employees I had also interviewed. I also surveyed all of the “ex

parte” filings made by each company in each proceeding to document meetings between

company staff and FCC staff or to introduce further evidence into the proceeding record.

Since the ex parte filings are much shorter than the comments, I reviewed each ex parte filing

and included it in the corpus as long as it addressed traffic management in some way (ex parte

filings often document wide-ranging meetings that span many topics).

To cover the perspective of FCC officials, I included in the corpus all of the Commission’s

notices and orders in the three proceedings. I also reviewed the relevant portions of

Commission statements from a number of other agency activities, including the Wireline

Broadband proceeding, the Cable Modem proceeding, and all of the large cable and telco

merger proceedings from the late 1990s through the mid-2000s. Since individual

commissioners often file their own statements when the Commission begins or ends a

proceeding, I included in the corpus all of the commissioners’ individual statements from this

larger set of proceedings and the three key proceedings.

Several public hearings were held as part of the Comcast proceeding, therefore I also included

commissioners’ statements and testimony from those hearings. During the Open Internet

proceeding, the FCC convened a series of meetings of its Technical Advisory Group, which

included engineers representing a diversity of stakeholder interests. Summaries of those

meetings were made public and included in the corpus. On a number of occasions throughout

the 2000s, commissioners articulated key policy proposals in speeches (for example, Michael

Powell’s “Four Freedoms” speech in 2004), and I included those in the corpus as well.

Page 18: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

100

In addition to the broadband provider and FCC documents, I included a selection of FCC

filings from other parties, technical documentation from broadband providers and equipment

vendors, press articles about salient events related to specific traffic management or

regulatory incidents, online forum and wiki postings from broadband customers discussing

traffic management, and broadband provider terms and conditions. These documents were

primarily selected on the basis of having been referenced in an FCC filing that I had

previously reviewed or having been mentioned by an interviewee. Therefore the document

sample reflected a small amount of snowball sampling.

United Kingdom The UK corpus was comprised of documentation produced by industry, Ofcom, and

government, plus a selection of supporting documents from a variety of sources.

For each of the six large broadband providers whose employees I had also interviewed, I

collected their filings from the EU telecoms package consultations, Ofcom’s public

consultation on net neutrality, and the UK government’s consultation on the implementation

of the telecoms package. Because the volume of commentary dedicated specifically to traffic

management in these consultations was far less than in the FCC proceedings, I supplemented

this set of documentation with relevant material from the companies’ web sites, their terms of

service documents, and corporate presentations from conferences and to investors.

I also reviewed the companies’ Key Facts Indicators (KFIs) – disclosures about traffic

management made as part of an initiative launched in 2010 within the Broadband Stakeholder

Group (BSG), an industry self-regulatory body. I included the BSG’s responses to the

consultations listed above, as well as the responses of the UK Internet Service Providers’

Association (ISPA) and the European Competitive Telecommunications Association (ECTA),

two trade associations that represent subsets of UK broadband providers.

From Ofcom I included speeches and topic briefings relevant to net neutrality that were

published prior to the publication of its net neutrality discussion document, the discussion

Page 19: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

101

document itself, and the agency’s report published at the conclusion of the discussion. I also

reviewed minutes from Ofcom board meetings where net neutrality or traffic management

was discussed.

UK government documents in the corpus included the government’s submissions to EU

consultations on the telecoms package reform and on net neutrality and the government’s

approach, consultation, and response regarding its implementation of the telecoms package.

Where other documents such as briefing papers or reports related to the Digital Economy Act

touched on net neutrality, I included those as well.

As in the US, I collected a variety of further supporting documents. These included relevant

press articles, consultant and analyst reports, blog posts, and postings by broadband users in

online forums. Because of the volume of traffic generated by the BBC’s iPlayer service and

the BBC’s somewhat outspoken role in the UK net neutrality discourse, I also included a

selection of writings from both technology and policy staff at the BBC.

4.4 Data Analysis Based on my three data collection methods, my data set was comprised of interview

transcripts, my interview journal, documents, and field notes. I performed thematic analysis

(G. W. Ryan and Bernard 2003) on each of these corpuses in turn, followed by further rounds

of analysis to triangulate across the data sources and to compare between the two countries.

I used my research questions, hypotheses, and their theoretical underpinnings to derive an

initial set of themes (represented as codes) at the beginning of the analysis. The initial codes

were focused on the technical, policy, economic, and social rationales for various traffic

management approaches; the role of the market, competition, and being able to switch

broadband providers; the role of existing regulatory frameworks; and the relationship between

industry and government.

Page 20: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

102

The majority of the codes, however, developed throughout the analysis, again reflecting a

combined deductive-inductive approach. I used a hierarchical code structure and went

through a process of code expansion as more nuance emerged within particular themes. For

example, many of the documents discussed the considerations that go (or should go) into

traffic management decisions. Over the course of the analysis, this code class was expanded

to include specific sub-codes concerning technical parameters (congestion, latency,

efficiency, bottleneck locations on the network), customers’ responses to traffic management,

regulatory constraints, the capital and operational expenditures necessary to support traffic

management, and a number of other considerations. Through each round of the analysis I

revisited the code structure and made adjustments to reflect the patterns emerging based on

new data sources. I also coded for document structure and language in some instances, noting

which arguments were presented prominently, keeping track of specific examples, and

recording the use of recurring metaphors.

I began the coding process using the NVivo software package, but due to instability issues

with the software, I ultimately built my own relational database in MySQL using NavicatLite

to store all information related to documents, codes, and coded text. I allocated separate

database tables for the US and UK data, allowing me to run separate or joint queries on the

data set as necessary for my analysis.

4.5 Ethical Considerations The interviews and participant observation conducted for this thesis at times involved

discussions about sensitive corporate and regulatory information. So as to protect the

interviewees and informants from any potential harm that could result from their participation

in the study and the publication of the thesis, I took a series of precautions. I made clear to

every participant at the outset of our interaction that the thesis would be made public and that

any quoted material would be anonymously attributed. I obtained the consent of interviewees

before recording the interviews and did not record in cases where they declined. A number of

Page 21: Chapter 4. Research Methodology · Chapter 4. Research Methodology 4.1 Introduction In light of the predominantly normative and theoretical nature of existing academic scholarship

103

interviewees requested that the recording be stopped for certain portions of the interview, or

that particular words that they had spoken not be quoted, and I obliged all of those requests.

With every quote that I did include in the thesis, I thought carefully about the selection of the

quoted words and the characterization of the speaker (“cable executive” or “former FCC

official,” for example) so as to ensure that their words could not be used to inadvertently

identify them by readers with background knowledge of the events discussed. All speaker

characterizations were chosen to encompass broad groups that include a large number of

people so as to eliminate the chance that any speaker could be uniquely identified.

I also took caution when reporting company-specific information that had previously not been

publicly disclosed in the press or by the companies themselves, regardless of whether the

information was reported in the thesis as a direct quote or not. I did not attribute this

information to individual companies, but rather to the industries to which they belonged (US

cable companies or UK DSL operators, for example). The goal of the thesis was not to reveal

specific information about particular companies, but to characterize traffic management

trends among different parts of the broadband industry in each country. As such, describing

traffic management practices by industry sector strikes the appropriate balance between

protecting potentially sensitive corporate information and contributing novel findings about

industry-wide traffic management practices.