Team Non-State Actors Process and Methodology Report

The main question that the team struggled to answer during the initial stages of the project was: How do we measure what the role of a non-state actor (NSAs) is?

Initial Collection: The team began by looking at individual countries to find information about specific NSAs. As a benchmark, each analyst compiled a list of approximately ten NSAs from each assigned country, including any of the following categories: businesses, terrorist groups, rebel groups, tribal organizations, trade unions, NGOs, etc. This took about two weeks. Additionally, the team created Analyst Journals to record thought processes.
  • The initial collection plan appeared to be ineffective at the time, but ultimately played an important part in the analytic process. This anecdotal approach proved useful when creating the Analysis of Competing Hypotheses in the later stages of the project. However, before this time, it did not adequately address the question of how to measure the role of NSAs; it was too fragmented and subjective to be useful.

Index Collection: The team looked for indices that rank ordered Sub-Saharan African countries. The indices found included the World Bank Doing Business Index, Transparency International Corruption Perception Index, Human Development Index, Reporters Without Borders Press Freedom Index, Freedom House Freedom in the World Index, Global Peace Index, Global Information Technology Report, Economic Freedom Index, and the Economist Intelligence Unit's Democracy Index (later on, another index was added pertaining to a country's stability: The Failed State Index). The team attempted to determine what each index meant for NSAs in terms of influence, which led to the the writing of Short Form Analytic Reports (SFARs) for each country by categorizing each index under political (including corruption), economic, and humanitarian sectors (see also Model for SFAR page for look at different indices examined and discussed, plus various Excel sheets and PowerPoints).
  • The Index Conceptual Model was used for this particular process. While this method was a step in the right direction, its efficacy was minimal. This method did not adequately address the roles of NSAs. During this time it became apparent that some indices showed differences between good (lawful) and bad (unlawful) NSAs. Team members produced several SFARs in a one week time frame, which were eventually deleted. As a result of this, the team discussed how to more effectively address the issue of good versus bad NSAs, which led to the creation of the next model.

First Method - Statistical Analysis of Indices: The team began analyzing the different indices and calculating a baseline number to determine how influential a NSA is in a particular country. (Here is the initial scale and initial country scores used). Each sector (political, economic, humanitarian) was represented on a five point scale, with a 5 indicating a greater role for NSAs, while a 1 indicated a lesser role.
  • This approach did not account for the different roles that lawful and unlawful NSAs held within each country. It was noted that while a certain index, like democracy, may account for lawful NSAs, it could also account for unlawful NSAs. The model, in its infancy, did not address this.

The team then developed the NSA Role Potential Spectrum, which compared Sub-Saharan African states to that of pure libertarian, totalitarian, and anarchic states.
  • It was with this method that the team was able to successfully answer the initial question of measuring the roles of NSAs. This model accounted for both lawful and unlawful NSAs, in terms of the socio-political environment. The reasoning behind this was that anarchic states have more potential for unlawful NSAs, while libertarian states have more potential for lawful NSAs. Moreover, totalitarian states either have a minimal role or no role potential for NSAs.
    • Libertarian States = Government Sanctioned NSAs
    • Anarchic States = Extra-Government NSAs

The NSA Role Potential Spectrum model represented a significant shift in regards to the methodology of the analysis of the target query. The assumption of the model is that within the realm of human systems there exists an equal potential for unlawful NSAs (e.g. human/drug/diamond traffickers, tribal organizations, insurgents) to utilize Extra-Government Role Potentials (e.g. violence, bribery, nepotism) as there are for lawful NSAs (e.g. corporations, NGOs, labor unions) to utilize Government Sanctioned Role Potentials (e.g. lobbyism, activism, mineral rights). If libertarianism and anarchy represent the extremes of these role potentials, totalitarianism marks the middle point. Following this line of reasoning the team selected a sample of countries most resembling totalitarian regimes. An average of any value applied to this totalitarian sample would mark the middle point of the Role Potential Spectrum.
Click to enlarge

The team narrowed down the indices to four: Democracy Index, Corruption Perception Index, Doing Business Index, and the Failed State Index. Using the rankings, coefficients were calculated for each country. In order to fit the model, the team used a sample of totalitarian states (North Korea, Cuba, Iran, Syria, Myanmar, Laos, and Libya) and calculated their mean, to anchor the middle of the spectrum. The number of rankings or scores to either side of the totalitarian sample mean were divided by five to determine the intervals of quintiles. The totalitarian sample mean was then set to 0 and each quintile in either direction was assigned a score of 1-5 respectively. State ranks were subtracted from the totalitarian mean and the result was divided by the interval of the quintile to identify its role potential score (also see figure to right).

Completion of Role Potential Spectrum: With the analysis completed, the team defined where each country landed on the model, and created definitions to fit each individual score. This data was then aggregated into average scores for all of Sub-Saharan Africa, and visualized in several different maps, which highlighted the overall score and one for each index.

Second Method - Geospatial Analysis: Independently, the team developed geospacial maps in Community Walk that outlined the number of businesses, NGOs, and terrorist groups in each Sub-Saharan African country. The team pulled every example of the target organizations (NGOs, Businesses, and Terrorist Groups) from a single source assessed to be equally representative of Sub-Saharan African countries. This process allowed the team to develop an efficient, rudimentary sampling technique free of internal analytic bias.
  • For the businesses section, the team acquired a sample of companies in Sub-Saharan Africa from Business Action for Africa. This site provided information on a wide array of companies with a presence in Sub-Saharan Africa. While the site contained descriptions for some of the companies and the countries in which they were located, the majority of them did not and the team was required to go to each company's website to determine their sphere of influence within the region. After the information was collected, it was put into an Excel Spreadsheet that highlighted the company, the country it has a presence in, and a description of the company. This information was then put into Community Walk allowing the team to look for geographic patterns in relation to the Role Potential Spectrum.

  • For the non-governmental organizations (NGOs) section, the team collected a sample of NGOs in Sub-Saharan Africa from the Duke Libraries NGO Research Guide. While there are several important topics within Sub-Saharan Africa that the team could have focused on, the team limited their collection to five specific types of NGOs including human rights, women's rights, development, environmental, and HIV/AIDS. The team believed these were the most influential types of NGOs within the region. After the information was collected, it was put into an Excel Spreadsheet that highlighted the NGO, the country it has a presence in, and the type of NGO. This information was then put into Community Walk allowing the team to look for geographic trends in relation to the Role Potential Spectrum. This process allowed the team to develop an efficient, rudimentary sampling technique free of internal analytic bias.

  • For the terrorist organizations section, the team collected information primarily from the MIPT Terrorism Knowledge Base. The team looked at each individual country and determined the number of active terrorist groups within each. After the information was collected, it was put into an Excel Spreadsheet that highlighted the group, the country it was actively involved in, and a brief description of its goals. This information was then put into Community Walk to look for geographic patterns in relation to the Role Potential Spectrum.

Third Method - Analysis of Competing Hypotheses (ACH): Initially the team created six hypotheses. The first three hypotheses examined whether the Role Potential score was correct:
  • The current score for NSAs is accurate.
  • The NSAs score should be lower.
  • The NSAs score should be higher.
The additional three hypotheses discussed the future Role Potential score:
  • The score will remain static over the next five years.
  • The score will increase over the next five years.
  • The score will decrease over the next five years.

After some discussion, the first three hypotheses were eliminated due to redundancy. The team then collected between eight and twelve pieces of evidence. The evidence would concern the four elements of the Role Potential Spectrum model which define the role potentials of NSAs: democracy, doing business, corruption, and stability. The team then carefully weighed each piece of evidence, determined the reliability of each source, and assessed whether each piece of evidence was consistent, inconsistent, or neutral to each hypothesis.

Final stages: The team wrote a SFAR for each country in Sub-Saharan Africa using the scores determined by the Role Potential Spectrum model and the assessments from the ACH. Within this process, the team aggregated the forecast data within each country and calculated the future role potential of NSAs in Sub-Saharan Africa for the year 2012. The team then visualized the data into one map for the future scores of all Sub-Saharan Africa.

Conclusion: The types of analyses done were only accomplished due to the enormous time commitments and tolerance to set backs and analytical failures of the team. At the beginning, the problem could have been approached several different ways, either by looking at a smaller sample of countries, or doing a purely anecdotal approach. To the team, an anecdotal approach did not adequately address the key question of how to measure the roles of non-state actors in Sub-Saharan Africa. The fundamental stepping stone which made the analysis of the question possible was the development of a model which was able to account for both Government Sanctioned Role Potentials and Extra-Government Role Potentials, and thereby represent the likely role potentials of both lawful and unlawful NSAs. Therefore, the team endeavored to develop a better way of answering the question, thus leading to the current Role Potential Spectrum model. Once this model was conceptualized and put into practice, all the previous collection efforts came to fruition in the forms of the statistical analysis, the geospatial analysis, and finally the Analysis of Competing Hypotheses.

Advantages and Disadvantages of Each Method

Role Potential Spectrum Analysis
  • Advantages
    • Accurately represented NSA situation in Sub-Saharan Africa.
    • Positively correlated with the geospatial analysis.
    • Created a prediction model for the roles of NSA in a country.
    • Able to represent both government sanctioned and extra government role potentials for NSAs.
    • Created a standardized foundation to measure environmental influences on the roles of NSAs to produce comparable across Sub-Saharan Africa states.
    • Guided further analysis (geospatial, ACH) to efficiently identify and assess significant indicative characteristics of the socio-political environments in Sub-Saharan Africa.
    • The state centric environmental approach allowed the analysts to effectively consider key factors across the entire socio-political environment of individual Sub-Saharan African countries.
  • Disadvantages
    • The major indices in unaltered states are unsuited for measuring the role of NSAs.
    • The matrix required several weeks to finish, and took time away from starting and completing other types of analysis.
    • Would be difficult for other analysts without statistical background to successfully complete.

Geospatial Analysis
  • Advantages
    • Independent from other analyses.
    • Information retrieved from uniform databases; not truly random, but relatively objective sample.
    • Simple and easy to do with the information being imported from Excel Spreadsheets directly into Community Walk.
    • Created a visual representation allowing the team to identify patterns and correlations among different sets of data, particularly with the statistical matrix.
  • Disadvantages
    • Much of the available information on NGOs, terrorist groups, and businesses was not in English, which made it difficult to collect a larger sampling of the information.
    • The team limited the NGO map to development, women's rights, HIV/AIDS, human rights, and environmental issues. This was not an exhaustive list of all the different NGOs operating within the region.
Analysis of Competing Hypotheses
  • Advantages
    • Able to predict an increase or decrease in the roles of NSAs.
    • Uniform sources throughout (Janes, Factbook, State Dept.)
    • Easy for team to complete (received in-class instruction on this type of analysis).
  • Disadvantages