Foreign language keywords are powerful research tools for international affairs. But without a quick process to remove bias from the words we use to filter online information, experts and generalists can fail to see how our inherent bias clogs the search filter.
Without de-biased keywords, we miss the most relevant solution or incriminating evidence. It’s especially true when using artificial intelligence for research. In a previous post, we illustrated the keyword development process with Arabic and Mandarin examples. If you missed it, join the International Affairs Thinker LLC email list to get the latest and the behind the scenes of this blog.
This post dives deeper to explain the thought process for each stage of the keyword de-biasing. This detailed post is intended to help you readily apply de-biasing to your international affairs research. Reach out if you want a tailored demo.
What does “de-bias” mean?
The word “de-bias,” sometimes spelt “debias,” is a verb meaning to remove human or computer bias. The adjective form “unbiased” is found in most dictionaries.
Many studies that test de-biasing strategies for specific human biases typically recommend inversion techniques. For example, to counter the action bias—the tendency to favor action over inaction—one can increase conditions that favor the former, such as awareness of uncertainty. In other words, look for what information would improve the likelihood of an intended outcome, and then show that confirming that information, in advance, is impossible. Heightened awareness of uncertainty would lead to more thoughtful actions.
Scientific studies typically look at specific biases to enable repeatability of experiments. In the case of international affairs research, repeatability isn’t always possible on the internet, nor is it always desired. When de-biasing keywords, experienced practitioners know the first words that come to mind are probably biased, but we’re unsure of the type of bias. The type matters less than having a tailored assumption checking process.
What is assumption checking?
One way to think about assumption checking is to define it as a Structured Analytic Technique (SAT). In Structured Analytic Techniques For Intelligence Analysis by Pherson and Heuer, assumption checking is described as:
“Analytic judgement is always based on a combination of evidence and assumptions, or preconceptions, which influences how the evidence is interpreted. The Key Assumptions Check is a systematic effort to make explicit and question the assumptions that guide an analyst’s interpretation of evidence and reasoning about a problem. Such assumptions are usually necessary and unavoidable as a means to fill gaps in the incomplete, ambiguous, and sometimes deceptive information with which the analyst must work. They are driven by the analyst’s education, training, and experience, plus the organizational context in which the analyst works.”
IAThinker added the bold to emphasize that this SAT attempts to be systematic in identifying the analysts’ assumptions when filling gaps between the evidence and their experience. This technique is used when analysts already have the evidence and are establishing their final judgement.
De-biasing keywords must occur at the outset of evidence gathering. Assumption-checking for keywords is more iterative than a single, late-stage systematized check.
The difference between types of assumption checking
Tailored assumption checking differs from the traditional SAT when applied to developing foreign language keywords in two important ways.
- Timing. Keyword development is the project’s first stage. The process must include de-biasing individual words—harder than spotting biased paragraphs, but essential because keywords determine what evidence you find.
- Process. Keyword assumption checking is iterative, not a single late-stage step. It does not occur in an analytic tradecraft gathering. Instead, practitioners test keywords on the internet, observe the reflection of their biases in results, and refine accordingly.
Questions to ask yourself as you iterate keywords
The previous post illustrated the four steps to develop keywords for international affairs research, even without having deep expertise in the language. IAThinker briefly described the relevant de-biasing best practices for each step. The goal was to develop keywords to find Arabic and Mandarin language examples of cognitive biases and heuristics. IAThinker wanted to use foreign language keywords to increase the chances of finding culturally relevant examples.
Below, IAThinker describes the best practices in more detail, with specific questions to ask yourself as a guide to iteratively de-bias single words.
Step 1. Translate the topic and related words with different translation tools. Remember the topic is only one of many keywords you should use. Organize original and translated words into a document.
- How many translation tools should I use? Translate the topic with at least two different tools. The goal is to identify and collect any variations across translations, especially if you’re not a native speaker. If there are multiple ways to refer to your topic, then these translations are the lowest hanging fruit you can’t overlook.
- How many synonyms or related words should I translate? If there are other words or phrases that jump out to you as describing your topic, translate them. You’ll find that some translations differ depending on grammatical category. Collect them all at this stage.
Step 2. Confirm the accuracy of these translations by plugging them into at least two search engines. Use at least one search engine that prioritizes privacy and won’t limit your results to previous searches. Translate results with browser translator for a quick check of multiple result pages.
- Does usage change when grammatical category shifts? This was true for translations of the word “heuristic,” which led IAThinker to use related words rather than attempting direct translations of “heuristic” in the previous post.
- Is the translated word used in the local language as you would use it? For example, in the previous post, IAThinker wanted to find local language examples of “cognitive bias.” The bulk of the initial search results were direct quotes of Daniel Kahneman and Amos Tversky’s book Thinking Fast and Slow on websites in the behavioral finance and economics field.
- If yes, do you want a direct translation from a non-local reference?
- In other words, do you want to know if they’re familiar with the topic, or if they use it within their own context? IAThinker wanted the later. Make note of these choices in your document of keywords.
- Did you do this initial check for every word (topic and related words) you translated?
Step 3. Iterate by using search operators or additional contextual words to eliminate non-local examples (or whatever issue that arose in Step 2). Iterate multiple times—check the keywords again by searching with at least two search engines and translating multiple result pages—to find specific results. Compare final results to your first set from Step 2 or get feedback.
- Which search operators will help me find better results?
- Which contextual keywords will help me find better results? In this case, IAThinker tried a few translated contextual words like “history” and “international affairs” with the original topic “cognitive bias.”
- How can you increase the reputability of these examples? If you’re not an expert in the language, you may not know many of the websites in the search results. In this case, use the foreign language keywords you’ve narrowed down in a well-known database, like a news platform. Another way to increase reputability is with the search operators like “filetype:pdf” or “site:[government top level domain].” IAThinker checked the top level domains for each of the 22 countries in the Arab League, plus mainland China.
- Are you cherry-picking? Compare your final results to each prior iteration to see if your final keywords are representative of reality. De-biasing keywords should not perpetuate unquestioned assumptions. In the “cognitive bias” example, IAThinker initially eliminated examples that translated as “prejudice” because these examples were about interpersonal situations, rather than interpreting historical or regional issues. If the vast majority of examples were about “prejudice” and only one example was about historical decision-making, then IAThinker would have been cherry-picking. This was not the case.
- If yes, or you want to double-check, ask: How can you narrow the topic while using a frequently used translation? In the case of “cognitive bias,” IAThinker selected specific biases to translate and check rather than relying on the general term. After again eliminating direct translations from non-local sources, there were still several results pages. This method showed that some biases are more often discussed in local sources than in English sources.
Step 4. Copy, paste, and save several full text examples. Use any translation tool. Organize the document so that each paragraph alternates between original and translated text. This format makes it easier to quickly check weird translations of specific words or phrases. Read the examples a different day. Additional ideas on how to further de-bias the keywords may come in this break or after reading the results.
Reflections on de-biasing the word “cognitive bias”
The irony of de-biasing the word “cognitive bias,” may have been obvious to the reader, but it wasn’t to IAThinker. Because neuroscience has shown how human interpretations of experience wire our brains, and that de-biasing is possible, IAThinker’s assumption was that the foreign keyword “cognitive bias” wasn’t inherently biased. It would be like searching for the noun “memory.” IAThinker assumed that “cognitive bias” was an accepted conceptual framework and the cultural context would arise from the specific examples.
- First assumption checked. The need for de-biasing quickly arose when noticing the different usages of the word “heuristics” as a noun and adjective. In the adjective form, “a heuristic approach” was a very common phrase in examples about education. In the verb form, several examples translated like “induction” and “inference,” which raised questions like the debate between Kahneman and Gigerenzer, so IAThinker avoided the rabbit hole for the time being.
In the case of developing Arabic and Mandarin language keywords, many local language examples of the word “cognitive bias” and specific heuristics like “recency effect” were described as inherently western constructs. In other Arabic and Mandarin examples, while the bias was described like it is in English, the solution was described as adopting “self-criticism,” which has a different cross-cultural meaning in English.
- Second assumption checked. When researching with the keyword “recency effect” many local examples were described and co-occurred with the “modernity bias,” which isn’t commonly included in western cognitive bias collections. IAThinker had only heard of it from Nassim Nicholas Taleb’s Antifragile. He describes modernity as, “the systematic extraction of humans from their randomness-laden ecology…. It is rather the spirit of an age marked by rationalization (naive rationalism), the idea that society is understandable, hence must be designed, by humans…” Taleb’s Antifragile does a good job of using western analytical style to argue for some traditional (non-modern) mental frameworks most common in non-western and historical contexts. He is, after all, originally from Lebanon, a founding country of the Arab League.
Shutdown
This post provides a deep dive into the foreign language keyword de-biasing process. Keyword development is the first step in any international affairs research project, and is the first place for a wrong turn. The four step process to removing the bias during keyword development together with the above questions for each step can easily be applied to any language and any topic.
Practitioners thrive in the international affairs knowledge industry by adopting best practices in foreign language research that include de-biasing. The three methods International Affairs Thinker LLC has distilled and provided for free start with de-biased keywords. Use the above keyword development process with the Crossthink Process, Authoritative Source Checklist, and Technical Expertise Builder to provide relevant, timely, convincing, and actionable insights to your government or corporate leadership.
Stop missing blogs with best practices tailored for your job by joining the International Affairs Thinker LLC email list. Reach out at any time for a demo for your team.
