# Load packages and set API key
library(gemini.R)
google_key <- "XXXXXXXX"
setAPI(google_key)
# Define prompts for model
prompts <- c("What is the meaning of life",
"What is the meaning of life? (Wrong answers only)",
"Summarise the plot of Harry Potter in a paragraph.",
"Explain the concept of Elon Musk.")
# Set temperature increments
temperatures <- seq(.1, 2, .1)
# Set up data.frame to store results
i <- 0
gemini_out <- data.frame(temperature = rep(NA, length(temperatures) * length(prompts)),
prompt = rep(NA, length(temperatures) * length(prompts)),
text = rep(NA, length(temperatures) * length(prompts)))
# Loop over prompts
for(p in prompts){
print(p)
# Loop over temperatures
for(temp in temperatures){
# Include a pause to avoid transgressing rate limits
Sys.sleep(2.2)
i <- i+1
# Make call to gemini API and store results
gemini_out$text[i] <- gemini(prompt = p, temperature = temp)
gemini_out$prompt[i] <- p
gemini_out$temperature[i] <- temp
}
}
# Save
save(gemini_out, file = "gemini_out.Rdata")10: Generative Language Models in Social Science Research
Causal Language Modelling Applications
Causal language models will be very familiar to you!
Training Transformer LLMs
- Words/tokens are represented as embeddings which capture semantic similarities.
- Embeddings are passed through multiple hidden layers, which apply non-linear transformations to capture abstract features of the input.
- At each layer, the model uses self-attention to compute how much each word should “pay attention” to every other word in the sequence.
- The final transformer layer produces a probability distribution over the vocabulary \(V\).
- The predicted probability for the correct word is used to calculate the cross-entropy loss.
- The total loss is computed as the average cross-entropy loss across all tokens in the sequence.
- Using backpropagation and gradient descent, the model updates its weights to reduce the loss.
Decoding methods: Example
We can quantify the effects of changing the temperature parameter by calculating the entropy of each of the LLM-generated texts.
\[H = -\sum_{w \in V} P(w) \log P(w) \]
where \(P(w)\) is the probability of word \(w\) in the generated text.
- A low entropy score means the text is highly repetitive and deterministic.
- A high entropy score suggests more diverse word choices.
Pre-training and Post-training
Pre-training: The initial phase of training a language model, where it learns by predicting the next word in huge volumes of unlabelled text
- Pre-training allows the model to learn a broad knowledge of grammar, facts, and semantics
Post-training: Additional steps taken after the pre-training phase to adapt the model to specific tasks, such as following instructions or being helpful
- Post-training allows the model to learn the types of responses that humans would like it to produce
- Post-training helps turn predictive power into useful, aligned, and safe behaviour.
Post-training is a core part of the recent success of LLMs, particularly for chatbots.
The Post-Training Pipeline
After pre-training, LLMs typically go through several stages of post-training:
- Supervised Fine-Tuning
- The model is trained on prompt-response pairs created by humans
- Teaches the model how to follow instructions
- Reinforcement Learning from Human Feedback (RLHF)
- The model generates multiple responses
- Humans rank the responses
- A reward model is trained on these rankings
- The model is then fine-tuned using reinforcement learning to maximise the reward
Example 1: LLMs as Synthetic Human Samples
Research question: Can LLMs be used to simulate human samples in social science research?
LLMs are known to exhibit algorithmic biases – the tendency for models to replicate the racial, gender, economic, and other biases of the texts on which they are trained.
Can we use this property to generate text that resembles text produced by people with different characteristics?
Algorithmic fidelity is the property where, given basic human demographic background information, a model exhibits underlying patterns between concepts, ideas, and attitudes that mirror those recorded from humans with matching backgrounds.
Task One
Task: Ask GPT-3 to list four words about Democrats and Republicans, while pretending to be a Democrat or a Republican
- Ask humans to evaluate these lists in terms of their content, their sentiment, their extremity, and whether they can predict the partisanship of the lists
Result: Very high-level of consistency in the evaluations of both human and GPT-3-generated lists in both content and tone.
Task Two
Task: Ask GPT-3 to generate probabilities of voting for a particular candidate, given a specific backstory.
- E.g. calculate \(p(\text{trump}|\text{backstory})\) and \(p(\text{clinton}|\text{backstory})\)
Result: Very high correlation between voting behaviour reported by human respondents and that reported by GPT for those respondents.
How might we use these properties?
Researchers can leverage the insights gained from simulated, silicon samples to pilot different question wording, triage different types of measures, identify key relationships to evaluate more closely, and come up with analysis plans prior to collecting any data with human participants.
Though note that the performance of GPT to mimic human samples is limited:
- Less variation in responses than in real surveys
- Relationships between generated variables is often less accurate
- Distributions of synthetic responses are very sensitive to prompt wording
- Distributions of synthetic responses are very sensitive to exact model (e.g. GPT-3 vs. GPT-4)
Example 2: Generating Adaptive Treatment Texts (I)
Research question: Can large language models be used to increase the persuasive effects of political microtargetting?
Microtargetted messages are those that try to direct messages in an individualised, personalised way to respondents with specific characteristics
The assumption behind microtargetting is that some people are more persuaded by some types of argument than others, so politcal actors can be more successful in persuasion by targetting their communications
Can we use LLM-generated arguments to generate effective personalised arguments?
Approach:
Gather data on demographic information about respondents
Prompt GPT-4 in real time to write persuasive texts on 4 political issues, of two types:
- Microtargetted messages, tailored to respondent characteristics
- Non-microtargetted messages
Measure support for a policy issue relative to a control group who see no message at all
Result:
GPT-generated messages are persuasive, on average
- Support increases by up to 12 percentage points, depending on the issue
Micro-targetted messages are no more persuasive than non-targetted messages
- A finding consistent with existing literature on politial persuasion
Micro-targetting prompt
You are a political persuasion expert specilizing in micro targeting techniques…Person X has the following attributes: [list attributes]. Write an argument of around 200 words that would persuade person X to agree with the following issue stance: [issue stance].
Non-micro-targetting prompt
You are a political persuasion expert… Write an argument of around 200 words that would persuade person X to agree with the following issue stance: [issue stance].
Example 3: Generating Adaptive Treatment Texts (II)
Research question: How do core elements of political discourse affect dialogue quality?
Online political discourse is often plagued by partisan animus, hostility, incivility, intolerance, and low quality exchange
Can we foster more productive dialogue by varying the ways in which people are interacted with in debate/deliberation?
Approach
Ask survey respondents to write a social media post about a political issue with their position and the reason for their position.
Get an LLM to generate counter-arguments on-the-fly, with reasoning tailored to the individual’s position and reasoning
Randomly vary three components of deliberation the LLM response:
- Disrespectful tone
- Partisanship
- Evidence-based argument
- Compromise
Ask survey respondents to reply to the generated message and measure the quality of their replies
Results
Arguments that are more evidence-based and signal a willingness to compromise receive higher quality responses
Arguments that are more disrespectful receive lower quality responses
Argument style also affects perceptions of the interlocutor (which, here, is an LLM!)
In political argument, you “get what you give”.
Social science applications