Public Opinion and the Survey Response

Political scientists know that survey responses cannot be taken at face value. For most respondents, most of the time, responses are not the product of deeply held, pre-existing, coherent views on particular policies. Rather, responses are made up on the spot, often based on whatever happens to be at the top of the respondent's head at the time[1]. Yet, when aggregated, these responses wield considerable influence over the direction of politics and policy.

Because they are first reactions, rather than records of pre-existing preferences, survey responses are often contradictory, highly sensitive to wording and framing, and unstable over time[2]. This, however, is not to say that survey responses fail to provide any information on the state of public opinion. Another thing political scientists know about survey responses is that, for a large proportion of the population, these responses can be represented in a low-dimensional space[3], although how large a proportion and how low-dimensional remains an active area of research[4].

What political scientists mean by low-dimensional is that across several issues, survey responses tend to cluster in particular patterns. For instance, in the US, knowing someone's stance on abortion helps inform us as to their stance on the environment, despite the lack of an obvious logical connection between the two issue areas. Political scientists call this kind of binding together 'constraint'[5]. Constraints emerge through a variety of sources including logical connections, but also through learning the common sense of what beliefs tend to 'go together' in a particular context.

We know that, generally, the voters for whom the low-dimensional hypothesis is most true are the most 'politically sophisticated[6] - those most capable at parsing and understanding political information[7]. While enough individuals hold coherent underlying preferences to make low-dimensional representations scientifically useful, many do not have stable views on specific policy questions. At the same time, it is important not to overstate this. A great many other respondents do not possess this degree of structure in their responses or beliefs, producing a more scattered and inconsistent set of survey responses as a consequence. 

So, although survey responses are made up on the spot, they do, to some degree, reflect an underlying disposition or partisanship. Respondents draw on their basic political orientation in order to inform their survey responses, but for many issues these are on-the-spot answers rather than pre-existing beliefs. This is directly visible in responses to fictitious issues: most respondents wisely refuse to respond, but those who do, respond in a pattern consistent with their partisan leanings[8].

Framing, Bias, and the Manipulation of Survey Responses

Because survey responses are typically on-the-spot reactions to prompts, relatively small adjustments to framing and questionnaire design can induce seemingly large differences in individual preferences[9]. Consider the difference between asking someone whether the government should act to reduce carbon emissions, and asking the same person to prioritise between taking the same action or keeping their taxes down. Most people would respond that they support reducing emissions in response to the first question, but many will prioritise keeping taxes down over taking the same action in the second question.

Two different ways of framing a given issue can therefore produce seemingly very different preferences, because respondents are reacting to two different propositions[10]. It is because of this basic fact that a seeming contradiction in public opinion can be found across different surveys. When voters express a desire for both greater defence spending and reduced taxation in different polls[11], or support reducing immigration while opposing reductions in the most common forms of immigration[12], it is in substantial part because they are simply reacting to different stimuli.

Related to the problem of framing is the phenomenon of survey response bias. Because they are reacting, rather than reflecting, survey respondents frequently satisfice their responses in many ways, which is to say they use cognitive shortcuts to select their answers[13]. Acquiescence bias, for instance, is a bias specific to Likert items (agree-disagree questions) wherein some respondents are more likely to agree than disagree with an item, irrespective of its content[14]. Agree-disagree format questions can therefore inflate apparent support for a particular policy proposal.

Another example is differential item functioning, where two individuals hold the same underlying belief, but respond to a survey scale differently[15]. This becomes particularly problematic when group-level differences in how an item is understood exist. One survey suggests that while young left-wing women, as compared to young left-wing men, prefer equality to freedom to a greater degree, they also apparently favour security over freedom to a greater degree[16].

Historically, favouring security over freedom was associated with right-wing authoritarianism, but it is plausible that these women are thinking of security in terms of social justice, and freedom in terms of the right-wing “freedom” to express deeply prejudiced beliefs without social consequence. It is difficult to know which way to interpret these results without further investigation.

Because survey responses are vulnerable to these issues of framing and questionnaire design, unscrupulous pollsters can exploit these to present results which fit their favoured narrative. One can present right-wing statements in a Likert format, safe in the knowledge that acquiescence bias will inflate apparent support for right-wing policies. Or one can frame a particular policy proposal in terms of a trade-off which makes the proposal seem less appealing – even if no such trade-off exists in reality. One can even simply ignore the nuanced problems of interpretation that arise in polling, and simply present a set of results as if they are the final word on the preferences of the public.

Presenting such misleading results serves the purpose of legitimising the survey designer's preferred policy position as if it were the preference of the public. In a democracy, this is a powerful weapon: politicians seek both for self-interested and selfless reasons to properly reflect the preferences of the public. Likewise, this kind of push polling through its effect on public discourse can exert an effect on the real preferences of the public, as the polling and surrounding discourse are consumed and members of the public update their own beliefs in response[17]. It is therefore necessary to be an intelligent consumer of polling, to be cognisant of potential sources of bias and of possible alternative framings, if one is to avoid being misled in this way. 

Why Surveys Still Matter

The survey response is therefore a methodologically problematic means of capturing the state of public opinion. Why then its prominence, in both day-to-day psephology and the academic study of public opinion? The desirability of accurate measurement of public preferences is clear. We cannot study public opinion without it, and with the exception of elections and referenda, politicians and parties do not receive any other clear signals of public preferences.

There are many methods that claim to represent public opinion. One does so, for example, when claiming an anecdote from personal experience reflects something about the beliefs of society. There is a certain vogue at present for ventriloquising the public, and particularly the working class, in this way. Vox pops, beloved by news reporters and despised by political scientists, are not much better: their results are filtered and selected by reporters, with none of the usual attention to framing effects or methodological rigour.

Similarly, our consumption of social media distorts - perhaps even more powerfully - our beliefs about public opinion. This is one of the great issues with the ownership of social media platforms by oligarchs such as Elon Musk: by altering the algorithm determining the content we see, our perception of what other members of the public believe is correspondingly affected[18].

What these methods lack is representativeness. Random sampling is the great innovation which motivates the use of the survey. It is a basic fact of statistics that as sample size increases, the typical error in a statistic such as the proportion of the public voting for a particular political party decreases – although past a certain sample size there are diminishing returns.

It is due to this property that one to two thousand randomly sampled individuals – with the necessary survey weights and other methods used to try and correct biases from the use of online sampling – are sufficient to estimate the current patterns of vote intention. Through the use of sampling, surveys therefore avoid the issues that beset anecdotal and unprincipled methods of representing public opinion. And by being methodologically rigorous in our interpretation of the responses, we can avoid many of the common mistakes and issues in less principled polling.

The Pitfalls of Poll-Driven Politics

Suppose we have collected some survey data, and we have been suitably scrupulous and methodologically rigorous in our interpretation of what it tells us about the state of public opinion. How can we use it in developing an electoral strategy? A naïve view of electoral strategy can be expressed in terms of spatial proximity. In this view, to improve its electoral performance, a party should move closer to the median voter on each given policy area. This view has a point – moving closer to voters is an effective means of appealing for their support. But it suffers from being oversimplistic. To make good use of this survey data, we require am understanding of how the preferences of voters relate to their vote choice.

Consider the issue of immigration. Right now, a majority in the UK plainly favour a reduction in immigration. It is wishful thinking to say otherwise. And it is clear that the current strategy of the Labour Party follows the naïve view of electoral strategy. In order to compete against Reform, the party has relentlessly focused on moving towards Reform voters on immigration. However, the political science literature offers no evidence that this strategy will be in any way effective, and plenty more evidence that it will be actively counterproductive.

The key reason why most political scientists would caution against this strategy is that Nigel Farage and Reform are what is known as issue owners on immigration. Issue ownership is the phenomenon where a particular party is so strongly associated with a particular stance on a particular issue, it becomes the natural choice if that issue is the key issue for a given voter[19]. And so voters for whom immigration is the most important issue will behave differently to voters for whom economics is the most important issue[20]. 

It is through introducing and making salient new dimensions of political competition that new parties break through into the mainstream, and this is precisely what has occurred for the radical right on immigration[21] and to a lesser extent on the environment for green parties. The greater the salience of the issue (i.e. the more important an issue is overall) a party owns, the greater the breakthrough[22].

When mainstream parties move right on immigration, they move closer to anti-immigration voters, but they also increase the salience, or importance, of immigration as an issue[23]. If immigration is the most salient issue and voters desire a reduction, there is little reason not to vote for the party most associated with that reduction. “Nigel Farage is right about immigration, vote for us anyway” is not a message likely to persuade Reform voters to change their vote, but is a message likely to alienate many of Labour's core supporters for no discernible gain. As Neil Kinnock recently put it, “appeasers get eaten”[24]. Authenticity is therefore as important a strategic constraint on political parties as the preferences of voters[25]. 

A deeper problem with the strategy of appeasing right-wingers on immigration is that it is one thing to express support for a reduction in immigration in principle, it is another to witness it in reality and remain supportive. Voters may express support for both tax cuts and increased spending, but it would be economic folly to pursue such a policy. Liz Truss capably demonstrated this simple principle when cutting taxes without cutting services: she acted as though economic constraints did not exist, but the Conservatives have not yet recovered from the reputational damage she wrought in doing so.

The same problem exists on the issue of immigration. One can, as Rishi Sunak did, pursue policies making the UK less attractive to international students, but doing so has caused an ongoing crisis in many universities due to the sudden drop in funding. One can, as Labour has, prevent refugees from being able to obtain citizenship, but doing so simply risks appearing cruel to Labour's own support base while failing to win over Reform voters who have no reason not to prefer Nigel Farage on immigration. Voters may ask for a reduction in immigration, but they will not thank the party that delivers it at cost to themselves, or in a way which appears fundamentally cruel. Relying simply on the fact that the polling suggests many voters support a reduction in immigration risks isolating a particular aspect of policy (net migration numbers) from fundamental constraints on policymaking.

Values, Trade-Offs, and the Role of Political Leadership 

There is therefore frequently a contradiction between the expressed preferences of voters, and the preferences revealed by their behaviour. Resolving such contradictions is the role of political parties and elected representatives. The public might wish for a reduction in immigration, but there is no evidence suggesting the public also support a funding crisis in higher education, a staffing crisis in health and social care, and even more sluggish economic growth. How to make a trade-off between these considerations depends on the values we hold.

For the Labour Party, decision-making on these trade-offs should be rooted in social democracy, just as they are rooted in conservatism for the Conservative Party, and in radical right populism for Reform. Holding true to these values in facing these problems can be in tension with some of the elements of good strategy and policymaking, but it is also crucial for the very same goals.

Successful party leaders must, at times, force their party to face up to uncomfortable constraints on policy including public opinion. Neil Kinnock and other modernisers played exactly this role for the Labour party during the 1980s. But successful governments require a vision of where they wish to lead the country, a sense of the changes they wish to make, and, at times, a willingness to try and shape public opinion by boldly arguing for their vision and policies. Without such a vision, they will rapidly find themselves becoming a reactive force, bouncing from crisis to crisis without ever succeeding in their political or electoral goals.

To properly understand when and how public opinion represents a genuine constraint on the space for government action, it must be measured. Measurement of public opinion is possible if done well and with respect to a well-evidenced theory of survey responding. Political strategy can be sensibly shaped by these measurements, but for this to occur it must be developed with an empirically grounded understanding of how the preferences and beliefs of voters relate to their vote choice.

Surveys can help a party to understand where its policies are popular and where they are not, to develop appealing framings and messaging to sell well-designed policies, and to understand which issues to emphasise and which to make secondary. Yet, measurements of public opinion should only be one consideration among many in policymaking and party strategy, and never the prime mover of politics.

Phil Swatton is writing here in an independent capacity. He holds a PhD Government from the University of Essex and works as a Data Scientist at the Alan Turing Institute.

Notes

1. Zaller, J. and Feldman, S. (1992) A Simple Theory of the Survey Response: Answering Questions Versus Revealing Preferences, American Journal of Political Science, pp.579-616. DOI: https://doi.org/10.2307/2111583
2. Feldman, S. (1989) Measuring Issue Preferences: The Problem of Response Instability, Political Analysis 1. DOI: https://doi.org/10.1093/pan/1.1.25
   Lauderdale, B.E., Hanretty, C. and Vivyan, N. (2018) Decomposing Public Opinion Variation into Ideology, Idiosyncrasy, and Instability, The Journal of Politics 80 (2). DOI: https://doi.org/10.1086/695673
3. Converse, P.E. (2002) The Nature of Belief in Mass Publics (1964), Critical Review 18 (1-3). DOI: https://doi.org/10.1080/08913810608443650 [note: this is a 2002 reissue of a 1964 article]
   Ordeschock, P.C. (1976) The Spatial Theory of Elections: A Review and a Critique, in Party Identification and Beyond. John Wiley and Sons
   Peffley, M.A. and Hurwitz, J. (1985) A Hierarchical Model of Attitude Constraint, American Journal of Political Science 29 (4). DOI: https://doi.org/10.2307/2111185
   Poole, K. (1998) Recovering a Basic Space From a Set of Issue Scales, American Journal of Political Science 42 (3). DOI: https://doi.org/10.2307/2991737
4. Broockman, D.E. and Lauderdale, B. (2025) “Moderates”, American Political Science Review (First View). DOI: https://doi.org/10.1017/S0003055424001333 [note: this is a response to Fowler et al. below, showing major flaws in their methodology and therefore findings]
   Fowler, J. et al. (2022) Moderates, American Political Science Review 117 (2). DOI: https://doi.org/10.1017/S0003055422000818
   Hare, C., Highton, B. and Jones, B. (2024) Assessing the Structure of Policy Preferences: A Hard Test of the Low-Dimensionality Hypothesis, The Journal of Politics 86 (2). DOI: https://doi.org/10.1086/726961
   Lauderdale, B.E., Hanretty, C. and Vivyan, N. (2018)
5. Converse, P.E. (2002)
6. Hare, C., Highton, B. and Jones, B. (2024)
   Lauderdale, B.E., Hanretty, C. and Vivyan, N. (2018)
   Zaller, J. (1992) The Nature and Origin of Mass Opinion. Cambridge University Press
7. Ruskin, C.L. (1987) Measuring Political Sophistication, American Journal of Political Science 31 (4). DOI: https://doi.org/10.2307/2111227
   Ruskin, C.L. (1990) Explaining Political Sophistication, Political Behaviour 12. DOI: https://doi.org/10.1007/BF00992793
8. Sturgis, P. and Smith, P. (2010) Fictitious Issues Revisited: Political Interest, Political Knowledge and the Generation of Nonattitudes, Political Studies 58 (1). DOI: https://doi.org/10.1111/j.1467-9248.2008.00773.x
9. Chong, D. and Druckman, J.N. (2007) Framing Theory, Annual Review of Political Science (10). DOI: https://doi.org/10.1146/annurev.polisci.10.072805.103054
10. Pack, M. (2025) Apparently inconsistent poll answers? Don't blame the public, in The Week in Polls. URL: https://theweekinpolls.substack.com/p/apparently-inconsistent-poll-answers [accessed 13/06/2025]
11. Pack, M. (2025)
12. Burn-Murdoch, J. (2023) Poll-driven politics does nobody any favours, in Financial Times. [accessed 13/06/2025]
13. Johns, R. (2006) One Size Doesn't Fit All: Selecting Response Scales for Attitude Items, Journal of Elections, Public Opinion, and Political Parties 15. DOI: https://doi.org/10.1080/13689880500178849
    Krosnick, J.A. and Schuman, H. (1988) Attitude Intensity, Importance, and Certainty and Susceptibility to Response Effects, Journal of Personality and Social Psychology 54 (6). DOI: https://doi.org/10.1037/0022-3514.54.6.940
    Roberts, C. et al. (2019) Research Synthesis: Satisficing in Surveys: A Systematic Review of the Literature, Public Opinion Quarterly 83 (3). DOI: https://doi.org/10.1093/poq/nfz035
14. Ray, J.J. (1979) Is The Acquiescent Response Style Problem Not So Mythical After All? Some Results From a Successful Balanced F Scale, Journal of Personality Assessment 43. DOI: https://doi.org/10.1207/s15327752jpa4306_14
    Swatton, P. (2024) Agree to Agree: Correcting Acquiescence Bias in the Case of Fully Unbalanced Scales With Application to UK Measurements of Political Beliefs, Quality and Quantity 54. DOI: https://doi.org/10.1007/s11135-024-01891-0
    Winkler, J.D., Kanouse, D.E. and Ware, J.E. (1982) Controlling for Acquiescence Response Set in Scale Development, Journal of Applied Psychology 67 (5). DOI: https://doi.org/10.1037/0021-9010.67.5.555
15. Aldrich, J.H. and McKelvey, R.D. (1977) A Method of Scaling with Applications to the 1968 and 1972 Presidential Elections,  American Political Science Review. DOI: https://doi.org/10.2307/1956957
    Hare, C. et al. (2015) Using Bayesian Aldrich-McKelvey Scaling to Study Citizens' Ideological Preferences and Perceptions, American Journal of Political Science 59 (3). DOI: https://doi.org/10.1111/ajps.12151
    King, G. et al. (2004) Enhancing the Validity and Cross-Cultural Comparability of Measurement in Survey Research, American Political Science Review 98 (1). DOI: https://doi.org/10.1017/S000305540400108X
16. Prosser, T. (2025) Do young left-wing liberal women favour security over freedom?, in The Path Less Travelled URL: https://www.thepathnottaken.net/p/do-young-left-liberal-women-favour [accessed 23/08/2025]
17. Perrin A.J. and McFarland, K. (2011) Social Theory and Public Opinion, Annual Review of Sociology 37. DOI: https://doi.org/10.1146/annurev.soc.012809.102659
18. Farrell, H. (2025) We're getting the social media crisis wrong, in Programmable Mutter. URL: https://www.programmablemutter.com/p/were-getting-the-social-media-crisis [accessed 12/06/2025]
19. Bélanger, É and Meguid, B.M. (2008) Issue Salience, Issue Ownership, and Issue-Based Vote Choice, Electoral Studies 27 (3). DOI: https://doi.org/10.1016/j.electstud.2008.01.001
20. Fournier, P. et al. (2003) Issue Importance and Performance Voting, Political Behaviour 25 (1). URL: https://www.jstor.org/stable/3657314
    Krosnick, J.A. (1988) The Role of Attitude Importance in Social Evaluation: A Study of Policy Preferences, Presidential Candidate Evaluations, and Voting Behaviour, Journal of Personality and Social Psychology 55 (2) DOI: https://doi.org/10.1037/0022-3514.55.2.196
21. Abou-Chadi, T., Cohen, D. and Wagner, M. (2022) The Centre-Right versus the Radical Right: The Role of Migration Issues and Economic Grievances, Journal of Ethnic and Migration Studies 48. DOI: https://doi.org/10.1080/1369183X.2020.1853903
22. Bélanger, É and Meguid, B.M. (2008)
    Hobolt, S. and De Vries, C. (2015) Issue Entrepreneurship and Multiparty Competition, Comparative Political Studies 48 (9). DOI: https://doi.org/10.1177/0010414015575030
    Meguid, B. (2008) Party Competition Between Unequals. DOI: https://doi.org/10.1017/CBO9780511510298
23. Abou-Chadi, T., Cohen, D. and Wagner, M. (2022)
    Krause, W., Cohen, D. and Abou-Chadi, T. (2022) Does Accomodation Work? Mainstream Parties and the Success of Radical Right Parties, Political Science Research and Methods 11 (1). DOI: https://doi.org/10.1017/psrm.2022.8
    Vasilopoulou, S. and Zur, R. (2024) Electoral Competition, the EU Issue and Far Right Success in Western Europe, Political Behaviour 46. DOI: https://doi.org/10.1007/s11109-022-09841-y
24. Rusbridger, Alan (2025) Neil Kinnock: Labour is '100 per cent wrong' on Reform, in Prospect Magazine. URL: https://www.prospectmagazine.co.uk/politics/labour-party/70081/neil-kinnock-labour-reform-starmer-farage. [accessed 13/06/2025]
25. Luebke, S.M. and Steffan, M. (2025) Who Values Political Authenticity? The Antecedents and Electoral Consequences of Voters' Preference for Authentic Politicians, Polity 57 (2). DOI: https://doi.org/10.1086/734727