Originally written for Qualitative Research Methods at University of Leeds, taught by Dr. Kahryn Hughes
Introduction
Sampling in qualitative methods is indescribably important, particularly due to the often messy nature of the data researchers are working with. Be it hours of recorded speech, pages upon pages of media documents, or stacks of photos from creative interviews, qualitative methods are known for producing large sums of interesting data. In this essay, I will look at questions of sampling I faced in the context of my masters dissertation, which uses discourse analysis to look at portrayals of authenticity in celebrity magazine profiles. This essay will follow with a brief description of the research and a review of literature surrounding sampling, before analysing my sampling strategy to explore the impacts a sample has on analysis. I will do this by looking at epistemology surrounding sampling in discourse analysis and my experience with the data, difficulties in regards to working with the sample, and decisions made in order to manage the sample size.
My dissertation, which looks at constructions of authenticity in celebrity magazine profiles, began with a sample of 23 celebrity profiles, from Rolling Stone and Vanity Fair magazines. Through discourse analysis, the research seeks to explore the co-construction of the “Star” by the journalist and the celebrity within the profile as well as both of their efforts to portray authenticity, pinpoint the interactions between authenticity and intersections of identity, and inquire into how celebrities and journalists manage the expectations and interests of the public. I chose discourse analysis due to its flexibility and the varied nature of the data (Wood and Kroger, 2000). Through preliminary analysis and purposive sampling, the sample was narrowed down to eight main articles of analysis (Wood and Kroger, 2000; Gill, 2018; Emmel, 2013). These were then looked at for and organised into overarching themes, rhetorical strategies, purposeful use of quotes, place and description, as well as the context in which the celebrity and journalist interacted. Preliminary conclusions show that celebrity profiles consistently rely on the duality between Stardom and Ordinariness to carry the narrative structure of the article, that there are “Required Topics” which the journalist is compelled to discuss, and that those required topics are frequently centered around intersecting sites of identity.
This research sits among a breadth of literature into stardom, ordinariness, authenticity, and media. Celebrity research is key to understanding both the place of stardom in the cultural sphere and the pressure the public puts on famous individuals. Not only do stars represent an incredible monetary value to society (Dyer, 1998, pp 17), they can also be understood as representations of the beauty standard, as exemplars of the myth of meritocracy, and as visible stand-ins for the extremely wealthy (Sternheimer, 2015) or as carrying the burden of representation (Tagg, 1988) for any number of intersecting identities. These ideas come to a head within the profile genre, which can be understood as a form of documentary (Corner, 2002) both of which claim to be a true representation of its subject, giving “insider” details on themes that the general population will understand and relate to (Ferris, 2001; Sacks, 1985), while still showing its subject as worthy of particular interest (Purcell and Moore, 2019). So too, this research fits into a critical history of the very concept of “authenticity”. Neoliberalism, in which individuals should take responsibility for themselves (Johnson, 2014) has created a view that not only a “true self” exists, but can be worked towards and achieved. However, many academics counter this assumption, suggesting instead that the self is socially constructed (Goffman, 1956), constantly changing (Eckert, 2003), and always performed (MacCannell, 1999). The questions of authenticity which this research asks have impacts on all types of qualitative research, particularly in the context of interviews and ethnography. The rest of this essay will explore the importance sampling has on my research.
Sampling Literature Review
Whilst literature on the methods and theories of qualitative analysis is detailed and never-ending, a large majority of literature about sampling specifically can be summarised in a few words: How big does my sample need to be? (It depends). Scholars have figured out how to sample and why to do it in a specific way, or explored specifics as to how sampling works differently in different forms of research; but by and large, there is no solid answer for how large a qualitative sample needs to be—something that has endlessly frustrated students.
On the incredibly vague end of the spectrum lies advice like that which Becker (2012) provided: “The only possible answer is to have enough interviews to say what you think is true and not to say things you don’t have that number for” (Becker, 2012, pp 15). As poetic as this advice is, it is not particularly helpful in the outset of a project, which is an issue that Emmel (2013) raises:
“There is no reasonable methodological way in which realist qualitative researchers can tell you the eventual number of cases in advance. If they provide a number it is likely that they are complying with external liabilities and powers from institutions, ethics review boards, funding bodies, and/or editors of journals.” (Emmel, 2013,
pp 154).
As such, a contradictory issue arises, one I struggled with in my own research. Students and funded researchers alike are frequently told they must pick and explain their sample size at the outset of their project, when frequently they will not know what their sample size needs to be until they have commenced their research. When pushed for a number, scholars often agree that samples should be (or at least, frequently are) between 12-30 cases, with students’ projects tending to be on the lower end of the range and funded projects sometimes being much much larger (Adler and Adler, 2012; Emmel, 2013). This is where I started my research, with a sample of 23 celebrity profiles that I believed I could analyse in entirety for an 8,000 word dissertation. As Becker (2012) and Emmel (2013) suggest, however, I did not really know what my sample size needed to be until I began my research and explored my methods more thoroughly.
I knew from the onset that I wanted to complete my research through the method of discourse analysis—research I was impressed with had used this method and I wanted to emulate it. However, I did not really understand what that entailed, particularly when it came to my sample size. When my supervisor first told me I should find a way to cut my sample size in half, I was confused and resistant to the advice, wanting to be thorough and impressive in my research. This idea that bigger is better is an idea rooted in quantitative research where larger sample sizes provide more representative, valid, and important findings (Emmel, 2013). After further reading and preliminary analysis, however, I understood. When looking at the specifics of discourse (language) even a so-called small sample can be “quite large in terms of language instances” (Wood and Kroger, 2000, pp 77).
A magazine profile for the cover star likely has many thousands of words, from a variety of speakers as well as the voice of the journalist themselves. This is a huge amount of information to work with, to the extent that a single article could be the subject of a single piece of research, depending on what the question is. For my research questions, I needed to be able to compare multiple profiles.
Sampling for Authenticity within Celebrity Magazine Profiles
There are a few main things I took into consideration when selecting the magazine articles I would conduct my research on. These were: location and timeframe (which went hand in hand), magazine brand, number of articles, and thematic focus. Each of these sections included a specific answer and decision I had to make. Before I explore and explain those decisions however, it is important to look at the epistemology and justification of purposive sampling such as mine.
Epistemology and Experience
Whilst I have already used literature to show that in discourse analysis the problem is usually that a sample is too large rather than too small (Wood and Kroger, 2000), which was the case in the start of this research, it is important to explain further as to why that is.
Discourse analysts understand language to be occasioned and contextualized (Wood and Kroger, 2000, pp 76), that is, to explore language and the impacts therein in the specific time and place at which it occurred. This epistemology leads to an analysis that cannot be generalized in the way quantitative work aims to, because that is the antithesis of its purpose. Discourse analysis of a specific work can be applied to other instances of discourse or research, but it is not done to create generalizations across an entire population or category, and so population-sized samples nor random-sampling are necessary (Wood and Kroger, 2000).
The size and scope of a piece of research can vary wildly depending on the research questions, particularly when working with already available and published media/writing rather than researcher-created discourse (such as transcripts of interviews).
“For example, Coupland and Coupland (1997) examined all national newspapers and the 23 most popular magazines in the United Kingdom over a period of 2 weeks in an analysis of environmental discourse, whereas the Wood and Rennie (1994) analysis of eight interviews was relatively circumscribed” (Wood and Kroger, 2000, pp 78).
There are an infinite number of celebrity profiles available online, and any combination of them could have produced entirely different analysis, or could have produced the same few themes I have come across. That is part of what I explore in my actual analysis, but it is also something I needed to be aware of whilst sampling. This awareness was aided by my prior knowledge of the magazine industry. I did my undergraduate degree in Magazine Publishing, at a renowned journalism school in the US where I had the opportunity to be taught by experienced journalists and editors who had worked with magazines from Vanity Fair to Buzzfeed to Vogue. On multiple occasions I was tasked with writing or editing profiles, which were then graded by these experienced professors. This education served as primer to my current research, allowing me a sort-of jump-start in terms of my sampling. I knew the basics of what I was looking for or working with before I even started sampling. This prior knowledge can greatly impact the research and aid in sampling or preliminary analysis, but I needed to make sure I was not resting on my laurels and missing details that someone unfamiliar with the genre might see (Wood and Kroger, 2000).
Magazine Location/Year
When thinking of portrayals of celebrity authenticity, I wanted to ensure an interesting and representative sample of profiles. As such, American magazines became the focus of my research early on. Most of the large multinational brands were either founded or headquartered in the United States (Hearst, 2021; Condé Nast, 2021), and many of the non multinational brands are still some of the highest grossing magazines in the world, so the US brands can be read as the industry standard in terms of writing style, interview methods, formatting, and audience expectations. It can also be argued that the United States is the main hub of celebrity culture internationally—both celebrities and magazines are usually based in Los Angeles or New York (Evans, 2017).
For this particular piece of research, once American magazines were decided on, it was important to choose a “normal” timeframe. Whilst an analysis of celebrity profiles during the Covid-19 pandemic, during an election year, or during other times of specific social change would be very interesting, those overwhelming topics would only cloud the analysis of authenticity. I also wanted it to be a fairly recent year, since the magazine industry has been changing rapidly, with major brands going through layoffs, changing formats, or shutting their doors completely (Ember and Grynbaum, 2017). Because of this, articles published in 2019 were chosen as the starting time frame. There were still, of course, important societal events going on during this year in the United States, but there was no one event that served as an undercurrent to every media interaction.
Magazine Type
Moving on from time and place, I needed to choose the specific magazine titles. I knew I wanted to sample from two magazines, to avoid the pitfall of having my analysis only be relevant to one specific magazine or writer’s style. So too, it was important that the magazines had a wide circulation and high level of prestige. Not only would this again ensure it was a representation of the celebrity profile, it would also serve as an integrated sampling of the type of celebrity under analysis. Only a certain “level” of celebrity will have enough draw and appeal to make the cover of and take up 10 pages of a large magazine. Whilst another sampling option would have been to seek out specific celebrities across various magazines, there are very few agreed upon ways of categorising celebrities, most of which are entirely subjective, so it would be far more complicated to only read articles about “A-list” celebrities, for example.
Which Magazines
From there, there were a few options. The two main magazine publishers in the US are Condé Nast and Hearst Publications, both of which have many titles, genres, and topics in circulation. It was important to choose one magazine from either of those companies, however it would be overlooking a great many different options if I only picked from those two—I needed to pick one title from one of those brands, and another from outside of those brands. Finally, I needed to decide what genre I wanted to focus on. Genres which have celebrities on their covers include lifestyle, fashion, sports, men’s, and general “popular culture”, among many others. Choosing, for example, lifestyle or fashion magazines would mean I was likely getting a very gendered sample, given that those genres are widely considered “women’s” magazines and thus have female celebrities on the cover, similarly, men’s lifestyle magazines tend to feature only men on the covers. I decided to avoid this obvious gendering of the sample, because although I wanted to look at how gender interacts with authenticity, I did not want it to be from a specifically gendered focus or audience. This all left me with wide-circulation popular culture magazines. This led me to decide on Vanity Fair (from Hearst) and Rolling Stone (Vox Media) magazines. Both contain “popular culture” in their descriptions of themselves, have a broad array of film and music stars, of various genders, sexualities, and races, and serve a broad audience which is not overwhelmingly gendered.
Sample Management and Impacts on Analysis
These two magazines published 23 issues in 2019, with each having one main “cover” profile per issue. Whilst a sample of around 30 is typical for social science research (Emmel, 2013), 23 profiles was too many for the scope and size of this dissertation.
However, after preliminary analysis I was able to quickly narrow down my sample size. Just as Wood and Kroger (2000) suggest, when familiar with the specific type of discourse, the researcher “will have some sense of the structures and functions that are likely to be critical and of their relative occurrence in a particular number of texts” (Wood and Kroger, 2000, pp 80). My first reading limited my sample to that of the “typical profile”. Some were removed due to differences in format (such as Q&A format rather than the typical profile structure), others because they featured two or more celebrities, were shorter than the average profile, or were from specialty editions. This brought my sample down to 13 “normal” profiles. And whilst that was a more manageable sample for the scope of this dissertation, through preliminary thematic analysis (Gill, 2018) I found that saturation was reached and particularly well-demonstrated in eight of the 13—five from Vanity Fair and three from Rolling Stone. This thematic analysis was conducted by reading the 13 articles repeatedly, marking down what the main themes of the articles were, and selecting the few themes that were most consistent and analytically interesting. This is similar to the method Gill (2018) used to narrow down three years worth of editions of Glamour magazine, going from around 150,000 pages in total, to choosing two issues at random from each year, and then identifying a recurring set of article types or genres (Gill, 2018, pp 16).
My sampling process began with a near infinite number of analytical possibilities and ended with a sample size of eight magazine articles, which highlighted key debates in celebrity, authenticity, and media research. Though this can feel like a tiny, irrelevant, or unimportant selection of magazines (at least according to my own internalized judgement), it resulted in the analysis of three key features of the celebrity profile as it relates to authenticity. These were: the Co-Construction of the Star, Required Topics (which included the Burden of Representation, the Star’s Work, and Justifying Nepotism), and the Star/Normal Binary.
These themes, while likely to be found in most magazine profiles, would not be able to be generalized or properly analyzed if the analysis was conducted with random sampling. By purposefully sampling, I was able to control, to the best of my ability, the presence of variables which would have muddied my analysis.
Conclusion
My research shows that even discourse that is created with the promise and intention to portray a truthful representation of a person can only do so much. All discourse is constructed and has ulterior motives (both on the side of the journalist and the celebrity, in this case). This research serves to question the very promise and existence of authenticity, something it would not be capable of doing if my sampling method attempted to create a sample which represented every type of magazine authentically. Whilst there are clear benefits to the “gold standard” of random sampling and huge sample sizes, these are most visible when it comes to quantitative research. Sampling is not a one-size-fits-all part of research. Not only must the sampling method be chosen carefully, it also needs to be re-visited and re-evaluated throughout the study to ensure both the sampling method and sample size is correct to answer the research questions. While undertaking this reflexive process can be extra work, avoiding it can lead to oversights or overgeneralizations which create work that isn’t as interesting or impactful as it could be. Using poor sampling and methods can even be seen as an ethical concern—wasting the resources and time of both researchers and participants for work that ends up being unimportant due to researcher error (Resnik, 2019).
In sum, sampling impacts every part of a piece of work. Researchers must ensure they are considering sampling seriously in its own right rather than seeing it as just a checkbox on a bureaucratic form. Over the course of my research, I have made this shift: going from the easy answer of “sample size=30” to picking my sample in order to best serve my research questions and make the most of my study.
References
Adler, P. A. and Adler, P. 2012. In: Baker, S.E. and Edwards, R eds, How many qualitative interviews is enough?. NCRM, National Centre for Research Methods Review Paper, pp 8-11.
Becker, H. S. 2012. In: Baker, S.E. and Edwards, R eds, How many qualitative interviews is enough?. NCRM, National Centre for Research Methods Review Paper, pp 15.
Bryman, A. 2012. In: Baker, S.E. and Edwards, R eds, How many qualitative interviews is enough?. NCRM, National Centre for Research Methods Review Paper, pp 18-20.
Condé Nast. 2021. About. [Online]. [Accessed 17 August 2021]. Available from: https://www.condenast.com/about
Corner, J. 2002. Performing the Real: Documentary Diversions. Television and New Media, 3, pp 255-269.
Dyer, R. 1998. Stars, London, British Film Institute.
Eckert, P. 2003. Sociolinguistics and authenticity: An elephant in the room. Journal of Sociolinguistics, 7, pp 392-431.
Ember, S. and Grynbaum, M. M. 2017. The Not-So-Glossy Future of Magazines. The New York Times.
Emmel, N. 2013. Sampling and Choosing Cases in Qualitative Research: A Realist Approach, SAGE Publications.
Evans, H. 2017. The Top 20 Cities That Celebrities Live In. Nimvo. [Online]. [Accessed 17 August 2021]. Available from: https://nimvo.com/top-20-cities-celebrities-live/
Ferris, K. O. 2001. Through a Glass, Darkly: The Dynamics of Fan-Celebrity Encounters. Symbolic Interactions, 24, pp 25-47.
Gill, R. 2018. Discourse analysis in media and communications research. In: Kearney, M. C. and Kackman, M. eds, The Craft of Criticism: Critical Media Studies in Practice. London: Routledge.
Goffman, E. 1956. The Presentation of Self in Everyday Life, The University of Edinburgh Social Sciences Research Center.
Hearst. 2021. About. [Online]. [Accessed 17 August 2021]. Available from: https://www.hearst.com/about
Johnson, P. 2014. Sociology and the critique of neoliberalism: Reflections on Peter Wagner and Axel Honneth. European Journal of Social Theory. 17(4), pp 516–533.
MacCannell, D. 1999. Staged Authenticity. The Tourist: A New Theory of the Leisure Class. Berkeley and Los Angeles: University of California Press.
Purcell, D. and Moore, C. 2019. Selling Southern Places: An Examination of Delta’s Sky magazine City Profiles. Southeastern Geographer, 59, pp 248-267.
Resnik, D. B. 2019. Stewardship of Research Resources. Accountability in Research, 26(3), pp 246-251.
Sacks, H. 1985. On doing “being ordinary”. In: Atkinson, J. M. and Heritage, J. eds. Structures of Social Action: Studies in Conversation Analysis. Cambridge: Cambridge University Press.
Sternheimer, K. 2015. Celebrity Culture and the American Dream: Stardom and Social Mobility, New York, Routledge.
Tagg, J. 1988. The Burden of Representation. Palgrave Macmillan.
Wood, L. A. and Kroger, R. O. 2000. Doing Discourse Analysis: Methods for Studying Action in Talk and Text, SAGE Publications.