Qualitative research gives rich insights into the quintessentially human aspects of software engineering as a socio-technical system. Qualitative research spans diverse strategies and methods, from interpretivist, in situ observational field studies, to deductive coding of data from mining studies. Advances in large language models and generative AI (GenAI) have prompted claims that artificial intelligence could automate qualitative analysis. Such claims are overgeneralizing from narrow successes. GenAI support must be carefully adapted to the data of interest, but also to the characteristics of a particular research strategy. In this Frontiers of SE paper, we discuss the emerging use of GenAI in relation to the broad spectrum of qualitative research in software engineering. We outline the dimensions of qualitative work in software engineering, review emerging empirical evidence for GenAI assistance, examine the pros and cons of GenAI-mediated qualitative research practices, and revisit qualitative research quality factors, in light of GenAI. Our goal is to inform researchers about the promises and pitfalls of GenAI-assisted qualitative research. We conclude with future plans to ad
There has been intense debate among qualitative researchers about whether generative AI is suitable for qualitative research. In this paper, we summarize the broader ongoing discussion of generative AI in qualitative research and its implications for software engineering researchers. The qualitative research approach, small-q (positivist or post-positivist) or Big Q (non-positivist), is among the major criteria for determining whether generative AI can be used in qualitative research. In addition to research philosophy and research approach, skills, ethics, and personal preferences also play a role in researchers' decisions about whether to use AI in qualitative research.
Qualitative researchers use tools to collect, sort, and analyze their data. Should qualitative researchers use large language models (LLMs) as part of their practice? LLMs could augment qualitative research, but it is unclear if their use is appropriate, ethical, or aligned with qualitative researchers' goals and values. We interviewed twenty qualitative researchers to investigate these tensions. Many participants see LLMs as promising interlocutors with attractive use cases across the stages of research, but wrestle with their performance and appropriateness. Participants surface concerns regarding the use of LLMs while protecting participant interests, and call attention to an urgent lack of norms and tooling to guide the ethical use of LLMs in research. We document the rapid and broad adoption of LLMs across surfaces, which can interfere with intentional use vital to qualitative research. We use the tensions surfaced by our participants to outline recommendations for researchers considering using LLMs in qualitative research and design principles for LLM-assisted qualitative research tools.
Among the various forms of reasoning studied in the context of artificial intelligence, qualitative reasoning makes it possible to infer new knowledge in the context of imprecise, incomplete information without numerical values. In this paper, we propose a formal framework unifying several forms of extensions and combinations of qualitative formalisms, including multi-scale reasoning, temporal sequences, and loose integrations. This framework makes it possible to reason in the context of each of these combinations and extensions, but also to study in a unified way the satisfiability decision and its complexity. In particular, we establish two complementary theorems guaranteeing that the satisfiability decision is polynomial, and we use them to recover the known results of the size-topology combination. We also generalize the main definition of qualitative formalism to include qualitative formalisms excluded from the definitions of the literature, important in the context of combinations.
Qualitative research, renowned for its in-depth exploration of complex phenomena, often involves time-intensive analysis, particularly during the coding stage. Existing software for qualitative evaluation frequently lacks automatic coding capabilities, user-friendliness, and cost-effectiveness. The advent of Large Language Models (LLMs) like GPT-3 and its successors marks a transformative era for enhancing qualitative analysis. This paper introduces QualiGPT, a tool developed to address the challenges associated with using ChatGPT for qualitative analysis. Through a comparative analysis of traditional manual coding and QualiGPT's performance on both simulated and real datasets, incorporating both inductive and deductive coding approaches, we demonstrate that QualiGPT significantly improves the qualitative analysis process. Our findings show that QualiGPT enhances efficiency, transparency, and accessibility in qualitative coding. The tool's performance was evaluated using inter-rater reliability (IRR) measures, with results indicating substantial agreement between human coders and QualiGPT in various coding scenarios. In addition, we also discuss the implications of integrating AI i
A powerful tool in control and systems engineering is represented by Nyquist plots, for which a qualitative representation often gives a clearer visualization of the frequency response function that is typically not given by computer programs, especially if portions of the Nyquist plot extend to infinity. This letter addresses the graphical analysis of the frequency response function, with the objective of enhancing the procedure for the qualitative construction of Nyquist plots. Several results supported by analytical proofs are derived for what concerns the low and high frequency behavior, which enable to improve the qualitative construction of Nyquist plots in the vicinity of the initial and final points.
This chapter demonstrates how computational social science (CSS) tools are extending and expanding research on aging. The depth and context from traditionally qualitative methods such as participant observation, in-depth interviews, and historical documents are increasingly employed alongside scalable data management, computational text analysis, and open-science practices. Machine learning (ML) and natural language processing (NLP), provide resources to aggregate and systematically index large volumes of qualitative data, identify patterns, and maintain clear links to in-depth accounts. Drawing on case studies of projects that examine later life--including examples with original data from the DISCERN study (a team-based ethnography of life with dementia) and secondary analyses of the American Voices Project (nationally representative interview)--the chapter highlights both uses and challenges of bringing CSS tools into more meaningful dialogue with qualitative aging research. The chapter argues such work has potential for (1) streamlining and augmenting existing workflows, (2) scaling up samples and projects, and (3) generating multi-method approaches to address important question
The typical phases of Bayesian network (BN) structured development include specification of purpose and scope, structure development, parameterisation and validation. Structure development is typically focused on qualitative issues and parameterisation quantitative issues, however there are qualitative and quantitative issues that arise in both phases. A common step that occurs after the initial structure has been developed is to perform a rough parameterisation that only captures and illustrates the intended qualitative behaviour of the model. This is done prior to a more rigorous parameterisation, ensuring that the structure is fit for purpose, as well as supporting later development and validation. In our collective experience and in discussions with other modellers, this step is an important part of the development process, but is under-reported in the literature. Since the practice focuses on qualitative issues, despite being quantitative in nature, we call this step qualitative parameterisation and provide an outline of its role in the BN development process.
Simultaneous localization and mapping (SLAM) are essential in numerous robotics applications, such as autonomous navigation. Traditional SLAM approaches infer the metric state of the robot along with a metric map of the environment. While existing algorithms exhibit good results, they are still sensitive to measurement noise, sensor quality, and data association and are still computationally expensive. Alternatively, some navigation and mapping missions can be achieved using only qualitative geometric information, an approach known as qualitative spatial reasoning (QSR). We contribute a novel probabilistic qualitative localization and mapping approach in this work. We infer both the qualitative map and the qualitative state of the camera poses (localization). For the first time, we also incorporate qualitative probabilistic constraints between camera poses (motion model), improving computation time and performance. Furthermore, we take advantage of qualitative inference properties to achieve very fast approximated algorithms with good performance. In addition, we show how to propagate probabilistic information between nodes in the qualitative map, which improves estimation performa
Computational developments--particularly artificial intelligence--are reshaping social scientific research and raise new questions for in-depth methods such as ethnography and qualitative interviewing. Building on classic debates about computers in qualitative data analysis (QDA), we revisit possibilities and dangers in an era of automation, Large Language Model (LLM) chatbots, and 'big data.' We introduce a typology of contemporary approaches to using computers in qualitative research: streamlining workflows, scaling up projects, hybrid analytical methods, the sociology of computation, and technological rejection. Drawing from scaled team ethnographies and solo research integrating computational social science (CSS), we describe methodological choices across study lifecycles, from literature reviews through data collection, coding, text retrieval, and representation. We argue that new technologies hold potential to address longstanding methodological challenges when deployed with knowledge, purpose, and ethical commitment. Yet a pragmatic approach--moving beyond technological optimism and dismissal--is essential given rapidly changing tools that are both generative and dangerous.
We define what it means for a joint probability distribution to be compatible with a set of independent causal mechanisms, at a qualitative level -- or, more precisely, with a directed hypergraph ${\mathcal{A}}$, which is the qualitative structure of a probabilistic dependency graph (PDG). When ${\mathcal{A}}$ represents a qualitative Bayesian network, QIM-compatibility with ${\mathcal{A}}$ reduces to satisfying the appropriate conditional independencies. But giving semantics to hypergraphs using QIM-compatibility lets us do much more. For one thing, we can capture functional dependencies. For another, we can capture important aspects of causality using compatibility: we can use compatibility to understand cyclic causal graphs, and to demonstrate structural compatibility, we must essentially produce a causal model. Finally, QIM-compatibility has deep connections to information theory. Applying our notion to cyclic structures helps to clarify a longstanding conceptual issue in information theory.
Qualitative research delves deeply into individual complex perspectives on technology and various phenomena. However, a meticulous analysis of qualitative data often requires a significant amount of time, especially during the crucial coding stage. Although there is software specifically designed for qualitative evaluation, many of these platforms fall short in terms of automatic coding, intuitive usability, and cost-effectiveness. With the rise of Large Language Models (LLMs) such as GPT-3 and its successors, we are at the forefront of a transformative era for enhancing qualitative analysis. In this paper, we introduce QualiGPT, a specialized tool designed after considering challenges associated with ChatGPT and qualitative analysis. It harnesses the capabilities of the Generative Pretrained Transformer (GPT) and its API for thematic analysis of qualitative data. By comparing traditional manual coding with QualiGPT's analysis on both simulated and actual datasets, we verify that QualiGPT not only refines the qualitative analysis process but also elevates its transparency, credibility, and accessibility. Notably, compared to existing analytical platforms, QualiGPT stands out with i
Concern has recently been expressed by HCI researchers as to the inappropriate treatment of qualitative studies through a positivistic mode of evaluation that places emphasis on metrics and measurement. This contrasts with the nature of qualitative research, which privileges interpretation and understanding over quantification. This paper explains the difference between positivism and interpretivism, the limits of quantification in human science, the distinctive contribution of qualitative research, and how quality assurance might be provided for in the absence of numbers via five basic criteria that reviewers may use to evaluate qualitative studies on their own terms.
Voice is a natural mode of expression offered by modern computer-based systems. Qualitative perspectives on voice-based user experiences (voice UX) offer rich descriptions of complex interactions that numbers alone cannot fully represent. We conducted a systematic review of the literature on qualitative approaches to voice UX, capturing the nature of this body of work in a systematic map and offering a qualitative synthesis of findings. We highlight the benefits of qualitative methods for voice UX research, identify opportunities for increasing rigour in methods and outcomes, and distill patterns of experience across a diversity of devices and modes of qualitative praxis.
A qualitative representation $φ$ is like an ordinary representation of a relation algebra, but instead of requiring $(a; b)^φ= a^φ| b^φ$, as we do for ordinary representations, we only require that $c^φ\supseteq a^φ| b^φ\iff c\geq a ; b$, for each $c$ in the algebra. A constraint network is qualitatively satisfiable if its nodes can be mapped to elements of a qualitative representation, preserving the constraints. If a constraint network is satisfiable then it is clearly qualitatively satisfiable, but the converse can fail. However, for a wide range of relation algebras including the point algebra, the Allen Interval Algebra, RCC8 and many others, a network is satisfiable if and only if it is qualitatively satisfiable. Unlike ordinary composition, the weak composition arising from qualitative representations need not be associative, so we can generalise by considering network satisfaction problems over non-associative algebras. We prove that computationally, qualitative representations have many advantages over ordinary representations: whereas many finite relation algebras have only infinite representations, every finite qualitatively representable algebra has a finite qualitative
We analyze the effect of higher order viscosity corrections in a spatially homogeneous and isotropic universe. The viscous effects are modeled by introducing a second-order term to the energy density in the equation of state and on the expansion factor. An autonomous dynamical system is then constructed to study the qualitative behavior of the energy density and expansion factor, employing the Raychaudhuri equation and the conservation of energy. We obtain the analytical expressions for the equilibrium points in terms of the model parameters, presenting several phase diagrams. The analysis of these diagrams reveals the presence of bifurcations, which manifest as changes in stability and qualitative behavior. Those changes lead to different configurations for the universe, such as bouncing, rebouncing and cyclic scenarios.
We review discourses about the philosophy of science in qualitative research and evidence from cognitive linguistics in order to ground a framework for discussing the use of Large Language Models (LLMs) to support the qualitative analysis process. This framework involves asking two key questions: "is the LLM proposing or refuting a qualitative model?" and "is the human researcher checking the LLM's decision-making directly?". We then discuss an implication of this framework: that using LLMs to surface counter-examples for human review represents a promising space for the adoption of LLMs into the qualitative research process. This space is promising because it is a site of overlap between researchers working from a variety of philosophical assumptions, enabling productive cross-paradigm collaboration on tools and practices.
Introduction - Norman K Denzin and Yvonna S Lincoln The Discipline and Practice of Qualitative Research PART ONE: LOCATING THE FIELD Qualitative Methods - Arthur J Vidich and Stanford M Lyman Their History in Sociology and Anthropology Reconstructing the Relationships between Universities and Society through Action Research - Davydd J Greenwood and Morten Levin For Whom? Qualitative Research, Representations and Social Responsibilities - Michelle Fine et al Ethics and Politics in Qualitative Research - Clifford G Christians PART TWO: PARADIGMS AND PERSPECTIVES IN TRANSITION Paradigmatic Controversies, Contradictions and Emerging Confluences - Yvonna S Lincoln and Egon G Guba Three Epistemological Stances for Qualitative Inquiry - Thomas A Schwandt Interpretivism, Hermeneutics and Social Constructionism Feminisms and Qualitative Research at and into the Millennium - Virginia L Olesen Racialized Discourses and Ethnic Epistemologies - Gloria Ladson-Billings Rethinking Critical Theory and Qualitative Research - Joe L Kincheloe and Peter McLaren Cultural Studies - John Frow and Meaghan Morris Sexualities, Queer Theory and Qualitative Research - Joshua Gamson PART THREE: STRATEGIES OF INQUIRY The Choreography of Qualitative Research Design - Valerie J Janesick Minuets, Improvisations and Crystallization An Untold Story? Doing Funded Qualitative Research - Julianne Cheek Performance Ethnography - Michal M McCall A Brief History and Some Advice Case Studies - Robert E Stake Ethnography and Ethnographic Representation - Barbara Tedlock Analyzing Interpretive Practice - Jaber F Gubrium and James A Holstein Grounded Theory - Kathy Charmaz Objectivist and Constructivist Methods Undaunted Courage - William G Tierney Life History and the Postmodern Challenge Testimonio, Subalternity and Narrative Authority - John Beverley Participatory Action Research - Stephen Kemmis and Robin McTaggart Clinical Research - William L Miller and Benjamin F Crabtree PART FOUR: METHODS OF COLLECTING AND ANALYZING EMPIRICAL MATERIALS The Interview - Andrea Fontana and James H Frey From Structured Questions to Negotiated Text Rethinking Observation - Michael V Angrosino and Kimberly A Mays de Perez From Method to Context The Interpretation of Documents and Material Culture - Ian Hodder Re-Imagining Visual Methods - Douglas Harper Galileo to Neuromancer Auto-Ethnography, Personal Narrative, Reflexivity - Carolyn Ellis and Arthur P Bochner Researcher as Subject Data Management and Analysis Methods - Gery W Ryan and H Russell Bernard Software and Qualitative Research - Eben A Weitzman Analyzing Talk and Text - David Silverman Focus Groups in Feminist Research - Esther Madriz Applied Ethnography - Erve Chambers PART FIVE: THE ART AND PRACTICES OF INTERPRETATION, EVALUATION AND REPRESENTATION The Problem of Criteria in the Age of Relativism - John K Smith and Deborah K Deemer The Practices and Politics of Interpretation - Norman K Denzin Writing - Laurel Richardson A Method of Inquiry Anthropological Poetics - Ivan Brady Understanding Social Programs through Evaluation - Jennifer C Greene Influencing the Policy Process with Qualitative Research - Ray C Rist PART SIX: THE FUTURE OF QUALITATIVE RESEARCH Qualitative Inquiry - Mary M Gergen and Kenneth J Gergen Tensions and Transformations The Seventh Moment - Yvonna S Lincoln and Norman K Denzin Out of the Past
This paper deals with enriched qualitative belief functions for reasoning under uncertainty and for combining information expressed in natural language through linguistic labels. In this work, two possible enrichments (quantitative and/or qualitative) of linguistic labels are considered and operators (addition, multiplication, division, etc) for dealing with them are proposed and explained. We denote them $qe$-operators, $qe$ standing for "qualitative-enriched" operators. These operators can be seen as a direct extension of the classical qualitative operators ($q$-operators) proposed recently in the Dezert-Smarandache Theory of plausible and paradoxist reasoning (DSmT). $q$-operators are also justified in details in this paper. The quantitative enrichment of linguistic label is a numerical supporting degree in $[0,\infty)$, while the qualitative enrichment takes its values in a finite ordered set of linguistic values. Quantitative enrichment is less precise than qualitative enrichment, but it is expected more close with what human experts can easily provide when expressing linguistic labels with supporting degrees. Two simple examples are given to show how the fusion of qualitative
Qualitative probabilistic reasoning in a Bayesian network often reveals tradeoffs: relationships that are ambiguous due to competing qualitative influences. We present two techniques that combine qualitative and numeric probabilistic reasoning to resolve such tradeoffs, inferring the qualitative relationship between nodes in a Bayesian network. The first approach incrementally marginalizes nodes that contribute to the ambiguous qualitative relationships. The second approach evaluates approximate Bayesian networks for bounds of probability distributions, and uses these bounds to determinate qualitative relationships in question. This approach is also incremental in that the algorithm refines the state spaces of random variables for tighter bounds until the qualitative relationships are resolved. Both approaches provide systematic methods for tradeoff resolution at potentially lower computational cost than application of purely numeric methods.