Problem Solving

Abstract

Problem solving is goal-directed thinking — a form of learning in the Medical Subject Headings taxonomy — in which a solver moves from an initial state to a goal state across a space of intermediate states when no direct route is known. The information-processing tradition of Newell and Simon recast this movement as heuristic search through a problem space, while the earlier Gestalt tradition emphasized sudden insight and the restructuring of a stuck representation. A parallel line of work contrasts the flexible, structural knowledge of experts with the surface-bound strategies of novices. Contemporary cognitive neuroscience now tracks the moment of insight to specific cortical signatures. This article surveys these traditions and presents interactive demonstrations of search, mental set, and expert memory.

Keywords: problem solving, problem space, insight, functional fixedness, expertise

Problem solving is what cognition does when habit and direct retrieval both fail: a goal is wanted, the means are not obvious, and the gap must be closed by a directed sequence of mental operations (Newell et al., 1958). A problem exists whenever a solver has a goal but no immediately available action that reaches it; solving is the process of bridging that gap. Problems divide roughly into the well-defined, whose starting point, goal, and legal moves are fully specified, like a puzzle or a proof, and the ill-defined, whose goal or permissible moves must themselves be worked out — a boundary Simon argued is one of degree rather than kind, since a nominally well-structured problem turns ill-structured the moment its demands outrun the knowledge the solver brings to it (Simon, 1973). Two great research traditions have carved the field: the Gestalt psychologists, who studied the flash of restructuring that turns a baffling situation transparent (Duncker, 1945), and the information-processing theorists, who modeled solving as a traceable search through states (Newell et al., 1958).

What Problem Solving Is

The information-processing account gives problem solving its sharpest definition. A problem is specified by an initial state, a goal state, and a set of operators — legal actions that transform one state into another — together with any constraints on their use (Newell et al., 1958). Solving is finding a sequence of operators that carries the initial state to the goal. This framing applies equally to a chess position, an algebra exercise, and the Tower of Hanoi, and it is what made problem solving computationally tractable: the same vocabulary describes a human protocol and a running program.

The Gestalt tradition had already isolated the phenomena that any such account must explain. Duncker's candle problem — mount a candle on a wall given a box of tacks — showed that solvers fail when an object is perceived only in its usual role, a block he named functional fixedness: the box is seen as a container, not as a platform (Duncker, 1945). Maier's two-string problem, in which two cords hanging out of reach must be tied together, showed that the decisive move — setting one cord swinging as a pendulum — rarely occurs to solvers unless an unnoticed hint nudges the representation, and that solvers who succeed often cannot report why (Maier, 1931). Both results point to the same lesson: how a problem is represented largely determines whether it can be solved at all.

Figure 1

A Problem as Search Through a Space of States

The Problem Space

Newell and Simon's central construct is the problem space: the set of all states reachable from the initial state by applying operators, together with the solver's knowledge of where in that set it currently stands (Newell et al., 1958). Because the space branches, its size explodes with depth — even the modest Tower of Hanoi has an exponential state graph, and chess is astronomically larger. Exhaustive search is therefore impossible for any interesting problem, and the science of problem solving is largely the study of the shortcuts that make search feasible.

These shortcuts are heuristics: rules that select promising moves without any guarantee of optimality. The best known is means-ends analysis, which repeatedly identifies the largest difference between the current state and the goal and selects an operator that reduces that difference, setting up a subgoal when the chosen operator cannot yet be applied (Newell et al., 1958). Hill climbing takes whatever step most improves the current state, at the risk of stranding the solver on a local maximum; working backward starts from the goal. The Tower of Hanoi makes the logic visible: the optimal solution decomposes recursively into subgoals and takes exactly 2^n − 1 moves for n disks, a structure the demonstration below lets the reader step through.

Insight and Impasse

Not all problems yield to incremental search. Some are solved in a sudden reorganization — the Aha! experience the Gestalt psychologists called insight, in which an impasse breaks and the solution appears whole (Duncker, 1945). Kaplan and Simon reconciled this with the information-processing view by treating insight as search in a space of representations: when the initial encoding of a problem admits no solution, progress requires switching to a different representation, and the felt suddenness reflects the abrupt change of search space rather than any non-computational magic (Kaplan & Simon, 1990). On this reading the nine-dot problem is hard because solvers tacitly bound the search to the square the dots imply, and a process model of that self-imposed constraint predicts which moves such solvers will and will not try (MacGregor et al., 2001). Ohlsson's representational-change theory supplies the general mechanism: an impasse breaks when the solver relaxes a constraint that an over-tight encoding has imposed or decomposes a perceptual chunk into its parts. Knoblich and colleagues gave that theory its sharpest test with matchstick-arithmetic problems, showing that difficulty tracks the two operations it names, so that problems requiring a solver to break a tighter constraint or a more cohesive chunk are reliably harder and elicit longer impasses (Knoblich et al., 1999).

Prior experience can itself become the obstacle. Luchins's water-jar experiments showed that a run of problems all solved by one formula installs a mental set, or Einstellung, so tenacious that solvers keep applying the learned method even when a far simpler one is available, and some miss an obvious direct solution entirely (Luchins, 1942). Weisberg's integrated theory argues that insight and routine analysis are not separate faculties but endpoints of a continuum, with the balance set by how far the solver's existing knowledge carries them before restructuring becomes necessary (Weisberg, 2015). Time away from a problem can also help: a meta-analysis confirms a genuine, if modest, incubation effect, strongest when the initial impasse was well established (Sio & Ormerod, 2009). What the break accomplishes is still debated: Gilhooly argues the gains come not from unconscious work but from the fading of the misleading mental set that produced the impasse, so that returning with a fresh start lets a more productive intuition surface (Gilhooly, 2016). The demonstration below reproduces the Einstellung design directly.

Cognitive neuroscience has since given insight a physiology. Recording during solutions that solvers themselves rated as insightful, Kounios and Beeman found a burst of high-frequency activity over the right anterior temporal lobe about a third of a second before the answer reached awareness, preceded by a shift in resting attention (Kounios & Beeman, 2014). Solutions arrived at by insight are also, on average, more likely to be correct than those ground out analytically, contradicting the folk suspicion that a hunch is untrustworthy (Salvi et al., 2016). The Aha! feeling is itself imperfect, however: it can accompany wrong answers, so the phenomenology of certainty and the fact of correctness are separable (Danek & Wiley, 2017). Ultra-high-field imaging localizes the insight response in part to dopaminergic and hippocampal circuitry, tying the moment of restructuring to reward and memory systems (Tik et al., 2018).

Expertise

The clearest single finding in the study of skilled problem solving is that experts do not simply search harder — they see the problem differently. Chase and Simon showed that chess masters briefly shown a position from a real game reconstruct it almost perfectly, while novices manage only a few pieces; but when the same pieces are scattered at random, the master's advantage collapses to nearly nothing (Chase & Simon, 1973). The master is not blessed with a photographic memory but has stored tens of thousands of meaningful configurations, or chunks, and perceives the board in their terms — a capacity that helps only when the board is meaningful. Working memory is the bottleneck the chunk circumvents: a handful of chunks each standing for a whole cluster of pieces packs far more into the same limited working memory than a handful of single pieces.

The same reorganization governs how experts categorize problems. Chi and colleagues found that physics novices sort problems by surface features — the inclined plane, the spring, the pulley — whereas experts sort the identical problems by the deep principle needed to solve them, such as conservation of energy (Chi et al., 1981). Because the expert's categories point straight at a solution method, classification and solution are nearly the same act. Experience also builds the machinery of analogy: Gick and Holyoak showed that solvers who have encoded one problem's solution schema can transfer it to a structurally similar problem in a new domain, though spontaneous transfer without a hint is surprisingly rare — the link between transfer of learning and problem solving (Gick & Holyoak, 1980). The demonstration below reproduces the chess recall dissociation.

Table 1
How Expert Problem Solving Differs From Novice Problem Solving
Note. The pattern is documented in chess (Chase & Simon, 1973) and in physics (Chi et al., 1981); the scrambled-material row is the control that rules out raw memory superiority.

Types of Problem Solving

Beyond being a subject in its own right, Problem Solving is a formal descriptor in the National Library of Medicine's Medical Subject Headings, which places it at tree positions F02.463.425.725, beneath Learning, and F02.463.785.810, beneath Thinking, and hangs its recognized narrower kinds beneath it. These subtypes are a classification built to index the literature, not a claim about the mind's natural joints. Table 2 lists the direct children of the descriptor.

Two cautions keep this taxonomy in its place. It is a classification for indexing, not a theory asserting these subtypes are mutually exclusive or exhaustive. And a MeSH subtype is a narrower topic, not a component process: listing a child under Problem Solving locates it in an index and says nothing, on its own, about the mechanisms this article describes.

Worked Example

Luchins's water-jar task shows mental set arithmetically. A solver is given three jars of fixed capacity and must measure out an exact target by filling and pouring. The first several problems are all solved by the same three-jar formula: fill the largest jar B, pour off the middle jar A once and the small jar C twice, leaving B − A − 2C. For a problem with A = 21, B = 127, C = 3 and a target of 100, this gives 127 − 21 − 2(3) = 127 − 21 − 6 = 100, and repeated success installs the set.

Now consider the critical problem: A = 28, B = 76, C = 3, with a target of 25. The entrenched formula yields B − A − 2C = 76 − 28 − 6 = 42, which overshoots the target and does not reach it. Yet a one-step method is available: A − C = 28 − 3 = 25 exactly. A solver freed of the set sees the direct solution immediately; a solver under Einstellung applies the long formula, fails, and often persists with it rather than stepping back (Luchins, 1942). The interactive Einstellung demonstration above steps through this exact sequence, computing both the set formula and the shortcut for each problem so the reader can watch the trap open and close.

Discussion

Problem solving is the domain where cognitive psychology first showed that a mental process could be modeled step for step. The information-processing tradition supplied a vocabulary — states, operators, problem space, heuristic search — precise enough to run as a program and to be checked against a solver's spoken protocol (Newell et al., 1958). Its weak methods, means-ends analysis chief among them, explain how bounded solvers cope with spaces far too large to enumerate. What that tradition initially handled least well was the phenomenon the Gestalt psychologists had made central: the sudden restructuring of a stuck representation. Kaplan and Simon's move — treating insight as search across representations rather than within one — folded the Gestalt data back into the information-processing frame without explaining insight away (Kaplan & Simon, 1990).

The two obstacles the Gestaltists identified, functional fixedness and mental set, remain the sharpest demonstrations that representation, not raw search effort, is often the limiting factor (Duncker, 1945; Luchins, 1942). The expertise literature completes the picture from the other side: with enough structured experience, the representation that novices must struggle to find is simply perceived, because the expert has stored the relevant patterns and organized knowledge around the principles that solve problems rather than the surfaces that describe them (Chase & Simon, 1973; Chi et al., 1981).

Current Directions

The most active frontier is the cognitive neuroscience of insight. Electrophysiology and imaging have converged on a reproducible signature — a right-temporal high-frequency burst, a preparatory shift of attention, and the involvement of reward and memory circuitry — that turns the once-ineffable Aha! moment into a measurable event (Kounios & Beeman, 2014; Tik et al., 2018). This work has also revised long-standing intuitions: insight solutions are more accurate than analytic ones on the same problems (Salvi et al., 2016), even though the subjective Aha! can attach to errors, which keeps the feeling and the fact of insight conceptually distinct (Danek & Wiley, 2017).

A second strand seeks an integrated theory that dissolves the old dichotomy between insight and analysis, treating them as regions of one continuum whose position is fixed by the reach of the solver's prior knowledge (Weisberg, 2015). A third revisits classic effects with modern methods and meta-analysis, confirming that incubation is real but bounded and pinning down the conditions under which stepping away from a problem actually helps (Sio & Ormerod, 2009).

Glossary

Chunk.

A familiar, meaningful configuration of elements stored as a single unit, allowing an expert to hold far more domain material in working memory than its raw element count would permit.

Einstellung effect.

The tendency, after repeatedly solving problems by one method, to keep applying that method even when a simpler solution exists; also called mental set.

Functional fixedness.

A block in which an object is perceived only in its customary function, preventing its use in a novel way that the problem requires.

Goal state.

The condition that defines a problem as solved; the target of the search through the problem space.

Heuristic.

A rule of thumb that selects promising moves and reduces search without guaranteeing an optimal or even a correct solution.

Incubation.

Improvement in the likelihood of solution after a period of not working on a problem, a genuine but modest effect strongest when an impasse was firmly established.

Initial state.

The situation in which a problem begins, before any operator has been applied.

Insight.

The sudden emergence of a solution through the restructuring of a problem's representation, accompanied by an Aha! experience; modeled as a shift to a new search space.

Means-ends analysis.

A weak method that repeatedly selects an operator to reduce the largest difference between the current state and the goal, creating subgoals when an operator cannot yet be applied.

Operator.

A legal action that transforms one state of a problem into another; the edges of the problem space.

Problem space.

The set of all states reachable from the initial state by applying operators, through which the solver searches for a path to the goal.

Restructuring.

A change in how a problem is mentally represented, replacing an encoding that admits no solution with one that does; the core of insight.

Subgoal.

An intermediate objective set up to enable an operator that cannot yet be applied, decomposing a hard problem into more manageable parts.

Weak method.

A general, domain-independent problem-solving strategy, such as means-ends analysis or hill climbing, that requires little knowledge and applies broadly but searches inefficiently.

Well-defined problem.

A problem whose initial state, goal state, and legal operators are all fully specified, in contrast to an ill-defined problem whose goal or moves must first be worked out.

Key Researchers

Mark Beeman (contemporary). Professor of Psychology at Northwestern University; with John Kounios he mapped the neural signature of insight, isolating a right-temporal gamma burst that precedes the Aha! moment. Faculty Page - Google Scholar

Michelene T. H. Chi (contemporary). Regents Professor at Arizona State University; she established that experts categorize problems by deep structure while novices sort by surface features. Wikipedia - Google Scholar

Karl Duncker (1903-1940). Gestalt psychologist at Berlin and Swarthmore; he devised the candle problem, named functional fixedness, and wrote the monograph On Problem-Solving. Wikipedia

Keith J. Holyoak (b. 1950). Distinguished Professor of Psychology at the University of California, Los Angeles; with Mary Gick he demonstrated analogical problem solving and schema induction. Faculty Page - ORCID

John Kounios (contemporary). Professor of Psychological and Brain Sciences at Drexel University; with Mark Beeman he used EEG and fMRI to isolate the Aha! moment and its brain correlates. Faculty Page - Google Scholar

Allen Newell (1927-1992). Computer scientist and psychologist at Carnegie Mellon University; with Herbert Simon he formalized human problem solving as heuristic search through a problem space. Wikipedia

Stellan Ohlsson (contemporary). Professor Emeritus of Psychology at the University of Illinois at Chicago; he formulated the representational-change theory of insight, in which impasses break through constraint relaxation and chunk decomposition. Faculty Page - Google Scholar

Herbert A. Simon (1916-2001). Polymath at Carnegie Mellon University and Nobel laureate; with Allen Newell he co-founded the information-processing account of problem solving and the theory of bounded rationality. Wikipedia

Frequently Asked Questions

What is problem solving in cognitive psychology?
It is goal-directed thinking that moves a solver from an initial state to a goal state through a sequence of mental operations when no direct route is available; the information-processing account models it as heuristic search through a problem space (Newell et al., 1958).

What is a problem space?
The problem space is the set of all states reachable from the starting point by applying legal operators, together with the solver's knowledge of where it currently stands; solving is finding a path through this space to the goal (Newell et al., 1958).

What is functional fixedness?
Functional fixedness is a block in which an object is seen only in its usual role, so a solver fails to use it in the novel way a problem demands, as in Duncker's candle problem where the tack box must become a platform (Duncker, 1945).

What is the Einstellung effect?
It is a mental set in which repeated success with one method makes a solver keep applying it even when a simpler solution exists; Luchins demonstrated it with water-jar problems in which solvers overlooked a one-step answer (Luchins, 1942).

What is insight in problem solving?
Insight is the sudden emergence of a solution through the restructuring of a problem's representation, felt as an Aha! moment; it can be modeled as a shift to a new search space when the initial representation admits no solution (Kaplan & Simon, 1990).

How do experts solve problems differently from novices?
Experts perceive problems in terms of stored meaningful patterns and organize their knowledge around deep solution principles, while novices attend to surface features; chess masters reconstruct real positions far better than novices but lose the edge on random boards (Chase & Simon, 1973).

Does taking a break help solve a problem?
Yes, modestly; a meta-analysis found a genuine incubation effect in which time away from a problem raises the chance of solution, an effect that is strongest once a firm impasse has been reached (Sio & Ormerod, 2009).

Are insight solutions more reliable than analytic ones?
On the same problems, solutions reached by insight are on average more likely to be correct than those reached by step-by-step analysis, though the Aha! feeling can also accompany wrong answers, so certainty and correctness remain distinct (Salvi et al., 2016).

References