Notes
Research, experiments, and reflections on AI and design, published along with the process of making them.
Learning and education (22)
How people learn at school and at work, and how to design teaching.
-
What Does Design Ask People to Let Go Of, and What Does It Let Go Of? Unlearning Considered through a Broad View of Design
2026-11-13
This note examines unlearning through a broad view of design that is not limited to form-giving, namely Simon's definition of design as the effort to change existing situations into preferred ones. It draws on 50 sources from design research, HCI, sustainability transitions research, organization studies, environmental psychology, and higher education research. What design asks people to let go of falls into three kinds: the prior ideas and frames of the designers themselves, the practices of users and societies, and artifacts and institutions themselves. The first kind has experimental evidence. Features of an example seen beforehand are carried into new designs, and instructing designers not to copy the example did not reduce copying (Vasconcelos and Crilly 2016 reviewed 25 studies). Engineering design faculty fixated to the same extent as novices (Linsey et al. 2010; Viswanathan and Linsey 2013). However, repeating features of prior solutions also has benefits, so fixation cannot be treated as always bad (Crilly and Cardoso 2017). The second and third kinds are covered mainly by conceptual and case studies: guidelines that take practices as the unit of design and create crises of routine (Kuijer et al. 2013), critiques that transitions research has favored novelty and neglected ending things (Heyen et al. 2017; van Oers 2024), a framework for undesigning technology (Pierce 2012), and an ethnography in which unmaking, rather than making, raised discussion (Sabie et al. 2022). Tonkinwise (2018) argued that design destroys what exists by replacing it, that users are required to unlearn, and that such changes are difficult to reverse. Placing these findings alongside the companion note on the learning sciences (old knowledge remains rather than disappearing, and habits change more with a change of context than with resolve) supports the following inference. If design helps unlearning, it does so not by erasing the old but by creating situations in which old cues are removed and new ways can be tried, such as life transitions like moving or changing jobs and settings for experimentation. A field experiment also found that an information-based intervention promoting sustainable behaviors worked for people within three months of relocating (Verplanken and Roy 2016). At the same time, what is let go of is not decided neutrally. Deliberate destabilization can reproduce unjust regimes (van Oers et al. 2021), and the tools of human-centered design can displace Indigenous practices (Akama et al. 2019). Few studies on the design side have measured unlearning again after a delay (Coskun et al. 2015 identified a lack of long-term evaluation).
-
Can What Has Been Learned Be Let Go? Evidence on Unlearning from Research on Memory, Conceptual Change, Habits and Skills, and Correction
2026-11-13
Unlearning, the idea that one should first forget and then learn again, was examined using 66 sources from the learning sciences and cognitive psychology, separately for definitions, memory, conceptual change, procedures, habits and skills, interventions, and measurement. The term spread from organization studies, and at the individual level its definition has shifted from discarding knowledge to reducing the influence of old knowledge. In memory research, old memories are more often made harder to retrieve than erased. Retrieving some items makes related items harder to recall (retrieval-induced forgetting, g = 0.35 across 512 samples; Murayama et al. 2014), and trying not to retrieve a memory also produces forgetting (d = 0.20 to 0.40 across 120 studies; Clark et al. 2026). Extinguished responses return with changes in context and time, and attempts to rewrite memories through reconsolidation have not replicated reliably in humans (Hardwicke et al. 2016; Elsey et al. 2018). In conceptual change research, adults who had studied science still held naive theories: across 10 domains, statements on which naive and scientific theories disagree were verified more slowly and less accurately (Shtulman and Valcarcel 2012). The delay remained in older adults around 65 years of age and in professional scientists, and in physicists with a doctorate, brain regions associated with inhibition were more active. For procedures, a familiar solution reduced the problem-solving ability of expert chess players to that of players about three standard deviations lower in skill (Bilalić et al. 2008), and for habits, participants who had linked a new route to cues through implementation intentions still chose the old route in about half of their responses (an experiment cited by Wood and Neal 2007). Among interventions, refutation texts, which state a misconception, refute it, and explain the correct conception, have a moderate effect (g = 0.41 across 44 comparisons; Schroeder and Kucera 2022), and refutation alone is not sufficient without an explanation (Kendeou et al. 2014). However, evidence on effects beyond one month and on changes in behavior is limited: in an experiment with teachers, the effect was no longer detected 45 days later and intentions to use the practices did not change (Ferrero et al. 2020). Corrections of science-relevant misinformation were, on average, not significantly effective (d = 0.11; Chan and Albarracín 2023). The literature describes unlearning not as erasing the old but as arranging conditions under which the new prevails while the old remains. The measures used to check whether something was let go (response times, speed of relearning, behavior) differ across studies, and few studies followed the same people over time.
-
Does the Quantity of Work and Practice Build Skill? Evidence on Repetitions, Case Volume, Years of Experience, Practice Scheduling, Working Hours, and Workplace Experience
2026-11-13
The belief that doing more makes people better was examined for everyday work and practice, separately for six kinds of quantity (repetitions, case volume, years of experience, practice time and its scheduling, working hours, and workplace experience), using 46 academic sources and 1 report. Repetition has a large effect early on. Among windshield installers, output per worker was 53% higher after one year than at hire (Shaw and Lazear 2008), and at an automobile assembly plant, defects per vehicle fell by more than 80% in the first eight weeks of production (Levitt et al. 2013). However, the gains slowed down, the plant's gains were stored in the organization rather than in individual workers, and experience depreciated when production stopped. In surgery, patients of high-volume surgeons had lower mortality (Birkmeyer et al. 2003), and an instrumental-variable analysis supported a causal effect of hospital volume on mortality (Hentschker and Mennicken 2018). Recurrence after radical prostatectomy continued to fall until the surgeon had performed about 250 operations (Vickers et al. 2007). Cardiac surgeons' outcomes, however, improved with their volume at the same hospital but not with their volume at other hospitals (Huckman and Pisano 2006). Years of experience are a rough proxy for quantity. The correlation between work experience and job performance was .27, rising to .43 when experience was measured as the number of times a task was performed (Quiñones et al. 1995). In medicine, 32 of 62 evaluations reported lower quality with more years in practice (Choudhry et al. 2005), and patients of older hospitalists had higher mortality, but the association disappeared among physicians with high patient volumes (Tsugawa et al. 2017). Among Japanese car salespeople, years of experience correlated significantly with sales in the 3 to 9 year groups but not in the group with 10 or more years (Matsuo et al. 1999). For practice, the same amount produces different retention depending on how it is scheduled. Spaced practice led to higher performance than massed practice (Donovan and Radosevich 1999, mean effect size 0.46). Increasing the number of mathematics problems in one session from three to nine did not change scores one or four weeks later, whereas splitting the same ten problems into two sessions one week apart greatly raised scores four weeks later (Rohrer and Taylor 2006). The 10,000-hour figure is the name Gladwell (2008) gave, as a magic number, to the research of Ericsson et al. (1993); Ericsson himself wrote that there was no evidence for a magic number. Long working hours reduce output per hour. In First World War munition factories, output was proportional to hours up to 49 hours per week, rose more slowly above that point, and peaked at about 63 hours (Pencavel 2015). Interns on schedules that included shifts of 24 hours or more made 35.9% more serious medical errors (Landrigan et al. 2004). Shortening residents' duty hours, however, did not improve educational outcomes, and studies reported less operative exposure and more handovers (Ahmed et al. 2014). No study was found that directly measured whether longer working hours help people acquire routines. For workplace experience, the effect of developmental challenge showed diminishing returns, which feedback offset (DeRue and Wellman 2009), and the evidential basis for the rule that 70% of workplace learning comes from on-the-job experience is weak (Clardy 2018). The conditions that the literature repeatedly identified for turning quantity into skill were an environment where outcomes become known quickly, accumulation within the same organization and the same kind of work, spaced repetition, and reflection and support from others.
-
Does Generative AI Raise University Students' Motivation and Engagement? Reading the Meta-Analysis by Xia et al. (2025)
2026-10-24
Xia, Li, Yang, Weng, and Chiu (2025, Computers and Education: Artificial Intelligence) synthesized 33 studies that compared university students who used generative AI with students who did not, and reported positive effects of g = 1.20 on motivation (12 studies) and g = 0.52 to 0.87 on four types of engagement. A reading of the full text shows that these numbers support a narrow claim. The included studies contain chatbots that do not generate text (one selects answers from a database, another runs scripted conversations) and a study published before the November 2022 cutoff, and the activity of the control groups was not coded. Most measures are self-report questionnaires, and no overall heterogeneity statistic appears in the text or tables. When the effects are recalculated from the published data, I² ranges from 85% to 93%, and the prediction interval for each of the five categories includes negative values. The large mean for motivation is pulled by three studies that used the tool in class only (g = 2.94); without these three studies, the mean is 0.58. The data also contain one control-group entry that copies the experimental group's change score, and six rows excluded without mention in the text. The claims that the effect is larger in science fields, in individual learning, and with ChatGPT are not supported by the moderator tables. Compared with findings in existing notes, higher self-reported motivation and engagement answer a different question from what remains after the AI is removed; one randomized experiment found no difference in intrinsic motivation, and another found higher essay scores without differences in knowledge gain and transfer. From the view that design should create environments where people can try and try again, the paper does not code whether the AI handed over answers, whether learners attempted a task before asking the AI, or how many times they revised, so it cannot show whether the effects supported trying or shortened it.
-
Learning by Practice with Knowledge Withheld, Learning by Knowledge with Practice Withheld: The Literature on Whether Withholding One Side Has Meaning
2026-10-24
The claim that output matters more than input rests on comparisons in which all learners first saw the same material, such as retrieval practice versus restudy. This note asks the next question: does withholding one side, knowledge (verbal explanation, instruction, worked examples) or practice (solving, learning from experience, moving the body), have meaning in itself? It organizes 88 references from the learning sciences, cognitive psychology, educational psychology, philosophy, and Japanese writing on learning. The starting point is the double dissociation reported by Berry and Broadbent: in dynamic control tasks, practice improved control without improving verbalizable knowledge, and verbal instruction improved questionnaire scores without improving control. Withholding knowledge has meaning when instructions to search for rules lower performance on tasks whose regularities are not salient, when describing a well-practiced motor skill in words erases the effect of practice, and when the perceptual differentiation of experts cannot be reached by verbal instruction; designs that delay instruction are a matter of order, not of withholding. Withholding practice has meaning when novices learn more from worked examples than from solving, when intuitions formed by everyday experience resist correction even after further observation, when experience teaches errors in environments with wrong or missing feedback, when rare and irreversible events cannot be learned from experience, and when statistical reasoning and second-language grammar are transmitted by explicit instruction. In the Japanese literature, Ikuta's not teaching is a designed arrangement of the learning setting rather than an absence of instruction, Suwa reads verbalization as preparation for rebuilding rather than harm, and the reception-learning side grounds instruction-first in the abstractness of the content. What separates the two sides is the salience of the regularity, the quality of environmental feedback, whether the concept runs against intuition, the frequency of the event, the learner's stage, and the conditions of performance. Experiments that directly compare withholding with giving are few, and a systematic review found no group difference in most comparisons for the claimed advantage of implicit motor learning.
-
What Has Design Education Carried So Far? (Industry Sources)
2026-10-11
Following the previous note (what juniors should do when generative AI takes away the entry-level work), this note examines, through industry sources, what school-based design education has carried compared with entry-level work in the workplace. Sources include primary documents from the UK Design Council and Creative PEC, the US BLS and the New York Fed, AIGA, and Japan's METI and MEXT; surveys of hiring and talent; and statements by people in design firms and schools. Three roles can be read from these sources. The first is the entry point to the profession: in the UK, 57% of designers hold a degree as their highest qualification (34% across all occupations), 82% of graduates working in design hold a creative degree, and the US BLS lists a bachelor's degree as the typical entry-level education for graphic designers. The second is cultivating ways of seeing and thinking: schools say they have centered education on asking questions, reading context, and making judgments. The third is that schools have not fully supplied job-ready skills: UK employers in design-intensive industries report more often than average that recruits from schools and universities are poorly prepared, and 21% of creative-industry employers say graduates are not prepared for their role (14% across all industries). A Pentagram partner recalls that his school work was a dry technical exercise compared with real projects and that as an intern he learned a great deal by watching the people around him. This suggests a division of labor in which schools supplied ways of seeing and the entry point while workplaces supplied hands and speed, and the workplace side of that division is now thinning with generative AI. Non-degree routes (short courses, bootcamps) have grown, but published outcome figures are scarce, and dropping degree requirements from postings has barely changed actual hiring. The academic counterpart is design-education-roles-literature.
-
What Has Design Education Carried So Far? (Academic Research)
2026-10-11
The previous note (what juniors should do when generative AI takes away the entry-level work) argued that what may be lost in entry-level work is less the volume of practice than the route of watching experts' judgment up close. This note examines, through research on design education, how far school-based design education has carried that route. Research describes three roles for schools. The first is simulating professional judgment through the studio: Schön argued that the architectural studio simulates real professional action, and later research positions the studio as the signature pedagogy of design education. The second is socialization into professional identity through critique: conversations in critique build tacit knowledge, informal peer critique externalizes judgment, and critique socializes students as autonomous decision makers. The third is handing down a view of what counts as good. Research also shows what schools have not carried. The egalitarian, autonomous identity that critique cultivates is said to be closer to academic ideals than to actual workplaces; critique is described both as an apparatus of power and as an occasion of fear; and Schön's idea of reflection itself has been criticized for lacking evidence. UX students and practitioners also report that a design philosophy developed in education is hard to enact in practice. In the movement to extend design thinking into general education, implementations in engineering education remain mostly course-based, with little direct-observation assessment. Schools have simulated a place to encounter experts' judgment but carry a gap with workplace judgment, and as generative AI thins entry-level work, the place that used to fill this gap may disappear. The industry counterpart is design-education-roles-industry.
-
Novice Group Work with a Shared AI Agent: A Literature Map of CSCL and AI-Supported Collaborative Learning (2026)
2026-08-22
Best practices for introducing a single shared AI agent into group work, organized from 48 academic publications spanning CSCL foundations, the ACLS (adaptive collaborative learning support) lineage, SSRL x AI, and LLM-era empirical studies: intervention timing, addressee design, AI-centered interaction drift and its condition-dependence, and mitigation of over-reliance and free-riding.
-
Does Being Able to Build Mean Being Able to Learn? Rereading the Learning Outcomes of End-User Development Through Motive
2026-08-14
An integrative summary that rereads end-user development (EUD) along two axes — who builds and why, and what is learned there — and maps it with 185 sources. It sets out that the users EUD research assumes are 'non-programmers who are nonetheless domain experts', a different kind of learner from the novice; that far transfer from building has not been supported since 1984; that rises in self-efficacy do not correspond to actual competence; that studies measuring learning outcomes under AI assistance with delayed tests are all but absent; that civic tech brings in the condition of unpaid public activity and separates the location of domain knowledge from the builder; and that EUD research has not dealt with the most basic motive of all, building for one's own need.
-
Issues for Design Education (Not Craft Education): From the Evidence on Failure Design and Cognitive Offloading
2026-08-08
Derives seven issues that arise when this session's accumulated evidence (failure-embedded learning design and its debate, generative-AI cognitive-offloading harms, the robustness assessment of technology-education harms, GBS and scenario learning) is transferred to design education, finalized after a critical review by an academic critic on the design-education canon. The core structure: what generative AI offloads by default (alternative generation, exploration, evaluative judgment) coincides with design education's learning objectives — yet the underlying premise that craft and design judgment can be severed is itself an unresolved issue challenged by the educational canon (Dewey, Lave & Wenger). Most of the evidence is extrapolated from well-defined domains; validation in ill-structured design problems remains an open gap
-
Patterns of Technological Harm to Education, and Their Refutations: A Robustness Assessment Across 25 Works
2026-08-08
Typologized claims of technology harming education (including the Google effect and cognitive offloading) into six mechanism-based patterns and checked 12 original works against 13 peer-reviewed refutations, replications, and boundary-condition reports. Many famous harms (the Google-effect Stroop experiment, laptop note-taking harm, media-multitasking harm, the smartphone mere-presence effect, screen-time harm) have failed replication or shrunk drastically in peer-reviewed work, while classroom attention distraction (RCTs) and passive answer-providing AI remain comparatively robust. Organizes both which harms a learning service should exclude and which refuted harms it should not over-fit to
-
Does Early Use of Generative AI Inhibit the Formation of Thought? A Literature Map of Cognitive Offloading and Learning
2026-08-08
Organized the academic evidence on the claim that early generative-AI use preempts the thinking that would otherwise have emerged from learners, across 16 works: 7 experimental reports of harm, 2 correlational/self-report studies, 3 theoretical frameworks, and 3 counter/conditioning works (plus one retraction record). Harm concentrates under three conditions: answer-providing unguarded AI, novice or struggling learners, and outcomes measured as retention/transfer without AI. The apparent contradiction with positive meta-analyses dissolves as a difference in what is measured. The claim is empirically supported when restricted to early use of unguarded, answer-providing AI
-
The Debate Over Designing Failure Into Learning: Four Lineages Versus Cognitive Load Theory
2026-08-07
Based on a 27-work corpus (14 response works added), organizes the debate between the four failure-design lineages (productive failure, desirable difficulties, impasse-driven learning, error management training) and the cognitive load theory camp (Kirschner, Sweller & Clark) into five contention points (timing of instruction, the validity of the 'minimal guidance' framing, measurement of effects, prior knowledge, the role of affect), with each side's claims, empirical results, and the unresolved conflicts in a contention table
-
Designing Failure Into Learning: From Expectation Failure to Productive Failure and Industry Implementations
2026-08-07
Organized the lineage of deliberately under-instructed learning design across two streams: 13 core works from four learning-science theories (productive failure, desirable difficulties, impasse-driven learning, error management training) and 13 technical learning services whose makers explicitly state intentional difficulty, identifying the industry-side thinness of post-failure consolidation as the gap against theory
-
Scenario-Based System Design Learning: An Industry Survey in the Context of Goal-Based Scenarios
2026-08-07
Collected 28 data points on industry services that teach system design with narrative, role, and mission, and checked them against Schank's seven Goal-Based Scenario components, identifying the scarcity of design-as-operations exercises and the absence of the resources component (just-in-time expert stories)
-
Tools for Learning System Design: Industry Intelligence
2026-08-07
An industry research note organizing tools for learning correct architecture design into four types: review (missing-element detection), exercise-with-feedback, reference, and notation
-
Pre-Production Validation and Learning for System Architecture: Industry Intelligence
2026-08-07
An industry research note organizing the means of validating and learning system architecture before real operation, across standards, independent surveys, developer testimony, and labor-market data
-
Prior Research on Goal-Based Scenarios: A 31-Item Literature Map from Schank's Theory to Components, Evidence, and Design Contexts
2026-08-07
A literature map of prior research on Roger Schank's Goal-Based Scenarios (GBS), organized into two streams, general theory and design contexts (31 items; 40 rows explored, 9 merged as duplicates, 0 excluded). (1) Theoretical background: the lineage from Dynamic Memory's case-based reasoning (a memory theory in which expectation failure drives learning) as the cognitive foundation, to the formulation of GBS as an implementation theory of learning by doing in the 1994 Journal of the Learning Sciences papers. (2) Components: the seven elements (learning goals, mission, cover story, role, scenario operations, resources, feedback; Schank, Berman & Macpherson 1999) and their design procedure. (3) Technical requirements: the ILS implementation lineage of multimedia simulation environments and the GBS Builder. (4) Evidence: application cases and quasi-experiments in statistics education, computing fundamentals, elementary programming, and VR training, identifying the thinness of evidence, with controlled studies concentrated in 2020s Turkish and Taiwanese quasi-experiments and no confirmable meta-analysis. (5) Comparisons with PBL, anchored instruction, Learning by Design, and Merrill's first principles. (6) Design contexts: canonization in instructional design, scenario-centred curricula in engineering education, and PBL practice in HCI studios, while identifying the absence of published research explicitly applying GBS to design studios as a gap.
-
Is Expertise a One-Way Street? Unlearning and the Blind Spot of Reversible Mastery
2026-07-19
A note that takes one asymmetry quietly assumed by the learning sciences and expertise research, namely that development is a monotonic climb from novice to expert while unlearning and regression are degradations to be minimized, and inverts it with the same move by which de Broglie derived matter waves: completing a symmetry. The inversion says that unlearning is a developmental process symmetric to learning, and that expertise is not a one-way ladder but a reversible, frame-relative state. The Einstellung effect and design fixation are the 'light quanta,' the side where this asymmetry has already been shown to break empirically. The inversion is offered not as an established fact but as a falsifiable hypothesis: a controlled experiment can test whether deliberate unlearning interventions outperform 'learning more' for adaptation to a changed environment. Novelty is strictly limited to the ontological inversion that treats unlearning as a developmental stage symmetric to learning, and to the reversibility of expertise. 17 references (16 with confirmed bibliography, 1 pending primary verification).
-
Scaffolding Learning and the Design of Interventions: A Literature Map of Cognitive Apprenticeship, Legitimate Peripheral Participation, and Theories of Mediation
2026-07-19
A single map of the published literature on how to design interventions that support learning and the development of expertise. It runs from scaffolding and the zone of proximal development (Wood, Bruner & Ross 1976; Vygotsky 1978), through cognitive apprenticeship and fading (Collins, Brown & Newman 1989), legitimate peripheral participation (Lave & Wenger 1991), reflective practice (Schön 1983, 1987), collaborative learning and intrinsic motivation (Dillenbourg 1999; Johnson & Johnson 1989; Deci & Ryan 1985; Ryan & Deci 2000), the general theories that treat artifacts as mediators (Latour 2005; Callon 1984), to the psychometric scales and interaction analyses that measure interventions (Dennis & Vander Wal 2010; McLain 2009; Sacks, Schegloff & Jefferson 1974; Fairclough 2003). 16 references (6 verified, 10 to be confirmed).
-
Research Currents in AI, Education, and the Learning Sciences: Evidence on Generative AI and Learning (2024–2026)
2026-07-19
A lightweight scoping synthesis of 39 peer-reviewed studies at the intersection of generative AI and general education and learning sciences. Organized into four clusters: large RCTs and meta-analyses, metacognitive laziness and cognitive offloading, assessment and academic integrity, and equity and the changing role of teachers. 39 references (plus one retracted paper noted separately).
-
AI Adaptation in Design Education — The Current State and Structural Challenges of Curriculum Reform
2026-07-09
Analyzes AI-era design education curriculum reform drawing on 22 academic publications and information from 8 educational institutions plus public bodies. Three structural findings: (1) research concentrates on creativity development (35.9%), assessment (27.6%), and curriculum design (22.4%), with 66.7% not reporting outcomes (Musiienko 2026); (2) four newly required skills are identified (agency, domain knowledge, imagination, and aesthetic judgment); (3) no update to NASAD's AI standards has been confirmed. Reflective practice is being reconceived as 'learning by co-doing.'