The Integration Problem

Capability Is Not Deployment, and Closing the Gap Generates New Expertise

Updated April 2026

Core Thesis

The dominant challenge in artificial intelligence is no longer building capable models. It is integrating them safely, effectively, and accountably into the structured professional practices that govern how things actually work. This integration problem is not a temporary engineering gap that scaling will close. It is a knowledge-creation frontier in its own right, one that requires deep domain expertise to navigate, produces genuinely new understanding, and will expand rather than contract the scope of meaningful human work.

Abstract

The dominant public narrative around artificial intelligence focuses on capability displacement: which human tasks will AI perform, and what will be left for people to do? This position, while understandable, obscures a deeper and more consequential challenge: the integration problem. Foundation models have achieved remarkable capability in language, vision, reasoning, and even physical control, but capability is not deployment, and deployment is not safe, effective, domain-appropriate integration into the structured practices that govern how things actually work in the real world. This essay argues that the gap between what AI can do in principle and what it can reliably do within the constraints of specific professional domains is not a temporary engineering inconvenience. It is a rich, largely unexplored research frontier that will require more human expertise, not less, and will generate new forms of knowledge, both by surfacing what was previously tacit and by producing genuinely novel understanding of how capability and practice interact. Drawing on recent developments in robotics, embodied AI, adaptive health interventions, epistemology, and philosophy, I outline why integration science is the critical intellectual challenge of the next decade, and why it represents an expansion of meaningful human work rather than its contraction.

---

1. The Capability Illusion

We are living through an extraordinary period of AI capability growth. Large language models generate fluent text, engage in complex reasoning, and write functional code. Vision-language models interpret scenes, follow instructions, and generalise across tasks they were never explicitly trained on. In robotics, systems like RT-2 [1] have shown that web-scale vision-language pre-training can transfer directly to robotic manipulation (a robot that has never seen a particular object can reason about how to grasp it because its language model has read about it). The Open X-Embodiment project [2] demonstrates that training across diverse robotic platforms produces policies that generalise better than any single-platform system. pi-zero [3] and related architectures learn dexterous manipulation from demonstrations, approaching human-level performance on specific tasks.

These results are consistent with what Rich Sutton identified in his influential 2019 essay "The Bitter Lesson" [4]: general methods that leverage computation (search and learning at scale) have consistently outperformed approaches that try to encode human knowledge directly into AI systems. The comprehensive foundation models report by Bommasani et al. [11] documents both the breadth of these capabilities and the open questions about deploying them responsibly. The history of AI, Sutton argues, is a history of researchers encoding their domain expertise into systems, only to be overtaken by simpler methods that scale with data and compute. The lesson is bitter because it means that much careful, expert knowledge-engineering work turns out to be temporary scaffolding that scaling renders obsolete.

If this trajectory continues unbroken, the implications for human expertise seem stark. If models can learn chess, protein folding, mathematical theorem-proving, and robotic manipulation without hand-coded domain knowledge, what role remains for human experts in any domain?

But this perspective contains a critical limiting assumption about real-world human expertise. It conflates AI capability on benchmarks and controlled tasks with reliable integration into the structured practices that govern real-world domains. These are categorically different problems, and the second is barely begun.

2. The Gap Between Capability and Integration

Deploying an AI system as a working component of professional practice is fundamentally different from demonstrating its capabilities in a research setting. Consider a yoga instructor coaching an elderly person through balance exercises. The instructor is not simply recognising poses and generating instructions. They are operating within a rich protocol shaped by safety norms, professional liability frameworks, individual learner histories, real-time perceptual judgments under uncertainty, and pedagogical principles about when to correct, when to encourage, when to back off, and when to refer. None of these structures exist as labelled training data. They are seldom written in textbooks, and where they are, the written form is a pale shadow of the tacit knowledge that governs actual practice.

This is not unique to yoga coaching. Medicine, law, education, engineering, social work, financial advising, crisis counselling, physiotherapy, aviation. Virtually every professional domain operates within an architecture of protocols, norms, constraints, escalation procedures, and accountability structures that have been refined over decades. These architectures are rarely formalised. They live in professional culture, in tacit expertise, in institutional memory, and in the judgment calls that practitioners make thousands of times a day without conscious deliberation.

The AI capability revolution has produced systems that can, in isolation, perform many of the component tasks within these domains. But performing a component task is not the same as integrating safely into the practice. The integration problem asks: given a capable AI system, what is the minimum viable architecture that allows it to operate safely, effectively, and accountably within a specific professional domain? What must be externally enforced? What can the model handle? How should uncertainty at the boundary between perception and action be managed? And, critically, who determines the answers to these questions, and how?

These questions span distinct sub-problems: safety constraint enforcement, epistemic (structure of knowledge) management under uncertainty (structure of knowledge), evaluation framework construction, expert feedback loop design, and accountability architecture. The difficulty of each varies by domain. A text-based legal research tool faces different integration challenges than an embodied coaching system operating under perceptual uncertainty with a vulnerable population. The argument that follows applies most forcefully to high-stakes domains involving physical action, perceptual ambiguity, or vulnerable populations, but the structural pattern (capable model, underspecified integration requirements) recurs broadly. Health informatics corroborates this: Coiera [25] has shown that "the last mile" between AI capability and clinical deployment is where most systems fail, because real-world sociotechnical complexity is systematically underestimated. Greenhalgh et al.'s NASSS framework (Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability) [26] similarly identifies how technologies that neglect the complexity of organisational, professional, and patient contexts are abandoned or produce unintended consequences.

Recent work on the evolutionary history of intelligence lends additional theoretical weight to the centrality of this problem. In recent work from Google Deepmind, Evans et. al. [31] argue that every prior "intelligence explosion" ~ from primate cognition scaling with social group size, to language enabling cumulative culture, to writing and bureaucracy externalising cognition into institutional infrastructure ~ was not an upgrade to individual cognitive hardware but the emergence of a new socially aggregated unit of cognition. Drawing on the major evolutionary transitions framework [34] and Tomasello's account of the "cultural ratchet" [33], they contend that intelligence has always been constitutively social: it advances through the complexification of the social systems in which cognitive agents are embedded, not through the scaling of individual minds. I frame it as "human intelligence is embedded in the civilisation, not in any single individual". This social perspective has a direct implication for the current transition. The intelligence explosion we are living through will not be delivered by scaling models alone. It will materialise (or fail to materialise) through the social-institutional fabric into which AI is woven.

The integration problem is not a footnote to the capability revolution; it is the mechanism through which any transformative intelligence gain is realised.

3. Why Integration Cannot Be Solved by Scaling Alone

Sutton's Bitter Lesson is a powerful empirical observation about capability development. But it does not straightforwardly extend to integration, for a set of reasons.

First, integration requires domain-specific safety properties that are not optimised by any standard task-level training objective. A language model trained to generate fluent coaching instructions has no loss function for "don't suggest a weight-bearing transition to someone who reported knee pain three sessions ago (in a yoga coaching scenario)". These constraints emerge from professional practice, clinical evidence, and ethical norms. They can, in principle, be learned from data, but the data would need to include the full context of professional judgment, including the cases where experts didn't act, the reasons they escalated, and the uncertainty that drove their decisions. Such data rarely exists in labelled, structured form. Much of it resides in what Polanyi [19] termed tacit knowledge: expertise that practitioners possess but cannot fully articulate, and that resists codification into training datasets. Dreyfus and Dreyfus [27] reinforced this insight, showing that expert practitioners operate through intuitive, context-sensitive judgment that resists decomposition into rules, a finding with direct implications for what can and cannot be learned from labelled data alone.

Second, integration requires epistemic management: the system's capacity to know what it doesn't know and to act on that meta-knowledge. In controlled research environments, models operate on clean inputs with well-defined task boundaries. In deployment, inputs are noisy, contexts are ambiguous, and the cost of confident-but-wrong action can be severe. A AI yoga coaching system that confidently corrects a pose it has misperceived due to camera occlusion is more dangerous than one that does nothing. The critical safety mechanism is not better perception; rather, it is the capacity to recognise insufficient perception and to choose information-gathering (diagnosis) over intervention (correction). This is a form of metacognition that current training paradigms do not reliably produce. As Guo et al. [20] have shown, modern neural networks are systematically miscalibrated, producing confident predictions even when wrong. Standard training rewards correct answers, not calibrated uncertainty. Recent work by Asadi et al. [29] reveals a more troubling variant of this problem. When multimodal AI models are asked about images that were never provided, they do not report missing input. Instead, they construct detailed descriptions of non-existent images, generate elaborate reasoning traces grounded in those fabrications, and produce confident conclusions. A phenomenon the authors term mirage reasoning. Unlike hallucination, which fills in details within a valid perceptual frame, mirage reasoning fabricates the perceptual frame itself (the models simulate). Critically, the resulting outputs are indistinguishable from those produced with real images, the effect is most pronounced in medical domains (where mirage-based diagnoses are biased toward pathology), and it is not reduced by scaling; newer, more capable models exhibit equal or higher mirage rates. This has a direct implication for integration: the failure mode that matters most in safety-critical deployment is not a model that gets the wrong answer, but a model that constructs a plausible simulation of understanding and cannot be distinguished from one that actually understands. Detecting and governing this behaviour requires domain experts who understand not just their professional practice but the specific epistemic failure modes of the models operating within it. This is a form of expertise that did not exist before the integration problem created it.

Third, integration requires accountability structures: traceable decision chains that allow failures to be diagnosed, attributed, and corrected. When a purely data-driven system makes an error, the space of possible causes is vast: training data, model architecture, prompt design, input distribution shift, or emergent behaviour. Professional domains require the ability to say why a decision was made and which component is responsible when something goes wrong. This is not a machine-learning problem. It is an architecture and governance problem, and it requires structured design that scaling does not automatically provide.

Recent findings in AI reasoning research offer additional supportive context. Kim et al. [32] demonstrate that frontier reasoning models such as DeepSeek-R1 and QwQ-32B do not improve simply by allocating more computation to longer chains of thought. Instead, they spontaneously generate complex, multi-perspectival internal debates (what Evans et al. [31] term "societies of thought") in which distinct cognitive perspectives argue, question, verify, and reconcile within the model's reasoning trace. This behaviour was not explicitly trained; it emerged under reinforcement learning pressure for accuracy alone. This findings, Evans et al. [31], treat as evidence that intelligence is constitutively social even within a single computational process. But from the standpoint of the integration problem, the society-of-thought phenomenon adds additional dimensions to the challenge rather than resolving it. A model that reasons through richer internal deliberation still has no loss function for domain-specific safety, no reliable mechanism for calibrated abstention under perceptual uncertainty, and no accountability trace that a professional governance framework can audit. What it does have is a more complex decision surface—one that domain experts must now evaluate, constrain, and govern. Richer internal reasoning generates more integration surface area, not less.

Human-feedback-based approaches such as RLHF (discussed in §6) can in principle address some of these requirements. But this reinforces rather than undermines the point: such approaches are not autonomous scaling. They require domain experts to design feedback, evaluate outputs, and adjudicate disagreements. Evans et al. [31] arrive at a compatible conclusion from a different direction: they argue that RLHF, as a fundamentally dyadic correction model, cannot scale to the billions of interacting agents their framework envisions, and that what is needed instead is institutional alignment. Specifically, role-based governance structures modelled on courtrooms, markets, and constitutional checks and balances. The parallel to the integration problem is direct. The professional practice architectures described throughout this essay (protocols, escalation procedures, accountability chains, expert feedback loops) are the types of institutional structures that any viable alignment framework must either replicate or engage with. The integration problem is not that AI systems can never learn these properties, but that learning them requires sustained human expertise of the kind the displacement narrative treats as expendable.

Emerging evidence from research I am doing on the potential of AI in the domain of advance care planning (ACP) illustrates this pattern [24, 30]. ACP is a deeply relational practice requiring genuine presence, adaptive dialogue, and trust-building. Aged care staff describe it as a dynamic, iterative process in which clinical knowledge and intuitive judgment must be continuously adapted to evolving patient needs, cultural contexts, and family dynamics [30]. Yet experienced health professionals report that institutional compliance pressures have already reduced it to procedural form-filling, displacing the relational expertise that makes it meaningful. This procedural drift, the systematic displacement of professional judgment by standardised process, demonstrates that integration failure is not a hypothetical risk of future AI deployment. It is an existing pattern that AI, applied without integration science, would accelerate.

4. The Discovery Frontier: Building the Evaluations We Don't Yet Have

Perhaps the most under-appreciated dimension of the integration problem is this: for most professional domains, we do not even have the evaluation frameworks needed to know whether integration is working.

The domains where AI has been most successfully deployed (game-playing, protein folding, code generation) share a crucial property: they have rich, well-defined evaluation criteria. Even these domains face non-trivial integration challenges in practice: AlphaFold's [22] integration into drug discovery requires substantial expert interpretation and validation, and code-generation tools raise questions of liability, correctness verification, and professional trust. But their evaluation criteria provide a foundation that most professional domains lack entirely. A chess move is good or bad relative to game outcome. A protein structure prediction is close to or far from experimental ground truth. Code compiles or it doesn't; tests pass or they fail.

Most professional domains have nothing comparable. What constitutes "good yoga coaching"? The answer involves biomechanical correctness, pedagogical timing, motivational appropriateness, safety constraint satisfaction, and adaptation to individual capacity. Furthermore, experts disagree about the relative weighting of these dimensions. There is no single ground truth. The evaluation framework itself must be constructed through careful collaboration between AI researchers and domain experts, and the act of constructing it generates new knowledge about the domain.

This observation has antecedents. Polanyi [19] established that much professional knowledge is tacit, and Nonaka and Takeuchi [21] showed that organisational knowledge creation depends on surfacing such tacit understanding. What is distinctive in the AI integration case is the granularity required. Methodologies are beginning to emerge for this: Sellars et al. [30] are using causal mapping and narrative techniques to elicit the tacit knowledge that aged care staff draw on during advance care planning, with the explicit aim of informing AI system design. The effort to integrate AI into professional practice does not merely apply existing domain knowledge. It surfaces knowledge that was previously tacit, unarticulated, or contested at a level of precision that general formalisation efforts rarely demand. When you try to formalise what constitutes safe yoga feedback for an elderly person, you discover that experts disagree about threshold conditions, that some safety rules are absolute while others are context-dependent, and that the notion of "adequate confidence" for a correction varies with the severity of the potential harm. This disagreement is not a problem to be resolved; it is data about the structure of professional judgment that did not exist in explicit form before the integration effort forced it to the surface.

The development of useful evaluation frameworks for existing human-oriented, protocol-driven workflows will, in many domains, require exactly this kind of deep discovery. It is intellectual work of the highest order, work that demands domain expertise, epistemological sophistication, and sustained collaboration between people who understand the practice and people who understand the technology.

5. Knowledge as Information with Causal Power

If integration work generates new knowledge, what kind of knowledge is it, and why should it be valued as genuine intellectual contribution rather than routine engineering? David Deutsch's epistemological framework offers a supportive lens [5]. For Deutsch, knowledge is not merely true belief or justified information. Rather, knowledge is information that has causal power in the world. A piece of knowledge is information that, once it exists, tends to cause things to happen: it shapes decisions, constrains designs, redirects behaviour. It persists because it is useful, and it has reach. Good knowledge applies beyond its original context.

Through this lens, the integration problem is not a diminished engineering exercise subordinate to the grand challenge of building capable models. It is a knowledge-creation frontier in its own right. Consider what a systematic investigation of the minimum viable safety architecture for embodied AI yoga coaching produces: an empirical map of which safety properties models can learn versus which require external enforcement. This is information with enormous causal power. It directly determines how anyone building such a system should allocate their engineering effort. It has reach, because the same structural question (what must be external?) arises in any domain where AI acts on uncertain perceptual input with real-world consequences.

Or consider the epistemic management principle, as a testable hypothesis: that the critical safety mechanism in embodied coaching is the system's capacity to diagnose before correcting, to abstain when uncertain, and to escalate when out of its depth. This is knowledge with causal power across every domain where AI acts under perceptual uncertainty: rehabilitation robotics, surgical assistance, autonomous vehicles, industrial co-bots, crisis response. Once you know that the dominant failure mode is confident-but-wrong intervention, and that the mechanism preventing it is explicit metacognition rather than improved sensing, you build systems differently. That insight, once demonstrated, cannot be unlearned.

This is what Deutsch means by knowledge with reach. It doesn't matter that it was discovered in the context of yoga coaching for older adults. The principle transfers because it addresses a structural feature of the relationship between uncertain perception and consequential action, a feature that recurs across domains.

The evolutionary perspective reinforces this point. Tomasello [33] identified a mechanism he calls the "cultural ratchet": the capacity of human language and social learning to accumulate knowledge across generations without any individual needing to reconstruct the whole. Each generation inherits the cognitive tools of its predecessors, adds to them, and passes the augmented toolkit forward. Writing, law, and bureaucracy extended this ratchet by externalising social intelligence into persistent institutional infrastructure [31]. The integration frameworks described in this essay (boundary characterisation maps, epistemic management principles, evaluation methodologies, feedback loop designs) are, in this sense, new entries in the cultural ratchet. They are institutional knowledge about how to weave AI safely into human practice: knowledge that, once produced, persists and transfers independently of the individuals who produced it. Integration science does not merely solve local engineering problems. It generates the durable, transferable institutional knowledge through which the current intelligence transition accumulates its gains.

6. The RLHF Parallel: Human Feedback as the Paradigm, Not the Exception

If integration produces knowledge with reach, the next question is practical: through what mechanism does human expertise actually enter AI systems? §3 noted that human-feedback-based training approaches represent part of the solution to the integration problem rather than an exception to it. The success of reinforcement learning from human feedback (RLHF) [6] in aligning language models illustrates why. RLHF works not because the model learns a fixed set of rules, but because human judgment continuously shapes the model's behaviour through preference signals. The human expert doesn't write a formal specification of "helpful, harmless, and honest." Instead, they provide examples, corrections, and comparative judgments that the model learns to approximate.

Recent robotics work extends this paradigm. DAgger-style interactive learning [7] has experts correcting robot behaviour in real time, gradually transferring skill through demonstration rather than programming. Learning from demonstrations, as the ALOHA [8] architecture shows, uses human behaviour as seed data that scales through model generalisation. SayCan [9] takes a complementary approach, combining large language model reasoning with learned affordance functions so that a robot selects actions that are both task-relevant and physically feasible. Code as Policies [10] translates natural-language instructions into executable robot control code, bridging reasoning and action without hand-coded control logic.

The key insight is that human expertise enters these systems not as formal rules to be compiled but as feedback that shapes behaviour; and this is exactly the pattern that professional domains will require. Experts in yoga, physiotherapy, clinical medicine, or any other professional domain don't need to write constraint specifications. They need to watch system outputs, flag problems, approve or adjust decisions, and provide the structured feedback that allows the system to learn the boundaries of safe practice. This is skilled, intellectually demanding work that amplifies rather than replaces expertise.

The integration problem, seen through this lens, is not about encoding human knowledge into AI systems (which Sutton's Bitter Lesson warns against) but about establishing the feedback loops through which human expertise shapes AI behaviour in specific domains. Designing those feedback loops, determining what experts should evaluate, how disagreement should be handled, how feedback should be structured to be learnable, and how the resulting system should be evaluated is a new form of professional practice that does not yet exist at scale.

7. The Fear is Misplaced; The Opportunity is Immense

Return to the displacement concern introduced at the outset. If AI capability is treated as a direct substitute for human labour, the logical conclusion is that more capability means less human relevance. But the integration problem inverts this logic. Greater AI capability creates more integration work, not less, because each new capability opens a new domain where the questions of safety, reliability, epistemic management, evaluation, accountability, and expert feedback must be answered. Consider how the emergence of capable vision-language models immediately raised integration questions in radiology, pathology, and dermatology that did not exist when those models could not interpret medical images. Each capability advance creates new integration surface area.

The same logic applies to advances in AI reasoning architecture. When reasoning models develop richer internal deliberation—spontaneously generating the multi-perspectival "societies of thought" that Kim et al. [32] document, the integration problem does not contract. Rather, it expands, because a model whose internal reasoning is more complex presents a more demanding evaluation and governance challenge for the professional domains in which it is deployed. The integration surface area grows with capability along every dimension, including the dimension of reasoning sophistication.

An important counter-argument deserves comment: as models grow more capable, might they also become better at self-integration, reasoning about safety constraints, calibrating their own uncertainty, and generating accountability traces without human guidance? This is possible in principle. But it introduces a meta-problem: someone must verify that the model's safety reasoning is correct, that its self-calibration is reliable, and that its accountability traces are meaningful. That verification requires exactly the domain expertise the displacement narrative treats as expendable. The integration problem does not disappear when models become more capable; it shifts to a higher level of abstraction, from "design the constraints" to "verify the model's own constraint reasoning."

The risk of not doing this work is equally instructive. In domains like advance care planning, experienced clinicians warn that AI handling preparatory or documentation tasks could erode the relational competencies (death literacy, emotional attunement, cultural sensitivity) needed when a human conversation becomes unavoidable [24]. This is not speculative: Parasuraman and Manzey [28] demonstrated that sustained reliance on automated systems systematically erodes the very skills needed when automation fails or is absent. Integration done without attention to expertise preservation does not merely fail to generate new knowledge; it actively destroys existing expertise.

The pattern holds even in disciplines where workflows are mostly computational. In materials informatics, machine learning models for predicting alloy properties have grown steadily more capable, yet multi-objective optimisation of real engineering materials remains stalled. The reason is structural: experimental databases have been built in isolation, each covering a single property domain (strength, corrosion resistance, fatigue life) with its own units, metadata conventions, and compositional descriptors. Fragmented descriptors, inconsistent metadata, and missing cross-property values prevent models from learning the coupled relationships that multi-objective design requires. In recent work on multi-principal element alloys [35, 36], we found that assembling a unified database through harmonisation, imputation, and generative augmentation, without acquiring a single new experimental data point, was sufficient to enable simultaneous optimisation across seven material properties, with R² improvements of up to +0.31 compared to single-domain training. The bottleneck was the structure of the data, not the capability of the models. Critically, the integration work itself was expert knowledge work throughout. Domain scientists had to decide which literature entries were comparable, map inconsistent terminology across sources (the same precipitate phase labelled three different ways in three databases), select the physical descriptors that govern corrosion behaviour, and judge whether synthetically generated compositions were metallurgically plausible. Each of these steps forced tacit understanding about how materials data is generated and what it means into explicit, operational form. This is the integration problem operating at the data layer: the human expertise required to make the data usable was greater than the expertise required to build the models that consumed it.

The same pattern appears in clinical neuroscience. A review of publicly available EEG datasets for seizure detection and prediction [37] found ten datasets in common use, each with different channel configurations, signal types, temporal structures, sampling frequencies, and preprocessing conventions. Seventeen characteristics differentiated the datasets; models trained on one could not reliably generalise to others. Determining which structural differences were clinically meaningful, as opposed to merely arbitrary, required sustained collaboration between neurologists and machine learning researchers. The integration work was again the work that mattered: characterising incompatibilities, identifying what any future standardised format would need to preserve, and distinguishing the conventions that carry clinical significance from those that are historical accidents.

The pattern recurs further. In surface mining, a survey of 208 studies applying machine learning to drill-and-blast optimisation [38] found that only 57% reported their data preparation steps, inconsistent units and feature definitions were pervasive, and 37% of studies excluded uncontrollable geological factors despite their known importance. Domain expertise was required simply to classify which variables are controllable and which are not: a taxonomic judgment that no model can make. And in software engineering itself, an empirical study of issues in open-source ML projects [39] found that ML-specific bugs take a median of 14 days longer to resolve than traditional software issues, require collaboration across data scientists, software engineers, and domain experts, and resist standard verification because the learned behaviour of the model cannot be directly inspected. The integration problem, in other words, operates at every layer: data infrastructure, domain modelling, and the software systems that implement ML themselves.

Consider the landscape of professional domains that are candidates for AI integration: clinical medicine, education, social care, environmental monitoring, legal practice, engineering, counselling, agriculture, disaster response. In each of these domains, the same pattern holds: AI models can perform impressive component tasks, but the structured practices that govern safe, effective, accountable deployment are largely uncharted. The evaluation frameworks do not exist. The safety architectures have not been investigated. The epistemic management strategies have not been identified. The feedback loops have not been designed.

This is not a problem that will be solved in a few years of engineering. Given the number of professional domains involved and the depth of investigation each requires, this is a research frontier likely spanning decades, one that will engage domain experts, AI researchers, ethicists, policy-makers, and practitioners in sustained collaboration. It will produce new forms of knowledge (empirical boundary maps, epistemic management principles, evaluation methodologies, feedback loop designs) that have causal power across domains. And it will create new forms of professional practice centred not on performing tasks that AI can also perform, but on establishing the conditions under which AI can perform those tasks safely, effectively, and accountably.

8. A Research Agenda for Integration Science

What would a research programme for integration science look like? Several directions open up.

Boundary characterisation studies. For each candidate domain, investigate which safety-relevant properties foundation models can internalise through scaling and which require external enforcement. This requires building experimental apparatus (constraint architectures that can be ablated) and running controlled comparisons. The output is an empirical map that directly informs engineering decisions.

Data infrastructure as integration work. As §7 illustrates across materials science, clinical neuroscience, surface mining, and ML software engineering [35–39], making fragmented data jointly usable is itself expert knowledge work that surfaces tacit domain understanding. Promising methods are emerging: shared metadata schemas and ontologies for harmonisation (e.g., OPTIMADE [40], EMMO [41]), mechanism-aware imputation (MissForest, variational and adversarial generative models) for structured missingness, LLM-assisted extraction of qualitative data from tables and text with human-in-the-loop validation, and expert-driven taxonomic classification of domain variables as a prerequisite to any modelling. A systematic comparative programme across data-intensive fields would identify shared principles for when and how data integration generates new domain knowledge rather than merely cleaning inputs.

Epistemic management architectures. Develop and test architectures for managing uncertainty at the perception-action boundary in safety-critical domains. The core question (how should a system behave when it doesn't know enough to act?) recurs across every domain where AI operates on uncertain real-world input. Calibrated abstention, diagnostic reasoning, and gated escalation are candidate mechanisms that need empirical evaluation.

Evaluation framework construction. Develop methodologies for constructing evaluation frameworks in domains where no ground truth exists. Design Science Research [14] provides one methodological backbone, iterating between artifact design and evaluation, but the deeper challenge is surfacing tacit expertise and structuring professional disagreement. Research on feedback [15, 16] shows that even the concept of "effective feedback" varies by context, timing, and learner state. The difficulty is compounded by the fact that the explainability tools intended to support evaluation are themselves formally ungrounded: Haufe et al. [42] show that popular feature attribution methods (SHAP, LIME, integrated gradients, and others) systematically assign importance to suppressor variables with no statistical association to the prediction target, because the field lacks use-case-dependent formal criteria of explanation correctness. If the instruments for interpreting models are unreliable, evaluation frameworks for deployed systems cannot be assembled from off-the-shelf XAI; they must be constructed with domain-specific correctness criteria defined through expert collaboration. The resulting evaluation frameworks are themselves intellectual contributions with reach beyond their domain of origin.

Expert feedback loop design. Investigate how domain experts can most effectively shape AI system behaviour through structured feedback: adjudication interfaces, preference signals, comparative evaluation, and disagreement documentation. The goal is to make expert knowledge integration scalable, sustainable, and learnable, following the RLHF paradigm but adapted to domain-specific constraints.

Causal evaluation of intervention components. Use micro-randomised trials [13] and related causal inference methods [17] to identify which components of AI-integrated practice actually drive outcomes. The just-in-time adaptive intervention (JITAI) framework [12] provides a natural structure for this: decision points, tailoring variables, intervention options, and decision rules. This is essential for moving beyond "AI system X works" to understanding why it works and which elements are doing the work. Recent work demonstrates the feasibility of this approach: Newby et al. [23] used an AI-enhanced response-adaptive trial with a contextual multi-armed bandit algorithm to evaluate competing brief digital interventions for psychological distress, showing that different interventions were most effective for different severity subgroups. This is a direct instance of using adaptive experimental methods to identify which components drive outcomes in specific populations. Reporting standards such as SPIRIT-AI [18] provide the transparency infrastructure for such evaluations.

Cross-domain transfer of integration principles. Test whether integration principles discovered in one domain (e.g., embodied coaching) transfer to others (e.g., rehabilitation robotics, clinical decision support). If epistemic management is a general design primitive for safe AI deployment, this should be demonstrable across domains.

9. Conclusion: The Work Has Barely Begun

The most important thing to understand about the current moment in AI is not that models are becoming more capable (they are, and will continue to be). The most important thing is that the gap between capability and safe, effective, accountable integration into professional practice is vast, largely unexplored, and intellectually rich. Closing that gap is not routine engineering. It is a knowledge-creation frontier that requires deep domain expertise, new evaluation methodologies, empirical investigation of safety architectures, and sustained collaboration between people who understand the technology and people who understand the domains it will enter.

The fear that AI will render human expertise obsolete mistakes a part of the problem for the whole. Models can increasingly perform component tasks. But the architecture of safe integration (knowing what to enforce externally, how to manage epistemic uncertainty, how to evaluate success, how to structure expert feedback, and how to ensure accountability) is fundamentally a human-knowledge problem, one that mobilises clinical expertise, institutional knowledge, and the meta-expertise required to articulate what practitioners know tacitly. What "safe" means varies across domains (physical harm in rehabilitation, fiduciary breach in finance, developmental risk in education), but the structural requirement for domain-grounded integration work is constant. And it is a problem whose solution creates new knowledge with reach and causal power, in Deutsch's sense: knowledge that, once it exists, changes how systems are built and how practices are structured across domains.

If the history of intelligence is, as the evolutionary account suggests [31, 34], a history of new forms of social organisation rather than individual cognitive upgrades, then the current transition will be no different. The intelligence explosion (as it comes) will not be delivered by a single colossal model. It will be built, domain by domain, through the institutional knowledge that integration science produces: the boundary maps, the epistemic management principles, the evaluation frameworks, the feedback loop designs, the accountability architectures. These are the social-institutional structures through which AI capability becomes AI intelligence in practice. They are the cultural ratchet for the age of artificial agents.

We are not at the end of the story of human expertise in an age of AI. We are at the beginning of a new chapter, in which the central intellectual challenge is not building capable models (the scaling paradigm has shown how to do that). It is understanding, domain by domain, how those models can be woven safely into the structured practices that govern human life. That work has barely begun, and it will require everything we know.

---

References

[1] Brohan A, et al. RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control. arXiv:2307.15818. 2023.

[2] Open X-Embodiment Collaboration. Open X-Embodiment: Robotic Learning Datasets and RT-X Models. arXiv:2310.08864. 2023.

[3] Black K, et al. pi-zero: A Vision-Language-Action Flow Model for General Robot Control. arXiv:2410.24164. 2024.

[4] Sutton R. The Bitter Lesson. 2019. http://www.incompleteideas.net/IncIdeas/BitterLesson.html

[5] Deutsch D. The Beginning of Infinity: Explanations that Transform the World. Allen Lane. 2011.

[6] Ouyang L, et al. Training language models to follow instructions with human feedback. NeurIPS. 2022.

[7] Ross S, Gordon GJ, Bagnell JA. A reduction of imitation learning and structured prediction to no-regret online learning. AISTATS. 2011.

[8] Zhao TZ, et al. Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware. RSS. 2023.

[9] Ahn M, et al. Do As I Can, Not As I Say: Grounding Language in Robotic Affordances. CoRL. 2022.

[10] Liang J, et al. Code as Policies: Language Model Programs for Embodied Control. ICRA. 2023.

[11] Bommasani R, et al. On the Opportunities and Risks of Foundation Models. arXiv:2108.07258. 2021.

[12] Nahum-Shani I, et al. Just-in-Time Adaptive Interventions (JITAIs) in Mobile Health. Ann Behav Med. 2018;52(6):446–462.

[13] Klasnja P, et al. Microrandomized trials: An experimental design for developing just-in-time adaptive interventions. Health Psychol. 2015;34S:1220–1228.

[14] Hevner AR, et al. Design Science in Information Systems Research. MIS Quarterly. 2004;28(1):75–106.

[15] Hattie J, Timperley H. The Power of Feedback. Rev Educ Res. 2007;77(1):81–112.

[16] Shute VJ. Focus on Formative Feedback. Rev Educ Res. 2008;78(1):153–189.

[17] Boruvka A, et al. Assessing Time-Varying Causal Effect Moderation in Mobile Health. JASA. 2018;113(523):1112–1121.

[18] Cruz Rivera S, et al. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Nat Med. 2020;26:1351–1363.

[19] Polanyi M. The Tacit Dimension. Doubleday. 1966.

[20] Guo C, et al. On Calibration of Modern Neural Networks. ICML. 2017.

[21] Nonaka I, Takeuchi H. The Knowledge-Creating Company: How Japanese Companies Create the Dynamics of Innovation. Oxford University Press. 1995.

[22] Jumper J, et al. Highly accurate protein structure prediction with AlphaFold. Nature. 2021;596:583–589.

[23] Newby J, et al. Brief Digital Interventions for Psychological Distress: An AI-Enhanced Response-Adaptive Randomized Clinical Trial. JAMA Netw Open. 2025;8(10):e2540502.

[24] Sellars M, Vasa R, Amadoru S, Hutchinson AM. Australian health professionals' perspectives on digital tools to enhance advance care planning. Under review. 2026.

[25] Coiera E. The Last Mile: Where Artificial Intelligence Meets Reality. J Med Internet Res. 2019;21(11):e16323.

[26] Greenhalgh T, et al. Beyond Adoption: A New Framework for Theorizing and Evaluating Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care Technologies. J Med Internet Res. 2017;19(11):e367.

[27] Dreyfus HL, Dreyfus SE. Mind Over Machine: The Power of Human Intuition and Expertise in the Era of the Computer. Free Press. 1986.

[28] Parasuraman R, Manzey DH. Complacency and bias in human use of automation: an attentional integration. Hum Factors. 2010;52(3):381-410.

[29] Asadi M, O'Sullivan JW, Cao F, Nedaee T, Fardi K, Li F, Adeli E, Ashley E. Mirage: The Illusion of Visual Understanding. arXiv:2603.21687. 2026.

[30] Sellars M, Hutchinson A, Vasa R. The Role of Tacit Knowledge in Advance Care Planning: Aged Care Staff Perspectives. BMJ Support Palliat Care. 2025;15(Suppl 3):A23.

[31] Evans J, Bratton B, Agüera y Arcas B. Agentic AI and the next intelligence explosion. arXiv:2603.20639. 2026.

[32] Kim J, Lai S, Scherrer N, Agüera y Arcas B, Evans J. Reasoning Models Generate Societies of Thought. arXiv:2601.10825. 2026.

[33] Tomasello M. The Cultural Origins of Human Cognition. Harvard University Press. 1999.

[34] Smith JM, Szathmáry E. The Major Transitions in Evolution. Oxford University Press. 1997.

[35] Ghorbani M, Li Z, Pasquini A, Vasa R, Birbilis N. Supervised machine learning for corrosion assessment of multi-principal element alloys using experimental and generative datasets. npj Materials Degradation. 2025;9:155. doi:10.1038/s41529-025-00700-9.

[36] Ghorbani M, Nash WT, Tawfik SA, Vasa R, Birbilis N. Why Progress in Multi-Objective Materials Design via Machine Learning Depends on Integrated Data Infrastructure. Under review. 2026.

[37] Wong S, Simmons A, Rivera-Villicana J, Barnett S, Sivathamboo S, Perucca P, Ge Z, Kwan P, Kuhlmann L, Vasa R, Mouzakis K, O'Brien TJ. EEG datasets for seizure detection and prediction: A review. Epilepsia Open. 2023;8:252–267. doi:10.1002/epi4.12704.

[38] Munagala V, Thudumu S, Logothetis I, Bhandari S, Vasa R, Mouzakis K. A comprehensive survey on machine learning applications for drilling and blasting in surface mining. Machine Learning with Applications. 2024;15:100517. doi:10.1016/j.mlwa.2023.100517.

[39] Lai TD, Simmons A, Barnett S, Schneider JG, Vasa R. Comparative analysis of real issues in open-source machine learning projects. Empirical Software Engineering. 2024;29:60. doi:10.1007/s10664-024-10467-3.

[40] Andersen, C.W., Armiento, R., Blokhin, E. et al. OPTIMADE, an API for exchanging materials data. Sci Data 8, 217 (2021). https://doi.org/10.1038/s41597-021-00974-z

[41] A. De Baas et al., "Review and Alignment of Domain-Level Ontologies for Materials Science," in IEEE Access, vol. 11, pp. 120372-120401, 2023, doi: 10.1109/ACCESS.2023.3327725.

[42] Haufe S, Wilming R, Clark B, Zhumagambetov R, Boubekki A, Martin J, Panknin D. Explainable AI needs formalization. npj Artificial Intelligence. 2026;2:42. doi:10.1038/s44387-026-00095-1.