Professional Practice in Evidence Synthesis

Professional credibility is built in the decisions other people can inspect

A title, certificate or publication can signal experience. None of them makes every future judgment correct. Professional credibility is stronger when other people can see how a question was framed, what standards governed the work, where specialist input was used, how conflicts were managed, what automation contributed, and how uncertainty was carried into the final communication.

That makes professional practice less about claiming expertise and more about making expertise auditable. A defensible evidence synthesist should be able to explain not only what was done, but why the method was appropriate, where the main uncertainty remains, what parts of the work were delegated, and which conclusions would change if key assumptions changed. The same expectation applies whether the work is academic, commissioned, used in a guideline, or prepared for health technology assessment.

Evidence synthesis supports decisions. It does not silently become the decision.

Evidence reviews often sit close to consequential choices. WHO guideline processes, NICE guidance development and HTA methods all depend on systematic evidence review, but they also separate evidence assessment from broader deliberation. GRADE Evidence-to-Decision frameworks make this separation explicit: research evidence is considered alongside other criteria that may include values, resources, equity, acceptability and feasibility.4567

The professional task is therefore to preserve the chain from evidence to interpretation. Effect estimates should not be presented as recommendations by implication. Low certainty should not be hidden inside polished prose. A commissioned review should not move a decision threshold simply because the client prefers a cleaner result. When evidence cannot answer the decision question at the requested level of confidence, saying so is part of the work.

Commissioned work changes the incentives, not the need for independence

Consulting and commissioned synthesis introduce additional pressures: fixed deliverables, contractual timelines, sponsor priorities, confidentiality requirements and requests to change scope after work has begun. None of those conditions automatically makes the work untrustworthy. They do make governance more important.

Conflict-of-interest rules are setting-specific. Cochrane, for example, applies a particularly strict policy: under its 2020 policy, at least two-thirds of the author team must be free of relevant financial conflicts, and the first and last authors must be free of such conflicts. That is a Cochrane rule, not a universal threshold for every journal, HTA body or consultancy.8 The transferable principle is transparency plus protection of methodological independence. Funding, employment and client relationships should be disclosed and managed according to the governing institution, journal, contract and law.

A professional boundary becomes necessary when a requested change would make the work misleading or methodologically indefensible. The appropriate response may be to explain the consequence, document the disagreement, revise the scope transparently, involve another expert, or decline the request. Registration and prespecification can help by preserving a visible record of what was intended before results or client preferences were known, but they do not remove the need for judgment.

Credit and accountability should follow the work

Large synthesis teams make authorship harder to interpret. CRediT offers 14 contributor roles, including methodology, formal analysis, data curation, software, supervision, validation and writing. It improves transparency about contributions, but CRediT is a contributorship taxonomy, not a substitute for a journal’s authorship criteria.9

ICMJE takes the accountability requirement further: authors must meet its authorship criteria and accept responsibility for the work. It also states that AI-assisted technologies should not be listed as authors because they cannot take responsibility for accuracy, integrity and originality. Human authors remain responsible for material produced with AI assistance and should disclose relevant use according to journal requirements.10

AI moves work. It does not move accountability to the machine.

AI is already changing evidence synthesis, but the durable professional question is not whether a particular tool is fashionable. It is whether the person responsible for the synthesis can justify the use of that tool, understand its limitations, detect consequential failure and report its role transparently.

The 2025 joint position statement from Cochrane, the Campbell Collaboration, JBI and the Collaboration for Environmental Evidence is unusually clear on this point. Evidence synthesists remain responsible for their syntheses; AI should be used with human oversight; judgment-related uses should be transparently reported; and AI should not compromise methodological rigor or research integrity.11 The associated RAISE work also emphasizes evaluation, validation, transparent limitations and context-sensitive decisions about whether a tool is suitable for a particular synthesis.12

This is deliberately different from prescribing one universal accuracy threshold. The acceptable risk of error depends on the task, the review context, the consequences of missed evidence, the availability of human checks and the evidence supporting the tool. The joint position statement explicitly frames AI use as a trade-off that may require piloting or calibration in the specific synthesis when existing evaluation evidence is insufficient.11

Teaching and supervision require more than having completed a review

Methodological experience is necessary for good supervision, but it is not identical to teaching competence. A supervisor has to surface the reasoning behind a decision, identify where a learner’s understanding breaks down, distinguish a harmless alternative from a biased method, and know when the project needs expertise outside the supervisory team. Published competency work in systematic-review services supports explicit assessment of knowledge and performance rather than assuming competence from participation alone.3

The evidence base for a single universal teaching framework in evidence synthesis is not mature enough to justify one here. That is itself useful. Professional development should avoid turning a local curriculum or credential into a global standard without validation. Teaching, mentoring and assessment are legitimate parts of professional practice, but the standards used to judge them should be named and bounded.

What is likely to remain stable

Tools will change faster than the principles around them. Living evidence systems, AI-assisted review platforms and increasingly agentic workflows are already changing how evidence work is organized. Cochrane’s scientific strategy identifies living evidence, methodological adaptation and AI integration as active areas of development.13 It would still be speculative to claim exactly which job titles or tasks will dominate five years from now.

Professional practice in evidence synthesis is therefore not a final stage that comes after methodology. It is the environment in which methodology has to survive contact with people, institutions, incentives and technology. The professional evidence synthesist does not need to know everything. They need to know what they can defend, what they cannot, who should be involved, and how to leave a transparent record of the choices that shaped the evidence.

That is also why this collection will grow by problem rather than by quota. Career pathways, competency development, consulting, teaching, evidence leadership and AI-enabled work raise different questions. Each deserves its own resource only when it solves a distinct problem that is not already owned elsewhere in the Academy. Practical tools will join the evidence-synthesis resource library ; explanatory guides will join the Academy’s methodology and decision-literacy library .