Billy·Oglesby

Journal of Health AI

The problem

A new international, peer-reviewed, gold open-access journal on the deployment, evaluation, and governance of AI across the health landscape needs editorial operations that scale without losing rigor, and it has to hold itself to the standard it publishes about. A journal about health AI that was casual about its own AI use would deserve the reviews it got.

The design

This page is written for a board director, not an engineer, because the journal is a governance exhibit as much as an editorial one.

The pipeline is a fixed sequence of manuscript states, and the automation is the platform (tracking, notifications, deadlines), not the judgment:

Journal of Health AI editorial pipeline Manuscripts flow from submission through editorial check, peer review, and decision to production and publication. Every editorial judgment (triage, reviewer selection, the final decision) is made by humans, shown in the accent color. The only AI step is the required author preflight before submission, which the author sees and answers and peer reviewers never see. Author preflight: required, AI-assisted author answers each finding · zero-data-retention · reviewers never see it Submitted Editorial check: an editor triages intake pauses at 10 untriaged submissions Under review: human reviewers single-blind · at least 3 completed reviews · reviewers may not use AI Decision: Editor in Chief only accept · revise · reject revision requested Accepted In production Published gold open access human decision point (every editorial judgment is one) planned, currently off

Three design decisions carry the governance weight:

  • Every editorial judgment is human. Triage is an editor's decision; review is single-blind with at least three completed human reviews per manuscript; the accept, revise, or reject decision belongs to the Editor in Chief alone. Intake pauses when ten submissions sit untriaged, so speed never outruns attention.
  • The AI-use policy is published, not implied. AI tools cannot be authors. Authors must disclose AI use, down to model names and versions. Reviewers may not use AI on manuscripts at all, a confidentiality line rather than a style preference.
  • AI works on the author's side, before review. Every submission and every revision goes through a required author preflight: a rule-based completeness check plus an AI review against the reporting guidelines, run on Claude through a zero-data-retention route. The author sees each finding and answers it before submitting. The preflight helps authors catch the common errors that slow review and publication. It is not part of peer review, peer reviewers never see it, and it has no role in any editorial decision. Editors see the findings and the author's responses so anything left can be fixed before publication. If the AI review is unavailable, the rule-based check still runs. The policy is public: journalofhealthai.org/policies.

The numbers

Figure Value
Reviews required per manuscript ≥ 3 completed, single-blind
Editorial decisions made by AI 0 (by policy)
Author preflight before every submission and revision required, AI-assisted
Production agents (formatting, DOI minting) 0 (human-run, by design)
Intake pause threshold 10 untriaged submissions
Submissions and days to first decision to come with the first issue

What went wrong and what I changed

  • September 17, 2026

    A malware incident reached the journal's server through a third-party analytics dependency.

    Changed: Took the server down and rebuilt it from scratch rather than cleaning it in place, and tightened what third-party code the platform is allowed to load.

What I’d do next

Publish the first articles and, with them, the journal's first public throughput numbers.