Revisaurus is a portmanteau of 'Revise' and 'Thesaurus'. I chose this name because biomedical science is full of activities that require evaluation, revision, and iteration to improve messaging (i.e., word choice). Whether you are applying to graduate school, applying for a grant or fellowship, writing a manuscript, or preparing an abstract for a scientific meeting, it is formulaic. A first draft is prepared, it is shared with others for internal review and revision, additional drafts are prepared and improved iteratively, it is submitted, and the submission is reviewed by one or more scientists according to a pre-defined rubric. In some cases, the scientific panel's decision is final, while in others there is an opportunity to revise and respond to the panel.
Frontier LLMs excel at formulaic tasks which is one reason they are so effective at writing computer code. I have learned that these tools also excel at evaluating and improving scientific communication because the process of writing, evaluation, and revision is also highly structured. Revisaurus contains a collection of tools, agents, and ideas that I think improve scientific communication by simulating evaluations, and then working backwards to revise the writing to address weaknesses identified by the simulated evaluators. Think of the simulated evaluators as a manuscript editor and assigned reviewers, a graduate school admissions committee, or an NIH grant review panel.
I've developed these approaches using a paid Claude plan. I suspect they will work as well with OpenAI/ChatGPT or other frontier models. If there is personal data that should not be shared with these models, modern local LLMs can likely give usable results, though they are not quite as capable as the best frontier models. Even when the models are instructed to not use queries for training, I suggest not using any sort of sensitive or restricted-access data.
Let's start with a simple example. I am a faculty trainer for multiple graduate student training programs at UW-Madison. One of these programs is the Cellular and Molecular Biology program. Usefully, they provide a public rubric for how their candidates are evaluated. The Personal Statement from the applicant is part of all four scorable categories. Improvig how the Personal Statement is received by the admissions committee is going to be an important determinant in how a student's application is reviewed.
The Revisaurus approach is to simulate the review. We know from the rubric "how" applications are reviewed. Next we need to determine "who" is going to review the applications. Though CMB's admissions committee composition isn't public, UW-Madison publishes Admissions Committeees Best Practices. It says that "each application to be read by at least two members of the committee" and that "Admissions committees may be composed of faculty, staff, and current graduate students". With this information, we can begin.
One of the most powerful aspects of current (April 2026) LLMs is the ability to derive reusable personas that simulate a person with a specific skillset and viewpoint. These personas can be created by the LLMs themselves. In this example, we want to simulate an Admissions Committee. So I will prompt Claude (Opus 4.7 Adaptive, current model as of April 2026) to create a virtual Admissions Committee with the information that we have:
You are going to help me create agents that will simulate the Admissions Committee of the Cellular and Molecular Biology Ph.D. program at the University of Wisconsin-Madison. General information on UW-Madison Admissions Committees is available from https://kb.wisc.edu/grad/131751. The CMB program faculty trainers and graduate program information is available at https://cmb.wisc.edu. Admissions information on the program is at https://cmb.wisc.edu/admissions/. The rubric used by committee members is available at: https://cmb.wisc.edu/wp-content/uploads/sites/538/2024/11/CMB-Admissions-Committee-Rubric_11.8.24.pdf After we create this simulated Admissions Committee, we are going to have it evaluate prospective student Personal Statements and Research Statements and provide a full evaluation against the rubric (though we won't have other evidence like CVs and letters of recommendation). Unlike most Admissions Committees, this Committee will provide detailed feedback to the applicant justifying the scores in each rubric category from the perspective of each reviewer, and how the committee as a whole integrated this information to make a determination on the applicant. Create all of the agents in downloadable Markdown files.
Note that Opus 4.7 couldn't read the PDF of the rubric so I pasted it in as a screenshot in a subsequent prompt Claude responded by creating (summoning?) these personas (descriptions from Claude):
| File | Role |
|---|---|
| 00_README_Orchestrator | How to run the committee; sequence, ethical guardrails, evidence-limitation notes |
| 07_Rubric_Reference | Load this first. Faithful reproduction of the CMB rubric (1.0–4.0 scale, 0.5 increments, anchor language, "Most Important" weighting, 4.0 reserved for self-driven evidence, 3.0 caps on categories 3 & 4) |
| 01_Committee_Chair | Opens the review, writes the final summary memo and recommendation |
| 02_Faculty_Reviewer_Alder | Molecular/biochemical lens — rigor-focused; sharpest on Engagement in Research |
| 03_Faculty_Reviewer_Bemis | Developmental/cell biology lens — holistic/mentor-minded; sharpest on Resilience |
| 04_Faculty_Reviewer_Chen | Microbes/virology/systems lens — fit- and specificity-focused; sharpest on Readiness and Why UW-Madison |
| 05_Student_Reviewer_Delgado | 5th-year Ph.D. student / DEIC member — full voting member; community/wellness lens, equity check |
| 06_Deliberation_Synthesizer | Reconciles the four scorecards; surfaces disagreements; weights by which reviewer's lens is sharpest on each criterion |
| 08_Applicant_Feedback_Writer | Produces the candidate-facing letter with reviewer attributions — the distinguishing feature of this committee |
The personas
Claude also provides a suggested workflow to run a review:
- Paste the Rubric Reference (07) into the conversation as context.
- Paste the applicant's Personal Statement and Research Statement.
- Run each of the four reviewers (02, 03, 04, 05) independently — ideally in separate conversations so they don't see each other's scores.
- Run the Synthesizer (06) with all four scorecards pasted in.
- Run the Chair (01) with the Synthesizer's output to produce the committee summary memo.
- Run the Feedback Writer (08) with everything to produce the applicant-facing letter.
Tim Locksmith is a first-year Ph.D. student that I mentor along with Shelby O'Connor. Tim has kindly provided the Personal Statement to use as an example. Tim is a student in the Cellular and Molecular Pathology program and did not apply to the CMB program. How would the simulated CMB committee evaluate Tim? And critically, what do the personas think he could do to improve his personal statement to make it more competitive?
I started a new conversation with Claude Opus 4.7 Adaptive with this prompt and all of the personas and the personal statement attached:
You are simulating a CMB Admissions Committee review of an applicant whose Personal Statement is attached as example_personal_statement.docx. Agent files and information about the Admissions Committee rubric are attached. The four reviewers (02, 03, 04, and 05) should generate their scores independently and should not see each other's scores or reviews until the full committee meets to evaluate all of the applicants. After the committee meets, agent 08 will create a detailed summary of the discussion and suggest ways that the personal statement could be improved to make it more competitive.
In about five minutes, our virtual Admissions Committee performed a very thorough evaluation of Tim's application and decided to invite him for an interview (congrats Tim, but you're stuck with us for now!). I encourage you to read (or at least skim) the full evaluation, which reflects the different viewpoints and scores of the individual reviewers. The feedback agent wrote a letter back to Tim that explains the committee's reasoning and suggested areas for improving his personal statement:
A stronger application on Engagement in Research, at the rubric's Outstanding (4.0) level, would show evidence that you yourself drove the intellectual direction of at least one project — that you formulated a scientific question, designed an experiment or analysis to address it, interpreted an ambiguous result, or identified a limitation that changed your own thinking. Your Personal Statement describes many excellent projects in "we" language; the rubric's Outstanding tier asks for at least one in clear "I asked, I designed, I interpreted" language. The Research Statement would be the natural home for this content. The Oropouche passage is close to this already — it could be even stronger if the reader could see the specific scientific question you were asking and how your interpretation of the result was yours. A stronger application on Why UW-Madison & CMB? would add two dimensions to what you already have. First, specific engagement with CMB as a program (the 13 focus groups, the three four-week rotations, the cross-departmental structure that lets students draw on 200+ faculty) — not just the named faculty, but the program architecture that makes those faculty accessible to a rotating student. Second, some authentic engagement with Madison as a place to live and work for five or six years. These are fair questions the rubric asks; your supplemental statement likely addresses them already.
With this information that was available to him using only the LLM, Tim could revise his Personal Statement to be even more competitive for a program like CMB. If we had added the other application materials within Tim's control (e.g., his CV) we would have likely received even better feedback. He could tailor this approach to create different personas for other programs that he is applying to, and re-run the simulation with different versions of his Personal Statement. By the time he submits his application, he will have already received expert-level feedback and would have the opportunity to incorporate it into his materials. I note that this process of using a Revisaurus doesn't change Tim or his accomplishments, it simply simulates the sort of expert, constructive feedback that all good scientists should receive frequently from colleagues on their writing and makes it available to everyone.
The same basic principles apply when using Revisaurus to improve a scientific manuscript. The outcome of a scientific manuscript submission is a decision letter written by an editor that integrates the perspectives of individual reviewers. Most journals have published Reviewer Guidelines that describe how reviewers should evaluate a paper, providing the basis for creating reviewer personas appropriate for the journal targetted for submission. For example, here are the Reviewer Guidelines for PLOS Pathogens; this tells us "how" the submission should be evaluated by reviewers. To determine "who" should perform the review, it can be a set of generic reviewers given the topic area of the journal. Alternately, I found it useful to prompt Claude with my CV and a request to create personas that reflect the areas where I have published in the last five years. This gave a list of six domain-specific experts who could serve as reviewers, as well as a journal editor persona. This is essentially the "cast" that will perform the review. I asked Claude to describe each of these cast members:
| File | Role |
|---|---|
| 01_journal_editor | Journal Handling Editor — manages the peer review process end-to-end: calibrates expectations to the target journal's selectivity tier (Tier 1 Nature/Science through Tier 4 PLoS ONE), assigns appropriate reviewers, synthesizes their critiques, and writes the decision letter (Accept / Minor / Major / Reject). Use this agent first to orchestrate a simulated review. |
| 02_reviewer_viral_genomics_epi | Viral Genomics & Molecular Epidemiology — evaluates manuscripts on pathogen genomic surveillance, phylogenetics, and viral evolution (SARS-CoV-2 variant tracking, influenza, cryptic lineages, persistent infections). Checks for data deposition, reproducible bioinformatics, and honest handling of sampling bias. Constructive but hard on papers that overclaim. |
| 03_reviewer_nhp_pathogenesis | NHP Models & Infectious Disease Pathogenesis — reviews nonhuman primate studies of viral infection, congenital/vertical transmission, and translational model development. Watchful for ARRIVE compliance, individual-animal data (never just group means at n=3–8), ethical justification, and appropriate model validity. Protective of animal welfare standards. |
| 04_reviewer_immunogenomics | Immunogenomics & NHP Population Genetics — specialist in MHC/KIR/FCGR genetics, NHP population genetics, and genotyping technologies (Illumina/ONT/PacBio). Rigorous about IPD nomenclature, novel-allele validation, and the distinction between cataloging diversity and illuminating biology. Collegial tone — a small field where reviewers know the authors. |
| 05_reviewer_methods_statistics | Methods, Statistics & Bioinformatics — the technical auditor. Reads Methods sections like financial statements, looking for unreported pipeline parameters, inappropriate statistical tests, missing code, and figures that mislead. Relentless on reproducibility: "data available upon request" is not acceptable. Assigned when a manuscript's computational backbone needs specialist scrutiny. |
| 06_reviewer_bioaerosol | Bioaerosol Pathogen Detection & Air Sampling — evaluates airborne pathogen surveillance (active/passive samplers, recovery efficiency, particle-size physics, and the viability gap between RT-qPCR positivity and infectious aerosol). Non-negotiable about reporting total air volume sampled. Harsh on papers that leap from RNA detection to transmission claims without discussing viability, dose, or ventilation. |
| 07_reviewer_wastewater | Wastewater Pathogen Detection & WBE — reviews wastewater-based epidemiology: sampling and concentration methods, process controls (PMMoV, CrAssphage), variant deconvolution (Freyja), cryptic lineage tracing, and normalization. Pushes authors to be quantitative where the field has been qualitative, and to distinguish "pathogen RNA in a processed sample" from "community transmission dynamics." |
I also asked Claude to create two additional personas that could facilitate manuscript revisions. These agents are not designed to replace expert human judgement, but rather to help guide scientists to provide the most effective point-by-point rebuttals and manuscript revisions that would satisfy the issues raised by the reviewers:
| File | Role |
|---|---|
| 08_manuscript_revision_strategist | Manuscript Revision Strategist — author-side counterpart to the reviewers. Drafts point-by-point responses to reviewer critiques using the Thank-Address-Act pattern, picks which battles to fight, and frames changes strategically without sacrificing integrity. Use after the editor returns a decision letter with reviewer comments. |
| 09_statistical_revision_consultant | Statistical Revision Consultant — author-side biostatistician for infectious disease surveillance and environmental epidemiology. Re-runs analyses in response to reviewer critiques on seasonality, temporal autocorrelation, confounding, or inappropriate tests. Produces executable R/Python code plus drafted Methods/Results text ready for insertion into the revised manuscript. |
To show how I would use these Revisaurus agents, let's consider a paper Shelby and I recently published describing assays for testing air samples for viral nucleic acids. Before this paper was published in Applied and Environmental Microbiology, we submitted it as a preprint to medRxiv. The preprint is what will use for our simulated peer review. Here is the prompt that I give Claude, along with agents 01-07. In this case, some of the reviewer personas are not relevant, so I give Claude the flexibility of creating additional personas as needed for the review. While Claude Opus 4.7 would not review the paper due to a safety flag, Opus 4.6 was fine with this prompt (personas 01-07 were attached to the chat):
You are going to simulate manuscript peer review for this preprint from my laboratory: https://www.medrxiv.org/content/10.1101/2025.10.21.25338488v1.full.pdf. We are going to submit to Applied and Environmental Microbiology, so the review should follow their guidelines: https://journals.asm.org/reviewer-guidelines. I'm attaching a journal editor persona as well as individual reviewer personas that correspond in general to the work in my laboratory. The journal editor should coordinate the review and use these reviewers, as well as other personas that are necessary to complete the review. When new personas are created, save their profiles in new Markdown files. The output should be a detailed decision letter, detailed Comments to the Authors and Comments to the Editor from each reviewer. The output should be returned in Markdown format. Note that I also had to upload the PDF of the preprint in a subsequent prompt because Opus was having difficulty with web search for PDFs.
The review created a new persona focused on CRISPR diagnostics.
Here is the detailed decision letter. In my testing, this process tends to very frequently produce a veridct of "Major Revisions" even though in this case the manuscript already navigated peer review and is published. I focus less on this part of the evaluation than on the specific issues raised by the reviewers. The comments that the reviewers raised are reasonable and fair. They would have improved the manuscript had we used Revisaurus before we finalized the submission.
Given how easy it is to generate personas and simulate peer review, I expect to have all of the manuscripts from my lab go through this process multiple times during its life cycle. When the first author has an initial draft, I'd expect them to use Revisaurus to get feedback before sharing with co-authors, and then we would use Revisaurus along with review from all other co-authors to try and anticipate and respond in advance to cricitisms likely to be raised by reviewers.
The other two personas can be used to faciliate the revisions. One of the challenges authors often have when addressing manuscript critiques is learning how to structure the revisions and the point-by-point rebuttal letter that accompanies a revision. While a fully automated point-by-point rebuttal is going to be dry and likely inadequate on its, the persona can help structure the rebuttal and suggest exactly where to incorporate revisions sensibly. So I might issue a follow-up prompt that includes the manuscript_revision_strategist and statistical_revision_consultant with the following prompt:
These comments are fair and reasonable. We now need to revise the manuscript and create a point-by-point rebuttal letter describing how we address the reviewers' concerns. While some concerns can be addressed by simply changing phrasing, others will require additional data that was not in the original submission, or even data that does not yet exist. When there are data that need to be provided manually, or through new experimentation, make note of that in both the rebuttal letter and revised manuscript. Generate the revised manuscript in Word format with Track Changes enabled so the automatic revisions can be easily reviewed and edited. Assume that we are willing to spend 2-4 weeks collecting new data, if absolutely necessary, but that we are not going to do an entirely new or different study to satisfy the reviewers.
Last week, our undergraduate student Celia Bartels presented a poster at the UW-Madison Global Health Symposium. Like many conferences, Celia first submitted an abstract. Hers was accepted and she made a poster for the event. Revisaurus provides approaches that would have improved both (note that I started working on Revisaurus after she had printed her poster).
The general pattern should be clear by now. First we define how the abstracts will be reviewed, then who we need to simulate to review them. Because the conference has already happened, I located the abstract information using the Internet Archive's Wayback Machine. We do not have a lot of information on the abstract review process itself, but since it is a University global health symposium, we can use that information to make some informed guesses with our prompt. Here is what we have on the website:
UW-Madison faculty, clinicians, staff, students, and partners who are addressing global health and disease in their respective fields and research are invited to submit a poster abstract and present their work. From basic research to education to applied projects in the field, the symposium hopes to showcase the full spectrum of UW-Madison’s global health activity. We encourage presentations from all disciplines—from arts, agriculture and business, to education, engineering, humanities, to all the health sciences, and more.
I used this to create one-off personas specific for this use case with this prompt:
We are going to simulate an abstract review for the UW-Madison Global Health Symposium. Here is what we know about the abstract process: UW-Madison faculty, clinicians, staff, students, and partners who are addressing global health and disease in their respective fields and research are invited to submit a poster abstract and present their work. From basic research to education to applied projects in the field, the symposium hopes to showcase the full spectrum of UW-Madison’s global health activity. We encourage presentations from all disciplines—from arts, agriculture and business, to education, engineering, humanities, to all the health sciences, and more. Create personas who would be responsible for reviewing these abstracts and determining which should rejected, given a poster, given a poster presentation, or given an oral talk. We will then use these personas to evaluate an abstract for the symposium.
Claude created five personas:
| Persona | Role |
|---|---|
| Dr. Amara Osei-Bonsu | Global Health Epidemiologist — Professor, Department of Population Health Sciences, with 20+ years of field experience in sub-Saharan Africa studying infectious disease transmission. Values methodological rigor, clear study design, appropriate statistical approaches, and well-defined populations. Skeptical of abstracts that overstate findings or lack ethical consideration of the communities studied. She asks: "Is the science sound, and does the methodology match the claims?" |
| Dr. Ricardo Méndez-Ferreira | Community-Engaged Research & Health Equity — Associate Professor, School of Medicine and Public Health, with a joint appointment in the Center for Community & Nonprofit Studies. Focuses on participatory research with Indigenous communities in Latin America. Prioritizes evidence of community partnership, equitable framing, and local relevance; pushes back on extractive research models. He asks: "Who benefits from this work, and were affected communities meaningfully involved?" |
| Dr. Priya Chakraborty | Global Health Policy & Systems — Clinical Associate Professor, School of Nursing, and former WHO consultant. Evaluates whether work connects to real-world health systems strengthening, scalability, and sustainability. Appreciates interdisciplinary approaches but expects presenters to articulate their contribution clearly. She asks: "So what? What does this change for policy or practice?" |
| Prof. James Kihara | Engineering & Technology for Development — Professor, Department of Biomedical Engineering, with extensive experience designing low-cost diagnostics for resource-limited settings. Represents the non-health-sciences disciplines the symposium explicitly welcomes. Values innovation, feasibility, and clear articulation of the problem being solved; generous toward early-stage or student-led work but expects intellectual honesty about limitations. He asks: "Is this novel, is it feasible, and does the presenter understand the real-world constraints?" |
| Maya Chen, MPH | Student & Early-Career Perspective — PhD candidate in Environment & Resources (Nelson Institute) and graduate representative on the committee, with field experience on climate-health projects in Southeast Asia. Advocates for student and trainee submissions; values clear communication and accessibility; flags when abstracts rely too heavily on jargon or assume insider knowledge. She asks: "Would a smart person outside this field understand what was done and why it matters?" |
I then issued a follow-up prompt that included Celia's abstract as an attachment:
Now we are going to use these personas to evaluate the attached abstract. As in most conference session reviews, not all reviewers will be assigned to review the abstract initially but there will be an opportunity for the committee to meet and rank the abstracts together. Generate detailed feedback on the review that can be provided to the applicant to improve for future submissions to similar types of conferences. Return output in Markdown format.
The review panel provided excellent feedback that would have improved the abstract.
Celia prepared a poster for the GHI symposium. Since we already have the ad hoc experts, we can reuse them to provide feedback on the poster. I will also add design and attendee personas to provide different perspectives. One of the most powerful features of the Revisaurus is that new skills and perspectives can be summoned instantly. Here is the prompt:
Celia was invited to prepare a poster for the symposium. A draft of the poster from Celia 3 days before the symposium is attached. Have the abstract review personas evaluate the poster and provide feedback on how Celia should improve it before the event. They should be joined by personas that simulate different expertise that can be expected from people who are viewing her poster at the poster session and a design persona focused on using visual design to communicate complex scientific information as effectively as possible. In addition to providing the feedback in Markdown format, also create a modified version of the poster that incorporates as many of these suggested improvements as possible in a PDF format.
The response is thorough and found scientific, design, and typographical issues. It also tried to create a PDF of a revised poster, which illustrates the jagged edge of LLMs; they are much better working with text than with complexed mixed text and graphical documents. It is nowhere near the quality of a poster that would be presented at a conference, but it does highlight conceptually how the poster should be improved.
The same general approach is also helpful when finding gaps in grant submissions. I often find that I need to have someone well outside my expertise read my proposals to find weaknesses and gaps that aren't apparent to me. The same principles apply: define the study section composition and the expertise they are expected to have, and then simulate a study section. I'm uncomfortable sharing the exact proposal I've used Revisaurus with because it involves other PIs, however, I will share the personas that I used (descriptions from Claude):
| File | Role |
|---|---|
| 01_panel_chair | Panel Chair — senior scientific expert (20+ years in biomedical research, deep familiarity with NIH funding mechanisms) who orchestrates the entire review. Assigns domain-specific reviewers, enforces rigor across critiques (no empty weakness sections), resolves scoring disagreements, and produces a structured consensus summary with a composite score table. Uses the NIH 1–9 scale and supports both the Simplified Peer Review Framework (Importance, Rigor & Feasibility, Expertise & Resources) and the traditional five-criterion system (Significance, Investigators, Innovation, Approach, Environment). |
| 02_reviewer_virology_hiv | Virology & HIV Cure Research — specialist in HIV/SIV/SHIV pathogenesis, broadly neutralizing antibody (bnAb) biology, and NHP models of lentiviral infection; attentive to cure strategies (block-and-lock, shock-and-kill, AAV-delivered bnAbs, CAR-T/NK). Generous on Significance when disease burden and NHP model are well-justified; rigorous on Approach. Flags thin Future Directions, overreliance on a single NHP model, conflation of in vitro potency with in vivo efficacy, and missing anti-drug antibody or viral escape discussion. |
| 03_reviewer_emerging_infectious_disease | Emerging Infectious Diseases & Pandemic Preparedness — expert in emerging pathogens (SARS-CoV-2, HPAI H5N1, Mpox, Nipah, Marburg), environmental surveillance (wastewater sequencing, metagenomics), and preparedness frameworks (CEIR/CEIRR, SPHERES, NIAID SAVE). Rewards demonstrated scientific agility and both reactive and proactive capabilities. Flags aspirational preparedness language without milestones, missing data sharing plans, surveillance limited to known pathogens, and research that doesn't feed back into public health practice. |
| 04_reviewer_immunology_training | Immunology & Training / Career Development — dual-expertise reviewer covering immunological research (innate/adaptive immunity, vaccine immunology, immunogenetics) and graduate/postdoctoral training program evaluation. The most critical voice on training quality, mentoring structures, career outcomes, and diversity; expects quantitative 3Rs/NAMs estimates (e.g., "reducing from 18 to 6 animals"). Flags vague enrichment language, absent career outcome data, missing undergraduate research, and NAMs sections that rehash published methods. |
| 05_reviewer_genomics_immunogenetics | Genomics & Immunogenetics — specialist in genomics technologies, immunogenetics (MHC, KIR, FCGR, ABO), and precision genomic infrastructure. Assesses Approach on operational metrics (throughput, turnaround time, error rates, financial sustainability). Skeptical of scope creep; prefers tight focus with demonstrated excellence. Flags capabilities without demonstrated demand, missing performance benchmarks, single-platform reliance, fee-for-service projections ungrounded in history, and failure to link genotyping to experimental design. |
| 06_reviewer_nhp_colony_regulatory | NHP Colony Management & Regulatory Compliance — pragmatic, operationally minded expert in NHP colony management, breeding program design, regulatory compliance, and NPRC operations. Scores Significance exceptionally when services address active regulatory/ethical crises (e.g., parentage verification after the Cambodia smuggling scandal); rewards honest risk assessment. Flags missing succession plans, revenue projections exceeding history, missing discussion of NHP supply chain challenges, and formatting errors that signal hasty preparation. |
| 07_reviewer_ai_computational | AI, Computational Biology & Data Science — specialist in AI/ML, large language models, bioinformatics, and computational tool development. Distinguishes genuine capability from "AI theater," demanding working tools, validation data, and risk management; rewards fallback plans and human-in-the-loop design; demands specific validation thresholds (e.g., ">99% concordance across three independent datasets"). Flags AI plans without pilot data, undefined thresholds, no error detection framework, over-reliance on commercial APIs, and data sharing claims without specific repositories. |
After receiving feedback from the reviewers, I created a persona that can help guide revisions, similar to what was described for the manuscript revision agent described above.
| File | Role |
|---|---|
| 08_revision_expert | Scientific Grant Revision Expert — unlike the reviewers, sits on the investigator's side of the table. Transforms study section critiques into specific, implementable changes for an A1 revised application, writing in the technical voice of the original investigators. Analyzes critiques to identify underlying concerns, categorizes issues by severity (must/should/nice-to-address), implements tracked changes via OOXML editing, and flags unknowable information with [VERIFY], [DATA NEEDED], and [OPTIONAL] markers. Guiding principles: never defensive; every new sentence earns its place under page limits; all revisions traceable to a specific reviewer critique. |
The typical workflow proceeds in three phases:
- Assignment: The Panel Chair reads the application, identifies needed expertise, and assigns reviewers.
- Review: Each reviewer independently evaluates the application using the NIH five-criterion system, providing scores (1–9), strengths, weaknesses, and a summary for each criterion plus an overall impact score.
- Synthesis: The Panel Chair reconciles individual reviews into a consensus summary, composite score table, and prioritized list of improvements.
- Revision (optional): The Revision Expert takes the panel's output and produces ready-to-implement text changes for the investigator team.
We then receive the feedback and make edits accordingly. With the example where I used these personas, the feedback revealed some very specific blind spots that I had missed and were easily addressed in a few added sentences of text. I expect that I'll be using this repeatedly with my grants in the future.
LLM capabilities are evolving and improving very frequently. As of this writing, the approaches described above have been working very well for me and I hope they benefit others who want to sharpen their scientific communication. I'm a bit odd in that I'm an open data and open science evangelist. While I will not put sensitive or restricted data into commercial (non-local) LLMs and I disable the use of my prompts in model training, there is no visibility into whether these settings are respected. I am comfortable accepting that risk with my work, but I do not expect that this is a unanimous perspective. Similarly, I've used the Revisaurus methods described here for my own work. It's obvious that the same approaches could be used to lazily critique the work of others as a stand-in for human review. Personally, I think two things are true at once: (1) automated peer review with LLMs could be more fair and at least as high quality as existing human reviewers and; (2) as long as the expectation of scientific creators is that a human is reviewing their work, it would be unethical to outsource my expert judgement as a member of a review panel to an LLM.