pubmedqa-pointer-generator-network

This repository contains a Google Colab notebook for training a sequence-to-sequence (Seq2Seq) model in a Pointer-Generator Network (PGN) using the PubMedQA dataset. The notebook uses Hugging Face Transformers and Datasets to fine-tune a BART model for biomedical question answering.

Trained Model: The fine-tuned model is available on Hugging Face at dev2004v/pgn_bart_pubmed_qa_80k

---

Project Objective

To train a text generation model that:

• Answers biomedical questions based on research abstracts
• Uses supporting context from scientific literature
• Produces long-form, context-aware answers
• Supports copying important terms from the input

---

Dataset

• Name: PubMedQA
• Split: pqa_artificial
• Source: Hugging Face Datasets

Fields used:

• question - The biomedical question to answer
• context - Research abstract providing supporting information
• long_answer - Detailed answer to be generated

---

Input–Output Format

Model Input

Question: <question_text>
Context: <abstract_text>

Model Target

<long_answer>

---

Workflow

1. Load PubMedQA dataset from Hugging Face
2. Split data into training and validation sets
3. Preprocess data into PGN-style prompts
4. Tokenize inputs and targets
5. Apply dynamic padding using a data collator
6. Configure Seq2Seq training arguments
7. Train the model using Seq2SeqTrainer

---

Training Features

• Dynamic padding for computational efficiency
• Gradient accumulation for effective batch sizing
• Gradient checkpointing for reduced memory usage
• Mixed-precision (FP16) training for faster computation
• Automatic best model selection based on validation loss

---

Requirements

The notebook installs all required dependencies automatically:

transformers
datasets
torch
accelerate

Recommended: A GPU-enabled Google Colab runtime

---

How to Run

1. Open the notebook in Google Colab
2. Select GPU as the runtime accelerator (Runtime > Change runtime type > GPU)
3. Run all cells sequentially
4. Model checkpoints will be saved to the output directory

---

Using the Trained Model

You can directly use the trained model from Hugging Face:

from transformers import BartTokenizer, BartForConditionalGeneration

model_name = "dev2004v/pgn_bart_pubmed_qa_80k"
tokenizer = BartTokenizer.from_pretrained(model_name)
model = BartForConditionalGeneration.from_pretrained(model_name)

# Example usage
question = "Does caffeine improve athletic performance?"
context = "Your research abstract here..."
input_text = f"Question: {question}\nContext: {context}"

inputs = tokenizer(input_text, return_tensors="pt", max_length=1024, truncation=True)
outputs = model.generate(**inputs, max_length=150, num_beams=4)
answer = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(answer)

---

Output

• Training and validation loss logs
• Saved model checkpoints per epoch
• Best model loaded automatically at the end of training
• Evaluation metrics on validation set

---

License

This project is intended for academic and learning purposes.

---

Acknowledgments

• PubMedQA Dataset
• Hugging Face Datasets - Dataset implementation used in this project
• Hugging Face Transformers
• Pointer-Generator Networks (See et al., 2017)