PathoGen - Histopathology Image Inpainting

PathoGen is a diffusion-based model for histopathology image inpainting. It enables realistic tissue pattern generation for filling masked regions in pathology whole slide images (WSI).

Model Description

Model Type: Diffusion model with custom attention processors
Task: Image inpainting for histopathology images
Architecture: UNet2DConditionModel with custom SkipAttnProcessor
Framework: PyTorch, Diffusers, PyTorch Lightning

Usage

Installation

git clone https://github.com/mkoohim/PathoGen.git
cd PathoGen
pip install -r requirements.txt

Download Weights

Download the attention weights and place them in your checkpoint directory:

from huggingface_hub import hf_hub_download

# Download attention weights
hf_hub_download(
    repo_id="mkoohim/PathoGen",
    filename="attention.pt",
    local_dir="./checkpoints"
)

Inference

from src.models.pathogen import PathoGenModel
from omegaconf import OmegaConf
from PIL import Image

# Load configuration
config = OmegaConf.load("configs/config.yaml")

# Initialize model
model = PathoGenModel(config)
model.load_attention_weights("./checkpoints/attention.pt")
model.eval()

# Load images
image = Image.open("your_wsi_crop.jpg")
mask = Image.open("your_mask.jpg")
condition = Image.open("your_source_image.jpg")

# Run inference
result = model(image, mask, condition)

Training

python train.py

See the GitHub repository for full training instructions.

Model Files

File	Description	Size
`attention.pt`	Trained attention module weights	~190MB

Training Details

Base Model: Stable Diffusion Inpainting UNet
Training Data: Histopathology whole slide image crops
Optimizer: AdamW
Learning Rate: 1e-5
Precision: Mixed precision (FP16)

Intended Use

This model is designed for:

Histopathology image inpainting and augmentation
Research in computational pathology
Data augmentation for pathology AI training

Citation

@misc{pathogen2025,
  title={PathoGen: Diffusion-Based Synthesis of Realistic Lesions in Histopathology Images},
  author={mkoohim},
  year={2025},
  url={https://huggingface.co/mkoohim/PathoGen}
}

License

This model is released under the MIT License.

mkoohim
/

PathoGen