TPB: AdaBoosting Text Prompts for Vision-Language Models

Official PyTorch implementation of "AdaBoosting Text Prompts for Vision-Language Models", accepted to ECCV 2026 as a Spotlight paper.

[Project Page] [arXiv]

Introduction

Natural-language prompts are interpretable and can be reused across heterogeneous vision-language models (VLMs), but existing few-shot text-prompting methods often gain little from additional labeled examples. Text Prompt Boosting (TPB) addresses this limitation with an AdaBoost-inspired framework that treats text-prompt classifiers as weak learners and sequentially combines them into a strong ensemble.

At each boosting round, TPB constructs class-wise prompt banks under the current sample weights. Misclassified examples receive more weight, directing the next prompt classifier toward unresolved cases. Because TPB operates in discrete text space, the learned prompt ensemble can be re-embedded and directly transferred to other VLMs.

Environment Setup

We recommend creating a Conda environment with Python 3.10:

conda create -n tpb python=3.10
conda activate tpb

Install PyTorch for your CUDA version. For example:

conda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 \
  pytorch-cuda=12.1 -c pytorch -c nvidia

Then install the remaining dependencies and the bundled Dassl.pytorch package:

pip install -r requirements.txt
pip install -e Dassl.pytorch

Dataset Installation

Follow Dassl.pytorch/DATASETS.md to prepare ImageNet and the other ten classification datasets used in the paper:

• Caltech101
• DTD
• EuroSAT
• FGVCAircraft
• Food101
• ImageNet
• OxfordFlowers
• OxfordPets
• StanfordCars
• SUN397
• UCF101

Copy the environment template and set the dataset and model-cache paths:

cp .env.py env.py

DATASET_ROOT = "/path/to/datasets"
MODEL_ROOT = "/path/to/model/cache"

The expected dataset layouts and names are defined in configs/datasets/.

Get Started

The following command runs 16-shot TPB with OpenAI CLIP ViT-B/32, 50 boosting rounds, and four training-time augmentations:

CUDA_VISIBLE_DEVICES=0 python adaboost_unified.py \
  --dataset eurosat \
  --shots 16 \
  --seed 1 \
  --n_est 50 \
  --algorithm samme.r \
  --backbone vit_b32 \
  --prompt_src augmented_total \
  --aug_train \
  --aug_times 4 \
  --group_suffix tpb \
  --no_date

Replace eurosat with one of the dataset identifiers in configs/datasets/. Results are written to output/, and prompt embeddings are cached in prompt_embeds/. The initial run may spend substantial time building the prompt cache; later runs with the same configuration reuse it.

Use the following command to see all training options:

python adaboost_unified.py --help

Cross-Model Evaluation

unified_transfer.py re-embeds saved TPB prompt ensembles and evaluates them on heterogeneous CLIP/OpenCLIP-family backbones. Its checkpoint types and target-model groups are listed in the command-line help:

python unified_transfer.py --help

Contact

If you encounter any problems or have questions about the code, please open a GitHub issue or contact Changhwan Sung.

Acknowledgements

This implementation builds on OpenAI CLIP, OpenCLIP, Dassl.pytorch, and CoOp. The multiclass boosting implementation is adapted from the SAMME.R implementation in scikit-learn.

The prompt-library construction and prompt assets are based on ProAPO. The augmented_total prompt pool combines class descriptions from DCLIP, CuPL, GPT4Vis, AdaptCLIP, and AWT, together with the OpenAI CLIP prompt templates.

The cross-model evaluation code additionally supports comparisons with ProAPO, CoOp, PEZ, and PromptSRC. We thank all authors for making their code and prompt resources publicly available.

Citation

If you find this repository useful, please cite:

@inproceedings{jin2026tpb,
  title     = {AdaBoosting Text Prompts for Vision-Language Models},
  author    = {Jin, Seokhee and Sung, Changhwan and Mun, Sunung and Kim, Hoyoung and Ok, Jungseul},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year      = {2026}
}