Performance Optimization

Tips for optimizing UAL Adapter performance.

Memory Optimization

Use Lower Ranks

# Reduces memory by 50%
ual.train_adapter(domain="example", texts=data, rank=8)  # vs rank=16

Gradient Checkpointing

model.gradient_checkpointing_enable()

Speed Optimization

Batch Processing

# Process multiple queries at once
results = dispatcher.batch_generate(queries, max_length=50)

Cache Embeddings

dispatcher = LoRADispatcher(cache_embeddings=True)

Projection Method

# Truncate is fastest
projector.project(weights, dim, method="truncate")

Quality vs Speed Tradeoffs

SVD: Best quality, slower
Interpolate: Good balance
Truncate: Fastest, may lose quality