UniversalAdapter

Universal Adapter Main Module

Core class that orchestrates architecture-agnostic LoRA transfer across different model families.

class ual_adapter.core.adapter.UniversalAdapter(base_model: PreTrainedModel | Module, tokenizer: PreTrainedTokenizer | None = None, device: str = 'auto')[source]

Bases: object

Main class for Universal Adapter LoRA functionality.

Handles training, exporting, importing, and transferring LoRA adapters across different model architectures.

__init__(base_model: PreTrainedModel | Module, tokenizer: PreTrainedTokenizer | None = None, device: str = 'auto')[source]

Initialize Universal Adapter.

Parameters:

  • base_model – The base model to work with

  • tokenizer – Optional tokenizer for the model

  • device – Device to use (‘auto’, ‘cuda’, ‘cpu’)

train_adapter(adapter_name: str, training_data: List[str] | Any, rank: int = 16, alpha: float = 16.0, dropout: float = 0.1, target_modules: List[str] | None = None, epochs: int = 3, learning_rate: float = 0.0001, batch_size: int = 8, **training_kwargs) → Dict[str, Any][source]

Train a new LoRA adapter.

Parameters:

  • adapter_name – Name for the adapter

  • training_data – Training data (texts or dataset)

  • rank – LoRA rank

  • alpha – LoRA alpha scaling parameter

  • dropout – LoRA dropout rate

  • target_modules – Modules to apply LoRA to (auto-detected if None)

  • epochs – Number of training epochs

  • learning_rate – Learning rate

  • batch_size – Training batch size

  • **training_kwargs – Additional training arguments

Returns:

Dictionary with training results

export_adapter(adapter_name: str, output_path: str) → None[source]

Export adapter to portable AIR format.

Parameters:

  • adapter_name – Name of adapter to export

  • output_path – Path to save AIR files

import_adapter(air_path: str, adapter_name: str | None = None, projection_method: str = 'svd', auto_project: bool = True) → Dict[str, Any][source]

Import adapter from AIR format.

Parameters:

  • air_path – Path to AIR format files

  • adapter_name – Name for imported adapter (uses domain from metadata if None)

  • projection_method – Method for dimension projection

  • auto_project – Whether to automatically project dimensions

Returns:

Import results dictionary

transfer_to_model(adapter_name: str, target_model: PreTrainedModel | Module, projection_method: str = 'svd', verify_compatibility: bool = True) → Tuple[Dict[str, Tensor], Dict[str, Any]][source]

Transfer adapter to a different model architecture.

Parameters:

  • adapter_name – Name of adapter to transfer

  • target_model – Target model to transfer to

  • projection_method – Method for dimension projection

  • verify_compatibility – Whether to verify architecture compatibility

Returns:

Tuple of (transferred weights, transfer report)

list_adapters() → List[Dict[str, Any]][source]

List all loaded adapters with their information.

clear_adapters() → None[source]

Clear all loaded adapters.

Overview

The UniversalAdapter class is the main entry point for working with UAL Adapter. It provides methods for training, exporting, and importing LoRA adapters.

Class Definition

class ual_adapter.core.adapter.UniversalAdapter(base_model: PreTrainedModel | Module, tokenizer: PreTrainedTokenizer | None = None, device: str = 'auto')[source]

Main class for Universal Adapter LoRA functionality.

Handles training, exporting, importing, and transferring LoRA adapters across different model architectures.

__init__(base_model: PreTrainedModel | Module, tokenizer: PreTrainedTokenizer | None = None, device: str = 'auto')[source]

Initialize Universal Adapter.

Parameters:

  • base_model – The base model to work with

  • tokenizer – Optional tokenizer for the model

  • device – Device to use (‘auto’, ‘cuda’, ‘cpu’)

train_adapter(adapter_name: str, training_data: List[str] | Any, rank: int = 16, alpha: float = 16.0, dropout: float = 0.1, target_modules: List[str] | None = None, epochs: int = 3, learning_rate: float = 0.0001, batch_size: int = 8, **training_kwargs) → Dict[str, Any][source]

Train a new LoRA adapter.

Parameters:

  • adapter_name – Name for the adapter

  • training_data – Training data (texts or dataset)

  • rank – LoRA rank

  • alpha – LoRA alpha scaling parameter

  • dropout – LoRA dropout rate

  • target_modules – Modules to apply LoRA to (auto-detected if None)

  • epochs – Number of training epochs

  • learning_rate – Learning rate

  • batch_size – Training batch size

  • **training_kwargs – Additional training arguments

Returns:

Dictionary with training results

export_adapter(adapter_name: str, output_path: str) → None[source]

Export adapter to portable AIR format.

Parameters:

  • adapter_name – Name of adapter to export

  • output_path – Path to save AIR files

import_adapter(air_path: str, adapter_name: str | None = None, projection_method: str = 'svd', auto_project: bool = True) → Dict[str, Any][source]

Import adapter from AIR format.

Parameters:

  • air_path – Path to AIR format files

  • adapter_name – Name for imported adapter (uses domain from metadata if None)

  • projection_method – Method for dimension projection

  • auto_project – Whether to automatically project dimensions

Returns:

Import results dictionary

transfer_to_model(adapter_name: str, target_model: PreTrainedModel | Module, projection_method: str = 'svd', verify_compatibility: bool = True) → Tuple[Dict[str, Tensor], Dict[str, Any]][source]

Transfer adapter to a different model architecture.

Parameters:

  • adapter_name – Name of adapter to transfer

  • target_model – Target model to transfer to

  • projection_method – Method for dimension projection

  • verify_compatibility – Whether to verify architecture compatibility

Returns:

Tuple of (transferred weights, transfer report)

list_adapters() → List[Dict[str, Any]][source]

List all loaded adapters with their information.

clear_adapters() → None[source]

Clear all loaded adapters.

Methods

Training Methods

UniversalAdapter.train_adapter(adapter_name: str, training_data: List[str] | Any, rank: int = 16, alpha: float = 16.0, dropout: float = 0.1, target_modules: List[str] | None = None, epochs: int = 3, learning_rate: float = 0.0001, batch_size: int = 8, **training_kwargs) → Dict[str, Any][source]

Train a new LoRA adapter.

Parameters:

  • adapter_name – Name for the adapter

  • training_data – Training data (texts or dataset)

  • rank – LoRA rank

  • alpha – LoRA alpha scaling parameter

  • dropout – LoRA dropout rate

  • target_modules – Modules to apply LoRA to (auto-detected if None)

  • epochs – Number of training epochs

  • learning_rate – Learning rate

  • batch_size – Training batch size

  • **training_kwargs – Additional training arguments

Returns:

Dictionary with training results

Export/Import Methods

UniversalAdapter.export_adapter(adapter_name: str, output_path: str) → None[source]

Export adapter to portable AIR format.

Parameters:

  • adapter_name – Name of adapter to export

  • output_path – Path to save AIR files

UniversalAdapter.import_adapter(air_path: str, adapter_name: str | None = None, projection_method: str = 'svd', auto_project: bool = True) → Dict[str, Any][source]

Import adapter from AIR format.

Parameters:

  • air_path – Path to AIR format files

  • adapter_name – Name for imported adapter (uses domain from metadata if None)

  • projection_method – Method for dimension projection

  • auto_project – Whether to automatically project dimensions

Returns:

Import results dictionary

Utility Methods

UniversalAdapter.list_adapters() → List[Dict[str, Any]][source]

List all loaded adapters with their information.

Usage Examples

Basic Training

from ual_adapter import UniversalAdapter
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("gpt2")
tokenizer = AutoTokenizer.from_pretrained("gpt2")

ual = UniversalAdapter(model, tokenizer)

# Train adapter
ual.train_adapter(
    domain_name="medical",
    texts=training_texts,
    rank=16,
    alpha=32
)

Export and Import

# Export trained adapter
ual.export_adapter("medical", "medical.air")

# Import to different model
target_model = AutoModel.from_pretrained("TinyLlama/TinyLlama-1.1B")
target_ual = UniversalAdapter(target_model)
target_ual.import_adapter("medical.air")

Managing Adapters

# List all loaded adapters
adapters = ual.list_adapters()

# Get adapter information
info = ual.get_adapter_info("medical")

# Remove adapter
ual.remove_adapter("medical")

See Also