Dimension Projection

Dimension-Adaptive Projection System

Handles dimension mismatches when transferring adapters between models with different hidden dimensions using SVD-based projection.

class ual_adapter.core.projection.DimensionProjector(variance_threshold: float = 0.95)

Bases: object

Handles dimension adaptation for LoRA weights using SVD projection.

This allows adapters trained on models with one dimension (e.g., 768) to be transferred to models with different dimensions (e.g., 2048).

__init__(variance_threshold: float = 0.95)

Initialize the dimension projector.

Parameters:

variance_threshold – Percentage of variance to preserve (0-1)

project_adapter(lora_a: Tensor, lora_b: Tensor, target_in_dim: int, target_out_dim: int, method: str = 'svd') → Tuple[Tensor, Tensor]

Project LoRA adapter to target dimensions.

Parameters:

• lora_a – LoRA A matrix (rank x in_features)

• lora_b – LoRA B matrix (out_features x rank)

• target_in_dim – Target input dimension

• target_out_dim – Target output dimension

• method – Projection method (“svd”, “truncate”, “interpolate”)

Returns:

Tuple of projected (A, B) matrices

analyze_projection_quality(original_delta: Tensor, projected_a: Tensor, projected_b: Tensor) → Dict[str, float]

Analyze the quality of a projection.

Returns metrics like reconstruction error, rank preservation, etc.

DimensionProjector Class

class ual_adapter.core.projection.DimensionProjector(variance_threshold: float = 0.95)

Handles dimension adaptation for LoRA weights using SVD projection.

This allows adapters trained on models with one dimension (e.g., 768) to be transferred to models with different dimensions (e.g., 2048).

__init__(variance_threshold: float = 0.95)

Initialize the dimension projector.

Parameters:

variance_threshold – Percentage of variance to preserve (0-1)

project_adapter(lora_a: Tensor, lora_b: Tensor, target_in_dim: int, target_out_dim: int, method: str = 'svd') → Tuple[Tensor, Tensor]

Project LoRA adapter to target dimensions.

Parameters:

• lora_a – LoRA A matrix (rank x in_features)

• lora_b – LoRA B matrix (out_features x rank)

• target_in_dim – Target input dimension

• target_out_dim – Target output dimension

• method – Projection method (“svd”, “truncate”, “interpolate”)

Returns:

Tuple of projected (A, B) matrices

analyze_projection_quality(original_delta: Tensor, projected_a: Tensor, projected_b: Tensor) → Dict[str, float]

Analyze the quality of a projection.

Returns metrics like reconstruction error, rank preservation, etc.