Converting LoRA Models for Invoke AI

Step-by-step guide to update your model with safetensor metadata

Key Highlights

Identify and convert the original format – Determine the current model format and apply proper conversion techniques.
Utilize established tools and libraries – Leverage Hugging Face’s PEFT, diffusers, and conversion scripts to generate safetensors.
Test and validate – Ensure that the properly converted model loads correctly in Invoke AI while maintaining compatibility with your base model variant.

Understanding the Conversion Requirement

Your LoRA model, based on the "pony" base model, currently lacks the safetensor metadata needed by Invoke AI. The safetensors format is a secure and efficient method for storing model weights, ensuring proper compatibility and faster load times when used with systems like Invoke AI. Converting your model involves both a file format change and the incorporation of the necessary metadata.

Step 1: Identify the Current Format

Examine Your Model File

Before starting the conversion process, check the current extension of your LoRA model – it might be in formats such as .ckpt or .pt. This information is crucial as the conversion steps may vary depending on the original file format.

Step 2: Convert to Safetensors Format

Using Conversion Tools and Libraries

To integrate safetensor metadata, you can utilize various resources and scripts available in the community. The conversion process often involves the following:

A. With Hugging Face Tools

The recommended approach involves leveraging the Hugging Face PEFT library. The following steps outline how to reformat a model:

Install the Hugging Face PEFT and diffusers libraries if they are not already installed.
Fine-tune or adapt the model using instant training methods (e.g., using a dreambooth technique) which generates a .pt file and a corresponding JSON configuration file.
Convert the generated file to the safetensors format using dedicated conversion scripts such as format_convert.py from the Lora-for-Diffusers repository.

Here is an example of a Python script that may be adapted for your needs:


# Import necessary libraries
from diffusers import StableDiffusionPipeline

# Specify your model location or id
model_id = "your_model_id_or_path"

# Load the model using diffusers
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype="float16")

# Save the model, selecting safetensors as the format
pipe.save_pretrained("path/to/save/model", save_format="safetensors")

B. Conversion Using Community Scripts

Often, the conversion requires adjusting file structure and metadata. You may need to use community-generated scripts found on GitHub or forums such as:

llama.cpp docker container for conversion in various operating systems (macOS/Linux/Windows).
Custom functions provided by projects like Lora-for-Diffusers to convert between binaries (.bin) and safetensors (.safetensors).

Ensure that any conversion script you use verifies the integrity and structure, including metadata, to be compatible with Invoke AI standards.

Step 3: Adding and Validating Metadata for Invoke AI

Ensuring Proper Format and Metadata

Convert your LoRA model and then check that the result includes all necessary metadata:

The file should be saved with the .safetensors extension.
Metadata must include configuration details (often provided as a separate JSON file) that comply with Invoke AI requirements.
Ensure proper directory placement in your Invoke AI system structure, typically in a dedicated loras folder.

Once converted, load the model in Invoke AI to confirm that it recognizes the updated format properly. This step is essential as it verifies the success of the conversion.

Step 4: Testing the Converted Model in Invoke AI

Verifying Compatibility

After placing the converted safetensor model in the appropriate directory:

Start or restart the Invoke AI server to reload models.
Navigate to the model management or LoRA section of the Invoke AI interface.
Select the converted model and adjust settings such as LoRA strength and additional parameters as per your needs.

If the invocation fails or the model does not appear:

Double-check the file naming conventions and directory paths according to the official Invoke AI documentation.
Consult relevant community forums or troubleshooting guides to address any compatibility issues that may arise.

Comprehensive Comparison Table

Step	Description	Tools/Resources
Identify Format	Determine current file format (e.g., .pt, .ckpt)	File inspectors, Invoke AI logs
Conversion	Use PEFT/diffusers for conversion to safetensors	Hugging Face libraries, conversion scripts (e.g., format_convert.py)
Add Metadata	Ensure file includes all necessary metadata	JSON config files, directory conventions
Testing	Validate model functionality in Invoke AI	Invoke AI interface, community troubleshooting

Additional Considerations and Best Practices

Understanding Compatibility

Pay careful attention to the compatibility between the version of Invoke AI you are using and your model. For example, ensure the model is built for the corresponding version of Stable Diffusion (SD1.5, SDXL, etc.). In some cases, additional parameters or adjustments may be required if the base model or LoRA adapter does not fully match the expected configuration.

Documentation and Community Support

Always refer to the latest official documentation for both Invoke AI and the libraries you choose to use. Active community forums such as GitHub discussions, Reddit threads, and documented guides on platforms like Restack.io can prove invaluable. These sources frequently provide updated scripts, debugging tips, and workarounds based on user experiences.

Why Converting to Safetensors?

Security and Speed

The safetensors format is designed to be a safer alternative to traditional serialization formats, such as pickle-based .pt files. Besides improved security, safetensors offer enhanced speed and reduced memory usage during model loading – a crucial factor for interactive systems like Invoke AI.

Consistency Across Platforms

Uniformity in metadata and file structures ensures that models behave consistently across different computing environments. By adhering to these standards, you minimize the risk of compatibility issues when moving models between platforms or collaborating in community environments.