IC-Custom Tencent: Complete Guide to Diverse Image Customization

IC-Custom, developed by Tencent ARC, is a powerful tool designed for advanced image customization. It provides two distinct modes — Position-aware and Position-free, enabling users to customize images with incredible precision or generate new images based on text prompts and reference images.

In this guide, you will learn everything about IC-Custom, including its features, environment setup, installation process, running scripts, and step-by-step usage for both customization modes.

Overview of IC-Custom

IC-Custom is designed for diverse image customization scenarios. It supports two main modes:

1. Position-Aware Mode

• Allows precise placement of objects within a specific area of the image.

• You need to input:

  • A reference image
  • A target background image
  • A customization location (selected via segmentation or drawing tools).

2. Position-Free Mode

• Focuses on subject-driven image generation.

• You provide:

  • A reference image
  • A text description of the desired scene.

• The model generates a new image containing the reference image’s ID in the specified style or context.

System Requirements

IC-Custom has been implemented and tested on the following environment:

Installation Guide

Follow these steps to install IC-Custom on your machine.

Step 1: Clone the Repository

git clone https://github.com/TencentARC/IC-Custom.git
cd IC-Custom

Step 2: Set Up Python Environment

Use Conda to create and activate a new Python environment:

conda create -n ic-custom python=3.10 -y
conda activate ic-custom

Step 3: Install Dependencies

Install IC-Custom and required packages:

pip install -e .
pip install -r requirements.txt

Step 4: Custom CUDA Versions (Optional)

If you need a different CUDA version, skip the torch packages in requirements.txt and manually install PyTorch and Xformers compatible with your CUDA version.

Example for CUDA 12.4:

pip3 install xformers torch==2.5.0 torchvision==0.20.0 torchaudio==2.5.0 --index-url https://download.pytorch.org/whl/cu124

Step 5: Install Flash Attention 2 (Optional)

If your device supports Flash Attention 2, you can install it for performance improvements:

pip install flash-attn==2.7.3

Check compatible versions on the Flash Attention releases page.

Model Checkpoints Setup

Before running inference or launching the app, you need to download model checkpoints.

Option 1 — Automatic Download

• Models are automatically fetched when running scripts.
• Requires a valid Hugging Face token (HF_TOKEN).

Option 2 — Manual Download

Use the provided helper script:

sh scripts/inference/download_models.sh $HF_TOKEN

Expected Directory Structure:

|-- models
    |-- clip-vit-large-patch14
    |-- ic-custom
    |   |-- dit_boundary_embeddings_0x1561.safetensors
    |   |-- dit_lora_0x1561.safetensors
    |   |-- dit_task_register_embeddings_0x1561.safetensors
    |   |-- dit_txt_img_in_0x1561.safetensors
    |-- siglip-so400m-patch14-384
    |-- t5-v1_1-xxl
    |-- ae.safetensors
    |-- flux1-fill-dev.safetensors
    |-- flux1-redux-dev.safetensors

Configure Local Paths

After downloading, edit the configs/**/*.yaml files (e.g., inference.yaml) to point to the correct local model paths.

For a compact per-model list with download links, check MODEL_CARD.md in the repository.

Running IC-Custom

You can run IC-Custom in two main ways: Gradio App or Inference Script.

1. Running the Gradio App

The Gradio interface automatically downloads required models.

sh src/app/run_app.sh $HF_TOKEN $HF_CACHE_DIR

• $HF_TOKEN: Hugging Face token for model access.
• $HF_CACHE_DIR (optional): Custom cache directory for models.

For more details on required vs optional models, refer to APP.md.

2. Running Inference Script

To run inference directly:

sh scripts/inference/inference.sh $HF_TOKEN $HF_CACHE_DIR

Parameters:

Quick Start Guide

IC-Custom offers two main customization modes: Position-aware and Position-free. Here’s how to get started.

Step 1: Initial Setup (For Both Modes)

• Launch the app.
• Choose your customization mode.
• Upload a reference image.

Step 2A: Position-Aware Mode (Precise Placement)

This mode is ideal when you need to customize specific areas of an image.

1. Select Input Mask Mode:

   • Precise Mask
   • User-Drawn Mask

2. Upload Target Image and Create Mask:

   • Click for SAM segmentation or brush manually.

3. Add Text Prompt (Optional):

   • Use VLM Auto-generate or Auto-polish buttons for automatic prompts.

4. Review and Refine Mask:

   • Tools available: Dilate, Erode, Bounding Box.

5. Run the Process:

   • Click Run to generate customized results.

Step 2B: Position-Free Mode (Subject-Driven Generation)

This mode generates a completely new image using only a reference image and text description.

1. Write a Text Prompt:

   • Clearly describe the target scene.
   • Example: "A bright yellow alarm clock sits on a wooden desk."

2. Use VLM Features (Optional):

   • Auto-generate or polish prompts.

3. Click Run:

   • The system generates a new image based on your description.

Quick Tips for Best Results

• Zoom Out: Use Alt + "-" or ⌘ + "-" for better navigation.
• Advanced Options: Adjust settings for optimal results.
• Mask Operations: Use dilate, erode, or bounding box for precise customization.
• Experiment with Seeds: Different seeds produce varied outputs.

Example Configurations

Using IC-Custom via API

IC-Custom supports API-based integration, allowing developers to build apps and services around its image customization capabilities.

• Built with Gradio backend for seamless interaction.
• API documentation is included in the repository.

Final Thoughts

IC-Custom by Tencent ARC is a versatile tool for anyone looking to create customized images with precision and creativity. Whether you need pixel-perfect editing with Position-aware mode or creative generation with Position-free mode, IC-Custom provides a powerful and open-source solution.

With its Gradio app, API support, and detailed documentation, it's easy for both beginners and developers to get started. Start exploring today and take your image customization to the next level.