Struct BlipCaptioner 

pub struct BlipCaptioner {
    device: Device,
    model: Arc<RwLock<Option<BlipForConditionalGeneration>>>,
    tokenizer: Arc<RwLock<Option<Tokenizer>>>,
    cache_dir: PathBuf,
    initialized: Arc<RwLock<bool>>,
    config: CaptionConfig,
}

BLIP-based image captioner using Candle.

Uses the Salesforce/blip-image-captioning-base model from HuggingFace Hub. Requires the vision feature to be enabled.

Fields

device: Device

Device to run inference on (CPU or CUDA)

model: Arc<RwLock<Option<BlipForConditionalGeneration>>>

BLIP model

tokenizer: Arc<RwLock<Option<Tokenizer>>>

Text tokenizer

cache_dir: PathBuf

Cache directory for model files

initialized: Arc<RwLock<bool>>

Whether model is initialized

config: CaptionConfig

Configuration

Implementations

impl BlipCaptioner

pub fn new(cache_dir: PathBuf) -> Self

Create a new BLIP captioner.

pub fn with_config(cache_dir: PathBuf, config: CaptionConfig) -> Self

Create with custom configuration.

fn preprocess_image(&self, image_data: &[u8]) -> Result<Tensor, CaptionError>

Preprocess image data into a tensor suitable for BLIP.

async fn generate_caption( &self, image_tensor: &Tensor, ) -> Result<String, CaptionError>

Generate caption from image embedding.

Trait Implementations

impl ImageCaptioner for BlipCaptioner

fn init<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<(), CaptionError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Initialize the captioner (load model, etc.).

fn caption<'life0, 'life1, 'async_trait>( &'life0 self, image_data: &'life1 [u8], ) -> Pin<Box<dyn Future<Output = Result<Option<String>, CaptionError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Generate a caption for image bytes.

fn is_initialized<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = bool> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Check if the captioner is initialized.

fn model_name(&self) -> &str

Get the model name.