AI Models

Raindrop provides access to a comprehensive suite of AI models through a unified interface. These models enable sophisticated capabilities including text generation, image processing, speech recognition, language translation, and content analysis directly within your applications.

All AI models are accessed through the same env.AI.run() interface, with model-specific input and output types ensuring type safety and clear documentation of capabilities.

Ready to implement AI capabilities?

Check out the AI Models reference documentation for detailed API information, available models, and implementation examples.

Language Models (LLMs)

Large Language Models generate human-like text, engage in conversations, and perform complex language tasks like analysis, summarization, and code generation.

What They Do

• Text Generation: Create articles, emails, stories, and other written content
• Conversational AI: Build chatbots and virtual assistants that maintain context
• Code Generation: Generate, explain, and debug code in multiple programming languages
• Analysis: Analyze sentiment, extract entities, and categorize text content
• Function Calling: Integrate with external APIs and tools by calling functions when needed

Available Models

See the AI Models reference documentation for the complete list of available LLMs including LLaMA, Mistral, DeepSeek, and specialized models.

When to Use

Use LLMs when you need to process or generate human language, build conversational interfaces, or perform complex reasoning tasks that benefit from language understanding.

Speech Recognition

Speech recognition models convert spoken audio into written text, enabling voice interfaces and audio content processing.

What They Do

• Audio Transcription: Convert speech recordings into accurate text transcripts
• Real-time Voice Input: Enable voice commands and dictation in applications
• Multilingual Support: Process speech in multiple languages and accents
• Timestamp Generation: Provide word-level timing for audio synchronization

Available Models

See the AI Models reference documentation for the complete list of available speech recognition models including Whisper variants.

When to Use

Use speech recognition when building voice interfaces, creating meeting transcription tools, processing audio content, or enabling accessibility features for voice input.

Image Processing Models

Image processing models analyze, classify, and understand visual content, enabling applications to work with photos, documents, and other visual media.

Image Classification

Identifies and categorizes objects, scenes, or concepts in images.

What It Does:

• Recognizes objects, animals, vehicles, and scenes in photos
• Categorizes images for content organization and search
• Enables content moderation based on visual content

Available Models: See the AI Models reference documentation for available image classification models.

Object Detection

Locates and identifies multiple objects within a single image, providing position information.

What It Does:

• Finds and labels multiple objects in complex scenes
• Provides bounding box coordinates for each detected object
• Enables advanced image analysis and automated tagging

Available Models: See the AI Models reference documentation for available object detection models.

Image-to-Text (Vision)

Converts images into descriptive text using multimodal models that understand both visual and textual information.

What It Does:

• Generates detailed descriptions of image content
• Answers questions about what’s shown in images
• Enables conversational interactions about visual content
• Supports accessibility by describing images for visually impaired users

Available Models: See the AI Models reference documentation for available image-to-text models.

When to Use Image Processing

Use image models for content moderation, automated tagging, accessibility features, or any application that needs to understand visual content.

Image Generation

Image generation models create new images from text descriptions, edit existing images, or transform images based on prompts.

What They Do

• Text-to-Image: Create original images from written descriptions
• Image-to-Image: Transform existing images based on text prompts
• Inpainting: Edit specific regions of images using masks
• Style Transfer: Apply artistic styles or visual transformations

Available Models

See the AI Models reference documentation for available text-to-image and image editing models.

When to Use

Use image generation for creating marketing visuals, prototyping designs, generating artwork, or building creative tools that need custom visual content.

Translation Models

Translation models convert text between different languages using neural machine translation.

What They Do

• Multilingual Translation: Convert text between 100+ language pairs
• Preserve Context: Maintain meaning and tone across languages
• Handle Specialized Content: Process technical, business, or casual text appropriately

Available Models

See the AI Models reference documentation for available translation models.

When to Use

Use translation models for building multilingual applications, localizing content, or enabling communication across language barriers.

Text Analysis Models

Text analysis models examine and categorize written content to extract insights, classify sentiment, or organize information.

Text Classification

Categorizes text content for sentiment analysis, content moderation, and automated organization.

What It Does:

• Analyzes sentiment (positive, negative, neutral)
• Classifies content by topic or category
• Enables content filtering and moderation

Available Models: See the AI Models reference documentation for available text classification models.

Text Summarization

Generates concise summaries of long text content while preserving key information.

What It Does:

• Creates executive summaries of long documents
• Extracts key points from articles or reports
• Condenses information while maintaining important details

Available Models: See the AI Models reference documentation for available text summarization models.

Text Embeddings

Converts text into numerical vectors that capture semantic meaning for similarity search and clustering.

What They Do:

• Enable semantic search by meaning rather than exact keywords
• Power recommendation systems based on content similarity
• Cluster related documents or content automatically
• Support retrieval-augmented generation (RAG) systems

Available Models: See the AI Models reference documentation for available text embedding models.

When to Use Text Analysis

Use text analysis models for content moderation, sentiment tracking, document organization, search functionality, or any application that needs to understand and categorize written content.