15,762 technical terms and definitions
flamingo multimodal model, gpt4v vision language, visual question answering vlm, multimodal large language model
CLIP, LLaVA, Flamingo, and GPT-4V represent different points along the evolution of vision-language model architecture: CLIP learns aligned image and text representations through…
vision-language model cross-modal alignment and joint representation, vision-language model cross-modal, model cross-modal alignment, cross-modal alignment joint
Vision-Language Model Cross-Modal Alignment and Joint Representation Introduction & Motivation Vision-Language Model Cross-Modal Alignment and Joint Representation addresses…
multimodal llm, llava visual instruction, visual question answering deep, image text model
A vision-language model (VLM) answers questions about, describes, or otherwise reasons over an image by combining a vision encoder's extracted image features with a large language…
multimodal ai
Vision-language models (VLMs) combine a vision encoder with a language model so the system can reason jointly over images and text, taking an image as input and producing…
multimodal ai
Advanced vision-language models (VLMs) achieve deep integration of visual and linguistic understanding through architectures that jointly process images and text. Modern…
multimodal alignment, contrastive language image pretraining, visual question answering vlm, image text models
CLIP, BLIP, and LLaVA each take a different approach to connecting vision and language: CLIP uses contrastive language-image pretraining to align image and text representations…
multimodal ai
Vision-Language Pre-training Objectives are the loss functions used to train foundation models on massive unlabelled data — teaching them to understand the relationship between…
multimodal ai
Vision-language pre-training objectives is the set of training losses used to teach multimodal models to align, fuse, and reason across visual and textual inputs - objective…
Vision-Language Pretraining Introduction & Motivation Vision-Language Pretraining addresses a central problem in multimodal learning and pretrain-transfer-benefit alignment: how…
Vision-Language Recommendation Introduction & Motivation Vision-Language Recommendation addresses a central problem in multimodal learning and item-description-alignment…
computer vision
Vision State Space Models (VSSM) are the sequence modeling successors that treat images as flattened sequences and apply linear state-space recurrences to achieve global receptive…
vision system alignment accuracy prediction machine learning, vision system, alignment accuracy prediction, vision system alignment accuracy prediction, system alignment
Vision System Alignment Accuracy Prediction with Machine Learning Introduction & Motivation Vision System Alignment Accuracy Prediction with Machine Learning addresses a central…
vit, patch embedding, image tokens, visual transformer
The Vision Transformer (ViT) showed that the Transformer architecture built for language works just as well on images, and that insight is the bridge to today's multimodal models.…
image patch transformer, visual attention, image transformer
The Vision Transformer applies the Transformer architecture, originally developed for language, directly to images by splitting an image into fixed-size patches and treating each…
Vision Transformer Architecture Introduction & Motivation Vision Transformer Architecture addresses a central problem in vision processing and patch-sequence-modeling prediction…
Masked Autoencoders for Vision Transformers Introduction & Motivation Masked Autoencoders for Vision Transformers addresses a central problem in vision processing…
vision transformer patch embedding and image tokenization prediction, vision transformer patch, transformer patch embedding, patch embedding image, embedding image tokenization
Vision Transformer Patch Embedding and Image Tokenization Prediction Introduction & Motivation Vision Transformer Patch Embedding and Image Tokenization Prediction addresses…
large vit, vision model scaling laws, billion parameter vision transformer, vit scaling
Vision Transformer Scaling is the study and practice of increasing Vision Transformer model size, dataset size, sequence length, and training compute to improve downstream…
computer vision
Vision Transformer Variants encompass the diverse family of architectures that adapt, extend, or improve upon the original Vision Transformer (ViT) for image understanding tasks…
image patch embedding, vit classification, transformer image recognition, visual attention mechanism
The Vision Transformer (ViT) processes an image by splitting it into a grid of fixed-size patches, linearly embedding each patch into a vector, and feeding the resulting sequence…
vision transformer vit, vision transformer, transformer vit, 21k cifar cnn mlp msa
Vision Transformer - ViT Introduction & Motivation Vision Transformer: apply transformer architecture to image classification. Patch embeddings, self-attention for spatial…
patch embedding, image transformer, vit attention, vision transformer training
Training a Vision Transformer involves feeding patch embeddings, combined with learned positional encodings since the patches themselves carry no inherent order, through stacked…
image patch embedding, vit architecture training, visual transformer classification, deit vision transformer
ViT architecture and training have inspired several variants, including DeiT (Data-efficient image Transformer), which specifically addresses ViT's original weakness of needing…
patch embedding image, vit self attention, image tokens cls, vit deit training
The Vision Transformer represents an image as a sequence of patch tokens, each combined with a learned positional encoding, and often includes an additional learnable…
patch embedding transformer, position encoding image, vision transformer scaling, vit vs cnn comparison
Vision Transformer architecture, compared directly against convolutional neural networks, trades the convolution's built-in assumption of local spatial structure…
patch embedding transformer, vit attention mechanism, vision transformer training, vit vs cnn comparison
The Vision Transformer (ViT) showed that the Transformer architecture built for language works just as well on images, and that insight is the bridge to today's multimodal models.…
vision transformers, masked image modeling, 1d 2d 300m bert clip coco, vision transformers & masked image modeling, vision transformers masked, transformers masked image, transformers masked
Vision Transformers & Masked Image Modeling Introduction & Motivation For most of the 2010s, convolutional neural networks were the unchallenged default architecture for computer…
101244 vision, vision transformers, ade20k cifar sota, loss functions for different tasks, loss functions different, functions different tasks, loss functions, functions different, different tasks
Loss Functions for Different Tasks Introduction & Motivation Loss Functions: measure prediction error. Cross-entropy, MSE, Triplet loss. Applications: objective definition…
vision transformers vit image classification transformers, vision transformers, vit image classification, 1k cifar clip cls mlp passed, vision transformers vit, transformers vit
Vision Transformers: ViT & Image Classification via Transformers Introduction & Motivation Vision Transformer (ViT): apply Transformer directly to image patches. Divide image into…
vision transformers, vit patch embedding, 1d 2d 86m cls cnn coco, vision transformers vit & patch embedding, vision transformers vit, transformers vit patch, transformers vit, vit patch
Vision Transformers: ViT & Patch Embedding Introduction & Motivation Vision transformers: apply transformers to images. Patch embedding: divide image into patches; embed linearly.…
multimodal ai
Visual commonsense reasoning is the multimodal reasoning task that infers likely intents, causes, or outcomes in scenes beyond directly visible facts - it requires combining…
Visual Commonsense Reasoning Introduction & Motivation Visual Commonsense Reasoning addresses a central problem in multimodal learning and commonsense-accuracy alignment: how…
evaluation
Visual Entailment is a reasoning task derived from textual entailment (NLI) — where the model must determine the logical relationship between an image (premise) and a sentence…
multimodal ai
Visual entailment is the task of determining whether an image supports, contradicts, or is neutral with respect to a textual hypothesis - it adapts natural-language inference…
multimodal ai
Visual grounding is the task of linking language expressions to corresponding regions or objects in an image - it is fundamental for interpretable multimodal interaction. What Is…
Visual Grounding and Localization Introduction & Motivation Visual Grounding and Localization addresses a central problem in multimodal learning and grounding-accuracy-recall…
visual inertial odometry, visual inertial, inertial odometry
Visual Inertial Odometry Introduction & Motivation Visual Inertial Odometry addresses a central problem in vision process control and inertial-measurement-unit-imu prediction: how…
Visual Instruction Following Introduction & Motivation Visual Instruction Following addresses a central problem in multimodal learning and instruction-accuracy-compliance…
multimodal ai
Visual Instruction Tuning is the training process that teaches Multimodal LLMs to follow human instructions — transforming valid pre-trained models (which might just describe…
vlm, llava, gpt4v, multimodal llm, vision language model, image question answering
A visual language model (VLM) processes both images and natural language jointly, commonly by feeding image features extracted by a vision encoder into a large language model…
multimodal ai
Visual Prompting is an interaction technique for computer vision models — where users provide visual cues (points, boxes, scribbles, or reference images) as inputs to guide…
multimodal ai
Advanced visual question answering is the multimodal task where models answer complex questions about images by combining object recognition, relation understanding, and language…
Visual Question Answering in Dialogue Introduction & Motivation Visual Question Answering in Dialogue addresses a central problem in multimodal learning…
visual question answering, visual question, question answering
Visual Question Answering Introduction & Motivation Visual Question Answering addresses a central problem in vision process control and vqa-task-image-question-answer prediction…
visual question answering, vqa, multimodal ai
Visual Question Answering (VQA) is a multimodal AI task where a system receives an image and a natural language question about that image and must produce an accurate natural…
visual question, question answering, 1m ai coco gqa id passed, visual question answering (vqa), visual question answering, question answering (vqa), answering (vqa)
Visual Question Answering (VQA) Introduction & Motivation Visual Question Answering: answer natural language questions about images. Vision-language reasoning; multimodal…
Visual Question Generation Introduction & Motivation Visual Question Generation addresses a central problem in multimodal learning and question-relevance-answerability alignment…
Visual Rationale Generation for Visual Question Answering Introduction & Motivation Visual Rationale Generation for Visual Question Answering addresses a central problem…
multimodal ai
Visual reasoning is the process of drawing logical conclusions from visual inputs by analyzing objects, attributes, relations, and scene context - it extends computer vision…
Visual Relationship Detection Introduction & Motivation Visual Relationship Detection addresses a central problem in multimodal learning and relationship-accuracy-specificity…