Visual Instruction Tuning: LLaVA: Large Language and Vision AssistantVisual Instruction Tuning
LLaVA represents a novel end-to-end trained large multimodal model that combines a vision encoder and Vicuna for general-purpose visual and language understanding, achieving impressive chat capabilities mimicking spirits of the multimodal GPT-4 and setting a new state-of-the-art accuracy on Science QA. The project can be found here.