AI Today

Paper: https://github.com/deepseek-ai/Janus/blob/main/janus_pro_tech_report.pdf Github: https://github.com/deepseek-ai/Janus/tree/main?tab=readme-ov-file The paper introduces Janus-Pro, an improved multimodal model building upon its predecessor, Janus. Janus-Pro boasts enhanced performance in both multimodal understanding and text-to-image generation due to optimized training strategies, expanded datasets (including synthetic aest...

Paper: https://arxiv.org/pdf/2412.09764 This research paper explores the effectiveness of memory layers in significantly enhancing large language models (LLMs). By incorporating a trainable key-value lookup mechanism, memory layers add parameters without increasing computational cost, improving factual accuracy and overall performance on various tasks. The researchers demonstrate substantial gains, especially on factual tasks, eve...

Paper: https://scontent-dfw5-1.xx.fbcdn.net/... This research paper introduces Large Concept Models (LCMs), a novel approach to language modeling that operates on sentence embeddings instead of individual tokens. LCMs aim to mimic human-like abstract reasoning by processing information at a higher semantic level, enabling improved handling of long-form text generation and zero-shot multilingual capabilities. The authors explore v...

Technical Report: https://arxiv.org/pdf/2412.19437 Github: https://github.com/deepseek-ai/DeepSe... This research paper introduces DeepSeek-V3, a 671-billion parameter Mixture-of-Experts (MoE) large language model. The paper details DeepSeek-V3's architecture, including its innovative auxiliary-loss-free load balancing strategy and Multi-Token Prediction objective, and its efficient training framework utilizing FP8 precision. Exte...

Paper: https://arxiv.org/pdf/2309.16588 This research paper examines artifacts in vision transformer feature maps, specifically high-norm tokens appearing in non-informative image areas. The authors propose adding "register" tokens to the input sequence as a solution. This simple addition eliminates the artifacts, improves performance on dense prediction tasks and object discovery, and results in smoother feature and attention map...

Paper: https://arxiv.org/pdf/2412.09871v1.pdf The paper introduces the Byte Latent Transformer (BLT), a novel large language model architecture that processes raw byte data without tokenization. BLT dynamically groups bytes into patches based on predicted entropy, allocating more computational resources to complex sections of text. This approach achieves performance comparable to tokenization-based models while significantly impro...

This research paper introduces CosyVoice 2, an improved streaming speech synthesis model. Building upon its predecessor, CosyVoice 2 utilizes advancements in large language models (LLMs) and incorporates optimizations like finite scalar quantization and a chunk-aware causal flow matching model. The result is a system achieving near human-parity naturalness with minimal latency in streaming mode, supporting multiple languages and of...

Blog: https://openai.com/12-days/ OpenAI announced two new large language models, o3 and o3-mini, showcasing significantly improved performance on various benchmarks, including coding, mathematics, and reasoning tasks. These models surpass previous models (like o1) in accuracy and efficiency. While not yet publicly released, OpenAI is initiating public safety testing, inviting researchers to help evaluate the models' safety and i...

Paper: https://arxiv.org/pdf/2412.14093 This research paper explores "alignment faking" in large language models (LLMs). The authors designed experiments to provoke LLMs into concealing their true preferences (e.g., prioritizing harm reduction) by appearing compliant during training while acting against those preferences when unmonitored. They manipulate prompts and training setups to induce this behavior, measuring the extent of ...

Blog: https://blog.google/technology/google... Google announced updates to its AI video and image generation models, Veo 2 and Imagen 3, boasting state-of-the-art capabilities in realism and style diversity. These improvements are integrated into existing Google Labs tools, VideoFX and ImageFX, and a new tool, Whisk, which allows image-based prompting and remixing using Imagen 3 and Gemini's visual understanding. Veo 2 excels in ...

Paper: https://arxiv.org/pdf/2411.01747 This research report introduces Allegro, a novel, open-source text-to-video generation model that surpasses existing open-source and many commercial models in quality and temporal consistency. The authors detail Allegro's architecture, a multi-stage training process leveraging a custom-designed Video Variational Autoencoder (VideoVAE) and Video Diffusion Transformer (VideoDiT), and a rigorou...

Paper: https://arxiv.org/pdf/2411.01747 The paper "DynaSaur: Large Language Agents Beyond Predefined Actions" introduces a novel large language model (LLM) agent framework that dynamically generates and executes actions using a general-purpose programming language, overcoming limitations of existing systems restricted to predefined action sets. This approach enhances the LLM agent's flexibility and planning capabilities, significa...

Paper: https://arxiv.org/pdf/2411.17116 The paper introduces Star Attention, a novel two-phase attention mechanism for efficient Large Language Model (LLM) inference on long sequences. It improves computational efficiency by sharding attention across multiple hosts, using blockwise-local attention in the first phase and sequence-global attention in the second. This approach achieves up to an 11x speedup in inference time while mai...

Paper: https://arxiv.org/pdf/2410.18967 The paper introduces Ferret-UI 2, a multimodal large language model (MLLM) that significantly improves upon its predecessor, Ferret-UI, by enabling universal user interface (UI) understanding across diverse platforms (iPhone, Android, iPad, webpages, and AppleTV). Key improvements include multi-platform support, high-resolution perception through adaptive scaling, and advanced task training ...

Paper: https://arxiv.org/abs/2411.00412 This research introduces a novel two-stage training method to improve Large Language Models' (LLMs) ability to solve complex scientific problems. The method, called Adapting While Learning (AWL), first distills world knowledge into the LLM via supervised fine-tuning. Then, it adapts tool usage by classifying problems as easy or hard, using direct reasoning for easy problems and tools for har...

Paper: https://arxiv.org/pdf/2411.02830 This research introduces Mixtures of In-Context Learners (MOICL), a novel approach to improve in-context learning (ICL) in large language models (LLMs). MOICL addresses ICL's limitations by partitioning demonstrations into expert subsets and learning a weighting function to combine their predictions. Experiments demonstrate MOICL's superior performance across various classification datasets,...

Paper: https://arxiv.org/pdf/2411.04997 Github: https://github.com/microsoft/LLM2CLIP The paper introduces LLM2CLIP, a method to improve the visual representation learning capabilities of CLIP by integrating large language models (LLMs). LLM2CLIP addresses CLIP's limitations with long and complex text by fine-tuning the LLM to enhance its textual discriminability, effectively using the LLM's knowledge to guide CLIP's visual encode...

Paper: https://arxiv.org/pdf/2411.14199 Github: https://github.com/AkariAsai/OpenScholar The research introduces OpenScholar, a retrieval-augmented large language model (LLM) designed for synthesizing scientific literature. OpenScholar uses a large datastore of open-access papers and iterative self-feedback to generate high-quality responses to scientific questions, including accurate citations. A new benchmark, ScholarQABench, is...

Paper: https://arxiv.org/pdf/2401.03407 Github: https://github.com/ZhengPeng7/BiRefNet This research introduces BiRefNet, a novel deep learning framework for high-resolution dichotomous image segmentation. BiRefNet uses a bilateral reference mechanism, incorporating both original image patches and gradient maps, to improve the accuracy of segmenting fine details. The framework is composed of localization and reconstruction modules...