Deep Dive in Research

The article introduces EDINET-Bench, a novel open-source Japanese financial benchmark designed to evaluate Large Language Models (LLMs) on complex financial tasks. This benchmark addresses the scarcity of challenging Japanese financial datasets for LLM evaluation, crucial for tasks like accounting fraud detection, earnings forecasting, and industry prediction. The EDINET-Bench dataset is automatically compiled from ten years of Jap...

The article introduces AutoThink, an innovative approach designed to enhance the inference efficiency and accuracy of reasoning Large Language Models (LLMs). AutoThink addresses the challenge of LLMs generating excessive or insufficient reasoning tokens, which leads to computational inefficiency and suboptimal performance. This system comprises two main components: a query complexity classifier that dynamically allocates the optima...

The article introduces System Prompt Learning (SPL), an innovative approach enabling Large Language Models (LLMs) to learn and refine problem-solving strategies through practical experience. This method addresses the current disparity where most developers lack the sophisticated system prompts that make advanced AI assistants so capable. SPL represents a "third paradigm" of LLM learning, augmenting traditional pretraining...

This article introduces OpenEvolve, an open-source implementation of Google DeepMind's AlphaEvolve, a system that leverages Large Language Models (LLMs) in an evolutionary framework to generate and optimize code. OpenEvolve allows users to evolve entire codebases by iteratively creating modifications using LLMs, evaluating them with automated metrics, and selecting promising solutions through an evolutionary process. The articl...

This paper introduces Pivotal Token Search (PTS), a novel method for improving the performance of large language models by focusing on critical decision points in their output sequences. Unlike traditional methods that treat all generated tokens equally, PTS identifies "pivotal tokens" that significantly influence the probability of a successful generation. By using a binary search algorithm to pinpoint these key tokens, ...

This episode introduces CameraBench, a large-scale dataset and benchmark designed to improve camera motion understanding in videos. It details a taxonomy of camera motion primitives developed with cinematographers, highlighting how motions can relate to scene content like tracking subjects. The authors describe a rigorous annotation framework and human study demonstrating how domain expertise and training enhance annotation accurac...

This epidsode introduces Step1X-Edit, an open-source image editing model designed to close the performance gap with proprietary models like GPT-4o. The developers created a large-scale, high-quality dataset and a new benchmark (GEdit-Bench) reflecting real-world editing instructions to train and evaluate the model. Step1X-Edit integrates a Multimedia Large Language Model (MLLM) with a diffusion-based image decoder to perform divers...

Visual reasoning is a core component of human intelligence and a critical capability

for advanced multimodal models. Yet current reasoning evaluations of multimodal

large language models (MLLMs) often rely on text descriptions and allow languagebased reasoning shortcuts, failing to measure genuine vision-centric reasoning.

To address this, we introduce VisuLogic: a benchmark of 1,000 human-verified

problems across six categories (e.g.,...

Reinforcement Learning with Verifiable Rewards (RLVR) has recently demonstrated notable success in enhancing the reasoning capabilities of LLMs, particularly in mathematics and programming tasks. It is widely believed that RLVR enables LLMs to continuously self-improve, thus acquiring novel reasoning abilities that exceed corresponding base models' capacity. In this study, however, we critically re-examines this assumption by measu...

Recent advances in large reasoning models (LRMs) demonstrate that sophisticated behaviors such as multi-step reasoning and self-reflection can emerge via reinforcement learning (RL) with simple rule-based rewards. However, existing zero-RL approaches are inherently ``on-policy'', limiting learning to a model's own outputs and failing to acquire reasoning abilities beyond its initial capabilities. We introduce LUFFY (Lea...

This text explores the nature of time from a computational perspective. It argues that time is not a fundamental coordinate but rather a consequence of the universe's computational processes. The author proposes that time is "the progressive doing of computation by the universe," and that our perception of time arises from our own computational limitations as observers. The text further suggests that the universe's computational ir...

This research paper describes the development and capabilities of "Movie Gen," a new suite of generative AI models that produce high-quality, realistic videos and audio. The paper highlights key advancements in text-to-video and video-to-audio synthesis, video editing, and video personalization. The authors detail their models' architecture, training procedures, and evaluation metrics, demonstrating superior performance compared to...