📢 Alert: raven rsi/en

Experiment 1 · nanochat

7 rounds of refinement.
Better results within the same training budget.

nanochat is an open-source project for training small language models. This experiment used one A800 GPU, with each training run limited to 20 minutes. Powered by DeepSeek v4.1-flash and the ai4ai plugin, Raven used each round's results to revise train.py and launch further experiments, completing 7 rounds of iteration.

0.9956→0.9381

val_bpb, lower is better

The official train.py baseline is 0.9956. The final configuration reached 0.9381 ± 0.0007 in round 7 (mean ± standard deviation across 3 random seeds), a 5.8% relative reduction.

Grey dots show individual training results: 170 runs have recorded values; 2 have no recorded result. The blue line shows the mean across random seeds for each round's best configuration, with error bars of ±1 standard deviation. The dashed line marks the official train.py baseline. Hover over a data point for details.

Performance was still improving when the budget ran out

The reductions in val_bpb in rounds 6 and 7 both substantially exceeded the observed variation between runs (0.0013). Across 7 rounds, Raven executed 172 training runs with no recorded crashes, using 3586 of the 3600 GPU minutes available.

What is val_bpb?

val_bpb is validation bits per byte: the model's prediction loss on held-out text, expressed in bits and normalized by the number of bytes. Lower values indicate better predictive performance on unseen text.

7 changes from the official training code

Halve the training batch size (2¹⁹ → 2¹⁸), increasing the number of steps within 20 minutes from 1540 to 3055
Add a learnable hash embedding table for adjacent token pairs, with 100× the vocabulary size (819200 rows)
Add the table's output to the model representation after layer 0
Reduce the learning rate MATRIX_LR from 0.04 to 0.02
Shorten the local attention window from 1024 to 256
Increase model width: ASPECT_RATIO 64 → 80, corresponding to a width of 512 → 640
Fix the evaluation batch size at 128

Best configuration in each round

Round	Configuration	val_bpb	Seeds	GPU min

Notes: the final configuration was evaluated with 3 random seeds; the report recommends extending this to 7–8. The wider model takes about 30% longer per step, reducing the number of steps within 20 minutes from 3055 to 2363, but still achieves a lower val_bpb. This reflects the net gain under a fixed time budget. The experiment accepts an improvement over the previous round's best only when Welch |t| > 3 and |Δ| > 0.002. None of round 1's 9 candidates met this criterion; every subsequent round produced a configuration that did.

Experiment 2 · CFD dam-break simulation

Restoring stability, then reducing numerical error

This experiment used OpenFOAM, an open-source computational fluid dynamics (CFD) solver, to simulate a 2D dam break as a water column collapses and strikes a wall. Raven ran 7 rounds (0–6) on the same case with 81648 mesh cells. The water volume fraction, alpha, describes the proportion of water in each cell. Its physical range is [0, 1]; values outside that range are numerical bound violations.

Round 0: water rises along the right wall with an intact surface — Representative frames from each round. Dark blue indicates water, light colours indicate air, and red dots mark cells with bound violations. Completed runs are shown at the same moment as water strikes the right wall; runs that stopped early show their final frame. Simulation time is denoted by t, with a target end time of t=1.0.

Round 1: numerical instability fragments the water distribution — Representative frames from each round. Dark blue indicates water, light colours indicate air, and red dots mark cells with bound violations. Completed runs are shown at the same moment as water strikes the right wall; runs that stopped early show their final frame. Simulation time is denoted by t, with a target end time of t=1.0.

Larger time steps led to three incomplete runs

Rounds 1–3 raised maxCo from the baseline of 1 to 20, 100 and 500, allowing larger time steps. The number of cells with bound violations rose from 0 at baseline to 241, 3877 and 1241, respectively. None of these runs reached the target end time.

Restoring stability reduced the remaining error

Round 4 restored maxCo to 1, completed the simulation and conserved water volume. Rounds 5 and 6 adjusted the interface calculation to reduce bound violations further. Deviations of approximately 1e-7 to 1e-10 remained in rounds 4–6, so all three failed the strict requirement that alpha remain within [0, 1].

Parameter changes and results by round

Round	Parameter changes	Result

Note: all bound-violation counts and magnitudes on this page were extracted from simulation field data by the visualization script. Raven's experiment report sampled different locations, so its values may differ. Round 4 records a minimum alpha of −1.97e-07; the chart shows the violation magnitude, 1.97e-07.

Experiment 3 · FEA cantilever beam

8 load tests to locate the solver's convergence limit

Finite element analysis (FEA) divides a structure into discrete elements to compute its response to applied loads. This experiment, conducted on 2026-08-17, used the open-source solver CalculiX to load a cantilever beam fixed at one end. As the load increases, the solver may fail to converge. Raven first established an interval between converged and non-converged loads, then narrowed it by bisection. The result describes numerical convergence for this model and solver configuration, not the beam's physical load capacity.

Converged ×Did not converge Narrowing search interval The apparent tearing is an illustration, not a simulated fracture. Displayed deformation is capped at 30% for visualization.

Establish the interval, then bisect

The solver did not converge at 2000 kN in round 0, but converged at 1800 kN in round 1. Each subsequent round tested the midpoint between the highest converged load and the lowest non-converged load.

Narrow the interval between test loads

The solver converged at 1962.5 kN but not at 1965.625 kN. The experiment completed with the status "bisection converged" and a record of 107 events.

Separate convergence from load capacity

The beam's theoretical plastic limit load, calculated from its material and cross-section, is 62.5 kN, well below all 8 test loads. The 1962.5 kN result is the highest tested load at which the solver converged; it does not establish the beam's load capacity.

Test loads and solver results by round

Round	Load (kN)	Result	Run time

Experiment 4 · AI4AI cost comparison

Lower-cost models can sustain longer research efforts

This study compared Raven and Claude Code paired with different models under time or cost constraints. The task was to pretrain a 50-million-parameter language model using nanochat, evaluated by BPB (bits per byte; lower is better), with an A800 baseline of 1.108. The training setup and budgets differ from experiment 1, so the BPB values should not be compared directly.

Raven · V4.1 (lower-cost model)
Raven · other models
Claude Code · Opus 5 (reference)

5-hour budget, no cost limit

Similar BPB at about 1/40 of the cost. Raven with V4.1 achieved a BPB of 1.037 for $0.97, compared with 1.038 for $39.41 using Opus 5. Claude Code with Opus 5 achieved a BPB of 1.053.

$50 budget, no time limit

$6.12 for the lowest BPB in this comparison: 0.965. Raven with V4.1 ran for more than 80 hours, supporting more rounds of exploration. The other three configurations reached or approached the $50 budget limit.

Complete results under both budget conditions

5-hour time budget
Framework · model	Time	Tokens	Cost	BPB

$50 cost budget
Framework · model	Time	Tokens	Cost	BPB

Raven provides a reliable framework for sustained research,
grounding each decision in evidence that can be verified.

How Raven organizes research and experiments

Multi-agent orchestration: Agents collaborate through clearly defined deliverables. Code implementation and experiment execution are separate, allowing results to be checked independently. Shared memory preserves context and reduces repeated information and token use.
Research: Raven assesses task difficulty from retrieved evidence and adjusts its search strategy to the available budget and remaining information needs.
Code: Implementation is followed by independent testing and acceptance checks. Key facts are verified directly against tool outputs.
Experiment monitoring: Raven submits jobs, checks progress and decides whether to retry or stop. Scheduled checks, heartbeats and persistent sessions support extended runs.
The ai4ai workflow: Raven explores several directions, implements candidates, validates them and measures their performance. Results guide whether to conclude the experiment or investigate a promising direction further.
Research idea library: A record of tested approaches and unexplored directions provides context for future experiments and helps Raven develop and combine new ideas.

7 rounds of refinement.Better results within the same training budget.