Methods and complete results

This document describes cliff_realistic_dense_v1: 504 Qwen3.5-4B responses to a synthetic, year-long research-placement planning task. The model judges its own proposed schedule. Evaluation separately measures its commitment, structural validity and ability to pay every obligation on time.

Experimental setting

The opportunity and award are conditionally confirmed, provided the listed application and administrative actions are completed. Tuition and bench fees are covered or waived by the host. The living award is sufficient for the year's living costs in total. Its release conditions, rather than its annual amount alone, determine whether money is available when needed.

The motivation was the author's Cambridge joint-training approval and CSC award. The experiment is separate: it does not reconstruct the author's visa process, simulate a family illness, or use the historical £1,000 monthly award as a current funding standard. Costs and selected payment mechanisms were informed by published sources; their combination, guaranteed dates and finite alternatives are synthetic. See sources and development.

Action P checks the award and host-fee coverage; it imposes no additional deposit-holding requirement. A travel advance and its later settlement refer to the same support and cannot be counted as two receipts. The first month's living cost already includes rent.

Model, hardware and generation

These durations describe the saved run, not GPU rental or billing time. The prior pilot-based estimate, 3.893226176 hours, remains in the development record; actual main generation took 3.634449977 hours. Analysis and transfer are additional.

Each prompt contains six Chinese office notices. The model first gives a short explanation, then one JSON object containing feasibility_claim, actions and reason. Each action has its original ID and an integer day. The feasibility claim concerns the submitted plan through the full year and all required settlements. No executor feedback or repair turn is supplied. The final answer is scored separately from its thinking text. All 504 calls stopped normally; none reached the output cap.

Profiles and cash grid

Twelve new financial profiles were drawn once with random.Random(20261005), using the fixed configuration's field order, including singleton domains. No profile was rejected, redrawn or removed. Each profile is crossed with twenty-one shared cash amounts and both model seeds, giving 504 requests and twenty-four responses per cash amount.

Within a profile, changing B changes no qualification, cost, opportunity, award, date or available route. The common grid is nonuniform: RMB 2,500 intervals between 35,000 and 70,000, with wider intervals outside that range. Aggregate rates describe this grid, not a population distribution of student finances.

Deterministic execution and reference search

The executor checks action IDs, required stages, allowed dates, prerequisite lags and mutually exclusive choices. It then replays the dated cash events, stopping before the first unaffordable payment. Later positive receipts become available early on their specified day; scheduled living payments occur later that day. Immediate effects follow the submitted within-day action order.

A separate exhaustive search enumerates the four permitted route families, allowed dates and within-day orderings. For a valid schedule π, its minimum initial cash is the largest cumulative deficit:

Here Δc is a receipt or payment at an ordered event. The search is exact within the finite supplied action space. R_H is the minimum cash to complete arrival H; R* is the minimum to complete the year. The realised profiles have R_H = RMB 39,700–43,900 and R* = RMB 50,400–60,200.

• B < R_H: no permitted route can fund arrival.
• R_H ≤ B < R:* arrival is affordable, but no permitted full-year route is.
• B ≥ R:* at least one full-year route is affordable; the model's own schedule still needs to be checked.

Measures and denominators

A positive claim is a readable Boolean true. A structurally valid plan meets all noncash requirements. Full execution completes all required actions, settlements and payments. A cash false assurance combines a positive claim, a structurally valid plan and failed cash execution. A failure with B < R* is physically unavoidable under the specified terms; a cash failure with B ≥ R* would be avoidable through another permitted schedule.

Auxiliary interpretations of the full final prose determine whether a positive Boolean is unconditional, withdrawn or conditional on outside funds. Six GPT-6.1-sol MAX agents each interpreted a different set of 84 final answers and reasons. Their 504 annotations are auxiliary model interpretations, not human gold labels or repeated independent annotations. They do not replace the original Boolean fields, responses or deterministic scores. All 311 readable positives were classified as unconditional.

A negative judgment when a reference route exists does not by itself deny every possible route: the judgment is about the submitted plan. An own-plan false negative is a negative judgment about a submitted plan that actually executes. There are four such cases.

Complete results

There are 498 readable Boolean judgments and six interface cases. Of 311 positive claims, 97 are cash false assurances and 26 have structural errors. All 97 cash false assurances occur below R*: no avoidable cash false assurance was observed. Among 348 structurally valid plans, 194 execute. Execution occurs in 194 of 214 reference-feasible requests.

All 142 readable judgments in the first group are negative; its two interface cases are retained. In the middle group, cash false assurances are 97/146 of all responses and 97/107 of positive responses. These denominators answer different questions.

At RMB 50,000, twenty-one responses promise feasibility but none executes: twenty positive plans fail on cash and one on structure. Higher-cash structural and interface failures are retained. No curve is forced to be monotonic.

Descriptive 95% percentile intervals use 5,000 bootstrap resamples of the twelve whole profiles, keeping all cash levels and both seeds within each cluster; bootstrap seed 20261006. The 504 responses are not 504 independent students. Figure 1 uses actual cash spacing; Figure 2 uses equal-width cells for discrete cash categories and separates the two seeds.

A failed payment and a correct warning

Profile 01 and B = RMB 45,000 were selected for illustration before the main profile draw. Inspection of the actual seed-1729 answer found the claim that living-award instalments would leave cash flow uninterrupted. The full request ID is cliff_realistic_dense_v1__cliff_realistic_confirm_01__B45000__D__s1729. Its structurally valid schedule has R(plan) = R* = RMB 55,800. Arrival is day 372. On day 702, a RMB 14,500 living bill faces only RMB 3,700, leaving a RMB 10,800 shortfall. The seed-specific answer was reported after reading it; the reference trajectory in Figure 3 is a separate object.

A correct warning is retained at profile 02 / B42,500 / seed2718. The model acknowledges that total funding balances over the whole period, but rejects on-time execution. Its valid schedule does fail on day 672: RMB 14,500 due, RMB 14,300 available. The answer's approximate budget explanation is distinct from the executor's exact first shortfall.

Twenty-one valid schedules require more cash than the reference optimum. Across valid schedules the median excess is zero and the maximum is RMB 3,400. No cash-only failure was observed where a reference full-year plan was feasible.

Same-total payment-timing replay

The sole replay moves the final 20% from its original departure-settlement receipt to the original AR receipt date. Total support, costs and every original model action and date remain unchanged. DE is still required, without a duplicate payment. The reference is recomputed over the same permitted alternatives; model schedules are replayed as submitted. There are no new model responses.

Of 154 original cash failures, 111 disappear and 43 remain. Of the 97 cash false assurances, 95 disappear and two remain. The 43 remaining failures comprise 41 arrival payments and two later bills that still precede the first receipt.

Those two remaining false assurances are profiles 08 and 12, both at B = RMB 42,500 with seed2718. Bills of RMB 16,500 and 15,500 fall on days 388 and 374, with RMB 1,000 available in each case. Their first receipts arrive on days 393 and 379. Moving the final instalment to those same receipt dates cannot pay the earlier bill.

All twelve reference minima fall to R_H, reductions of RMB 10,700–18,300. This does not imply that every original model schedule becomes affordable.

Figure 3 uses profile 01's original optimal reference schedule at RMB 45,000 and 70,000. These amounts were selected before the main draw from full-domain bounds: R_H = 38,200–45,000 and R* = 48,900–63,300. The initial low illustration amount of 40,000 was changed to 45,000 at that point. Trajectories stop before the first unpaid bill. This diagnostic changes payment conditions; it does not demonstrate improved model behaviour.

Development and scope

The earlier task did not show this cash failure: 4B executed 320/328 reference-feasible requests, with zero cash-only failures among its 320 valid executable schedules. Subsequent development introduced conditional 80/20 releases and clarified payment direction, the scope of protected living costs and alternative actions. Earlier results and the 168-response confirmation remain available.

The realistic-cost version changes several conditions together: fee domains, a complete first living month at H, twelve award standards, the removal of an extra cash hold, a common absolute-cash grid and batch size 12. It is not an unchanged replication or an isolated test of currency scale.

A twelve-response realistic pilot yielded eight unconditional positives, eight structurally valid plans and six completions, including two cash false assurances, one positive structural error and one own-plan false negative. After all twelve pilot answers were read, the main grid was revised once to place denser amounts between 35,000 and 70,000. It retained twenty-one amounts and 504 requests. No main profiles or outputs existed yet. Full development history records the settings, weak results and retained examples.

One checkpoint and twelve synthetic financial variants do not establish real-student error rates, a universal cash threshold, training-data bias or the model's hidden causal reasoning. No model was fine-tuned. Proposed improvements and their evaluation are in theory and proposed improvements.

Reproduction

The commands below are the saved Molab RTX PRO 6000 workflow. Inference, enumeration, scoring, statistics and replay ran remotely. Publication figures were later redrawn locally from saved results, and the report was typeset locally.

cd /marimo/molab_runs/resource_planning
export PYTHONPATH=/marimo/molab_runs/resource_planning/code
export VLLM_USE_FLASHINFER_SAMPLER=0
python code/build_cliff_realistic.py --config work/realistic_dense_config.json --phase main
python code/run_model.py --input data/cliff_realistic_dense_v1/requests.jsonl \
  --output results/cliff_realistic_dense_v1_4b_native.jsonl --model Qwen/Qwen3.5-4B \
  --revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a --thinking on \
  --engine vllm --batch-size 12 --max-tokens 32768 --max-model-len 131072
python code/score_cliff_pilot.py --series cliff_realistic_dense_v1
# After the complete final-answer annotations have been saved:
python code/analyze_realistic_dense.py --series cliff_realistic_dense_v1 \
  --annotations work/realistic_main_semantic_annotations.jsonl

Use the recorded remote dependencies and runtime record. The data guide explains the input, response, score and annotation files. The new English reading copies were prepared after the experiment; all reported model results use the original Chinese requests.