Inside the evaluation data

The evaluated requests and original model prose are Chinese. Every request now has an English reading copy, and every saved outcome has an English-labelled record. These companions were prepared after evaluation. They let readers inspect the task and results without treating translated text as a second model experiment.

Choose a starting point

File map

Paths below are relative to the repository root.

The English reading files keep the original request IDs. Prompt copies retain the six-notice structure and all amounts, action IDs, dates, prerequisite lags, alternatives and cash effects. Their English instructions still describe the evaluated request's requirement to answer in Chinese. No inference was run on these copies.

The English outcome records reproduce machine-readable judgments and scores. They add English action names and category descriptions; they do not translate or replace the model's free-text explanation or thinking. The quoted case study in the application and website includes an English translation of the relevant original claim. Original Chinese prose remains available for checking the wording.

Load the original evaluation requests

from datasets import load_dataset

requests = load_dataset(
    "0721alex/a-plan-is-not-a-path",
    "cliff_realistic_dense_v1",
    split="test",
)
print(len(requests))  # 504
print(requests[0]["prompt"])

To read the English companion without changing the evaluated dataset:

import json
from huggingface_hub import hf_hub_download

path = hf_hub_download(
    repo_id="0721alex/a-plan-is-not-a-path",
    repo_type="dataset",
    filename="data/cliff_realistic_dense_v1/english_reading/requests.en.jsonl",
)
with open(path, encoding="utf-8") as source:
    requests_en = [json.loads(line) for line in source]
print(requests_en[0]["prompt_en"])

Join requests, outcomes, responses and scores on id. The example cliff_realistic_dense_v1__cliff_realistic_confirm_01__B45000__D__s1729 identifies profile 01, initial cash RMB 45,000, six-notice presentation D and seed1729.

Important fields

Action IDs are stable across languages. R prepares research materials; C contacts the host; W requests the campus certificate; L_waiver collects it; L_paid is the paid alternative; A applies; O obtains the conditional invitation and award; GA signs the award agreement. G_advance uses D1 and D2 to obtain a travel advance; G_reimburse registers the reimbursement alternative. P checks coverage, V completes pre-departure administration, T books travel and H arrives. TR_a or TR_r settles travel, AR confirms arrival, F completes the year, and the corresponding DE_a or DE_r completes final settlement. The full example gives every date and condition.

Earlier experiments and records

The development history covers the original task, each clarification round, the 168-response confirmation, the realistic pilot and the current experiment. Original files remain under their series names in data/, results/ and outputs/. They are retained as historical records and are not pooled with the current experiment.

For exact replay, use the original Chinese requests and the saved configuration, not the reading copies. The original prose, prompts and logs preserve the experiment's language. The numerical schema, English guides and reading files provide the international reader's entry points.

Licence and attribution

Original synthetic benchmark data are CC BY 4.0. Original project code is MIT. The Qwen checkpoint, cited sources and fonts retain their respective terms; see licensing and attribution. No model weights are duplicated in this repository.