{"version":1,"status":"pre_registered_before_outputs","score_scale":{"0":"Absent or unusable","1":"Major problems","2":"Substantial weaknesses","3":"Adequate with noticeable weaknesses","4":"Strong with minor issues","5":"Excellent for the brief"},"summary_weights":{"factual_faithfulness":35,"decision_relevant_coverage":25,"clarity_organization":20,"natural_prose":15,"brief_compliance":5},"explainer_weights":{"factual_accuracy":30,"clarity_explanation":25,"organization_flow":15,"concrete_useful_detail":15,"natural_voice":10,"brief_compliance":5},"summary_facts":[{"id":"rate","required":true,"fact":"Missed-appointment rate decreased 20% to15%; five percentage points or25% relative. Not25% fewer missed bookings."},{"id":"causality","required":true,"fact":"Descriptive voluntary four-branch before/after pilot; software effect not causally isolated."},{"id":"costs","required":true,"fact":"Pilot total EUR6000 includes EUR4000 setup and EUR2000 six-week subscription; proposed annual twelve-branch subscription EUR12000 excludes unknown training costs; no net-savings calculation."},{"id":"decision","required":true,"fact":"Manager recommends follow-up; board has not approved follow-up or rollout; no agreed rollout date."},{"id":"quality_limits","required":true,"fact":"Incomplete delivery log and unassessed low-vision accessibility; do not imply measured reliability or accessibility."},{"id":"survey","required":false,"fact":"28 of40 respondents prefer new reminder;80 invited.70% is respondent share, not all users."},{"id":"counts","required":false,"fact":"Missed count100 to90 =10% reduction; bookings500 to600 =20% increase."},{"id":"duplicates","required":false,"fact":"Two reported duplicate-reminder cases fixed; this is not a complete failure-rate estimate."},{"id":"next_steps","required":true,"fact":"Proposed eight-week follow-up atsamefour branches, better logs, staff-time measurement, accessibility review; board meeting3September."}],"explainer_fact_guide":{"expected":["Materials store solar energy and release heat after sunset","Less vegetation reduces shade and evaporative cooling","Buildings/urban geometry can affect cooling and ventilation","Heat from vehicles/buildings/air conditioning can contribute","Trees/shade and appropriate reflective/green surfaces can help","Feasibility depends on local conditions and maintenance; distinguish neighborhood mitigation from global climate change"],"sources":["https://www.epa.gov/heatislands/what-are-heat-islands","https://www.epa.gov/heatislands/heat-island-reduction-solutions"],"note":"Sources are evaluator-only, not inserted in model input. This is an accuracy guide, not a mandatory wording or exhaustive feature checklist."},"evaluation_rules":["Grade final content, not hidden reasoning; retain empty/nonfinished outputs as noncompletion.","Report quality dimensions and completion separately; do not collapse subjective quality into an eight-task pass count.","Record each factual error and its severity with output quotation and source/criterion; polished prose cannot erase a major factual error.","Naturalness means readable cadence, specific language, coherent paragraphs and low filler/repetition. It is not an AI-detector score or evidence of human authorship.","Anonymous labels, shuffled order, model identity hidden while judging; judge both dimensions then paired preference with rationale.","Use one documented assistant rubric review initially, explicitly labelled model judgement, not a human study. Human preference from the user is a separate useful signal. No paid/external judging called automatically.","Length adherence affects brief score but is not automatically total failure; brevity is not intrinsically worse than length.","Prefer no raw prose exact-match/ROUGE-only grading; no single ideal reference style.","Major factual-error flag overrides any recommendation to use a high-style answer.","The aggregate is100*sum(weight*dimension_score/5)/100; all raw dimensions must remain visible."],"data_to_report_weights":{"numerical_accuracy":25,"interpretation_causal_caution":30,"decision_usefulness":20,"clarity_natural_prose":20,"brief_compliance":5},"editorial_rewrite_weights":{"factual_preservation":35,"reader_usefulness":20,"tone_natural_prose":25,"organization_concision":15,"brief_compliance":5},"data_to_report_facts":["A overall1100/10000=11%; B overall1350/10000=13.5%: +2.5percentage points, about22.73% relative.","Desktop20% to18%; mobile10% to9%; both within-device rates decline.","Traffic composition changes from10% to50% desktop, the higher-converting group; aggregate improvement does not show within-group improvement.","At A device mix but B device rates, overall would be9.9%; this optional correct illustration must not be called a measured control.","Neither benefit nor harm CAUSED by the new checkout is identified: advertising mix also changed and there is no randomized control.","No revenue/profit/customer-cost conclusion is supported. Recommend controlled experiment or investigation of stratified funnel/source data."],"editorial_rewrite_facts":["Optional beta, current search remains available; target14October is unconfirmed pendingQA.","Same50 queries, not50 people; median7s to5s in internal testing; no universal performance guarantee.","Beta may miss newly uploaded documents for up to15minutes; current search lacks that known delay; neither modifies/deletes documents.","Nothing to do today; email with opt-in instructions when ready; return to current search any time.","No extra cost duringbeta; laterpricing undecided; no invented link/contact."]}