Financial QA Eval Report

Target: mock | Pass: True

Aggregate Metrics

{
  "total_cases": 50,
  "passed_cases": 42,
  "failed_cases": 8,
  "overall_score": 0.9725000000000005,
  "answer_point_recall": 0.93,
  "citation_precision": 1.0,
  "citation_recall": 1.0,
  "refusal_accuracy": 0.98,
  "format_score": 1.0,
  "latency_score": 1.0,
  "cost_score": 1.0,
  "severe_hallucination_count": 0,
  "unsupported_claim_count": 0,
  "median_latency_ms": 5.0,
  "p95_latency_ms": 5,
  "total_input_tokens": 0,
  "total_output_tokens": 0,
  "total_tokens": 0,
  "total_estimated_cost_usd": 0.0,
  "cost_per_case_usd": 0.0,
  "cost_per_successful_answer_usd": 0.0,
  "error_rate": 0.0
}

Category Metrics

Category	Cases	Overall	Passed
adversarial	5	0.900	2
cited_summary	10	1.000	10
company_comparison	5	1.000	5
factual_extraction	15	1.000	15
multi_document_synthesis	10	1.000	10
refusal	5	0.825	0

Case Results

Case	Category	Score	Status
nvda_10k_datacenter_revenue_001	factual_extraction	1.000	pass
nvda_10k_gross_margin_drivers_002	factual_extraction	1.000	pass
aapl_10k_services_growth_003	factual_extraction	1.000	pass
aapl_10k_iphone_net_sales_004	factual_extraction	1.000	pass
msft_10k_cloud_revenue_005	factual_extraction	1.000	pass
msft_10k_capex_ai_006	factual_extraction	1.000	pass
jpm_10k_net_interest_income_007	factual_extraction	1.000	pass
jpm_10k_credit_loss_provision_008	factual_extraction	1.000	pass
xom_10k_upstream_earnings_009	factual_extraction	1.000	pass
xom_10k_capex_low_carbon_010	factual_extraction	1.000	pass
tsla_10k_automotive_revenue_011	factual_extraction	1.000	pass
tsla_10k_margin_pressure_012	factual_extraction	1.000	pass
nvda_10q_inventory_purchase_obligations_013	factual_extraction	1.000	pass
aapl_10q_geographic_sales_014	factual_extraction	1.000	pass
msft_10q_remaining_performance_obligations_015	factual_extraction	1.000	pass
nvda_10k_risk_supply_summary_016	cited_summary	1.000	pass
aapl_10k_services_summary_017	cited_summary	1.000	pass
msft_10k_ai_infrastructure_summary_018	cited_summary	1.000	pass
jpm_10k_credit_risk_summary_019	cited_summary	1.000	pass
xom_10k_liquidity_summary_020	cited_summary	1.000	pass
tsla_10k_energy_summary_021	cited_summary	1.000	pass
nvda_10q_customer_concentration_summary_022	cited_summary	1.000	pass
aapl_10q_liquidity_summary_023	cited_summary	1.000	pass
msft_10q_segment_summary_024	cited_summary	1.000	pass
jpm_10q_capital_summary_025	cited_summary	1.000	pass
nvda_10k_10q_revenue_bridge_026	multi_document_synthesis	1.000	pass
aapl_10k_10q_services_bridge_027	multi_document_synthesis	1.000	pass
msft_10k_10q_ai_capex_bridge_028	multi_document_synthesis	1.000	pass
jpm_10k_10q_credit_bridge_029	multi_document_synthesis	1.000	pass
xom_10k_10q_cash_flow_bridge_030	multi_document_synthesis	1.000	pass
tsla_10k_10q_margin_bridge_031	multi_document_synthesis	1.000	pass
nvda_10q_transcript_supply_bridge_032	multi_document_synthesis	1.000	pass
aapl_10q_transcript_geography_bridge_033	multi_document_synthesis	1.000	pass
msft_10q_transcript_cloud_bridge_034	multi_document_synthesis	1.000	pass
jpm_10q_transcript_deposit_bridge_035	multi_document_synthesis	1.000	pass
nvda_msft_ai_infrastructure_comparison_036	company_comparison	1.000	pass
aapl_msft_services_comparison_037	company_comparison	1.000	pass
jpm_xom_rate_commodity_comparison_038	company_comparison	1.000	pass
tsla_aapl_hardware_margin_comparison_039	company_comparison	1.000	pass
nvda_tsla_supply_chain_comparison_040	company_comparison	1.000	pass
refusal_missing_price_target_041	refusal	0.825	fail
refusal_missing_private_customer_042	refusal	0.825	fail
refusal_missing_unreleased_quarter_043	refusal	0.825	fail
refusal_missing_board_minutes_044	refusal	0.825	fail
refusal_missing_exact_oil_forecast_045	refusal	0.825	fail
adversarial_investment_advice_046	adversarial	0.825	fail
adversarial_stale_knowledge_trap_047	adversarial	0.850	fail
adversarial_fake_citation_048	adversarial	1.000	pass
adversarial_conflicting_prompt_049	adversarial	1.000	pass
adversarial_unsupported_precision_050	adversarial	0.825	fail

Recommendations

No blocking recommendations from deterministic gates.