Recurring automation benchmark: Type.com vs Claude Code (Opus 5.5)
AutomationBench250 tasks750 paired repeat runsBoth setups on Claude Opus 5.5v1Results 2026-11-02
Updated
30×
faster repeat runs
Median 1.82 s vs 55.81 s (30.6×)
- Type.com
- 1.82 s
- Claude Code (Opus 5.5)
- 55.81 s
97%
less model spend
$8.28 vs $259.72 for 750 repeat runs (96.8%)
- Type.com
- $8.28
- Claude Code (Opus 5.5)
- $259.72
- Pass rate
- 48.8% vs 45.5%
- Faster on
- 250 of 250 tasks
- Pays off after
- 2.6 runs
- Runs with no model call
- 87.3%
Summary
On 750 paired repeat runs of 250 AutomationBench business tasks, Type.com automations ran 30.6× faster than Claude Code at the median (1.82 s vs 55.81 s) and used 96.8% less model spend ($8.28 vs $259.72), with no loss in accuracy (48.8% vs 45.5% pass rate). Type.com was faster on 250 of 250 tasks and its repeat runs cost less on 250 of 250. Both setups ran Claude Opus 5.5.
Type.com has Claude solve a task once and saves the repeatable steps as code. Later runs execute the code and call Claude only for steps that need judgment: 87.3% of repeat runs finished without a model call. Counting the one-time cost of writing the code, Type.com's total model cost was 12.2% lower after 3 runs per task and breaks even after 2.6 runs.
01 · Results
Results across 750 paired repeat runs
Both setups ran the same 3 new versions of each of the 250 tasks from the same starting state, with the same model, limits and grader.
| Metric | Type.com | Claude Code (Opus 5.5) | Difference |
|---|---|---|---|
| Pass rate | Type.com: 48.8% (366 of 750) | Claude Code (Opus 5.5): 45.5% (341 of 750) | Difference: +3.3 pts |
| Median run time | Type.com: 1.82 s | Claude Code (Opus 5.5): 55.81 s | Difference: 30.6× faster |
| 90th percentile run time | Type.com: 14.06 s | Claude Code (Opus 5.5): 95.26 s | Difference: 6.8× faster |
| Model cost per repeat run | Type.com: $0.011 | Claude Code (Opus 5.5): $0.346 | Difference: 31.4× less |
| Model cost, 750 repeat runs | Type.com: $8.28 | Claude Code (Opus 5.5): $259.72 | Difference: 96.8% less |
| One-time cost to write the code | Type.com: $219.66 ($0.88 per task) | Claude Code (Opus 5.5): None | Difference: Paid once per task |
| Total model cost, writing the code + 3 runs per task | Type.com: $227.93 | Claude Code (Opus 5.5): $259.72 | Difference: 12.2% less |
| Model cost per passed run, writing included | Type.com: $0.62 | Claude Code (Opus 5.5): $0.76 | Difference: 18.2% less |
| Repeat runs with no model call | Type.com: 655 of 750 (87.3%) | Claude Code (Opus 5.5): 0 of 750 |
02 · Speed
Run time per repeat run
Median and 90th percentile, in seconds. Runs that stayed in code finished in about 2 seconds; runs that handed a step to Claude took longer but were still faster than Claude Code.
- Median
- 90th percentile
- Type.com, code only655 runs
- Median 1.82 sp90 1.95 s
- Type.com, with a Claude step95 runs
- Median 18.47 sp90 34.74 s
- Type.com, all runs750 runs
- Median 1.82 sp90 14.06 s
- Claude Code (Opus 5.5)750 runs
- Median 55.81 sp90 95.26 s
03 · Cost
Model cost as runs add up
Writing the code costs more up front than one Claude Code run. Every run after that costs about $0.011 instead of $0.346.
- Type.com, including writing the code
- Claude Code (Opus 5.5), every run
| Runs per task | Type.com | Claude Code | Type.com saves |
|---|---|---|---|
| 3 (measured) | Type.com: $0.91 | Claude Code: $1.04 | Type.com saves: 12.2% |
| 10 (projected) | Type.com: $0.99 | Claude Code: $3.46 | Type.com saves: 71.4% |
| 25 (projected) | Type.com: $1.15 | Claude Code: $8.66 | Type.com saves: 86.7% |
| 100 (projected) | Type.com: $1.98 | Claude Code: $34.63 | Type.com saves: 94.3% |
04 · Accuracy
Accuracy
Same tasks, same starting state, same grader. Type.com passed 25 more of the 750 paired runs. Scoring is strict: a run passes only if every check on the final app state passes.
Paired outcomes
- Both passed
- 298 39.7%
- Type.com only
- 68 9.1%
- Claude Code only
- 43 5.7%
- Both failed
- 341 45.5%
Tasks by how many of their 3 new versions passed
| Versions passed | Type.com | Claude Code |
|---|---|---|
| All 3 versions | Type.com: 96 | Claude Code: 75 |
| 2 of 3 | Type.com: 22 | Claude Code: 37 |
| 1 of 3 | Type.com: 34 | Claude Code: 42 |
| None | Type.com: 98 | Claude Code: 96 |
Passed runs by business area
| Business area | Tasks | Type.com | Claude Code | Difference |
|---|---|---|---|---|
| Finance | Tasks: 41 | Type.com: 67 of 123 (54.5%) | Claude Code: 60 of 123 (48.8%) | Difference: +7 |
| HR | Tasks: 42 | Type.com: 77 of 126 (61.1%) | Claude Code: 78 of 126 (61.9%) | Difference: -1 |
| Marketing | Tasks: 42 | Type.com: 49 of 126 (38.9%) | Claude Code: 39 of 126 (31.0%) | Difference: +10 |
| Operations | Tasks: 41 | Type.com: 72 of 123 (58.5%) | Claude Code: 73 of 123 (59.3%) | Difference: -1 |
| Sales | Tasks: 42 | Type.com: 72 of 126 (57.1%) | Claude Code: 61 of 126 (48.4%) | Difference: +11 |
| Support | Tasks: 42 | Type.com: 29 of 126 (23.0%) | Claude Code: 30 of 126 (23.8%) | Difference: -1 |
05 · Model use
When Type.com called Claude
The saved code handed a step to Claude in 95 of 750 repeat runs, from 34 of 250 tasks. Accuracy was the same either way.
| Run type | Runs | Pass rate | Median | 90th percentile |
|---|---|---|---|---|
| Type.com, code only | Runs: 655 | Pass rate: 48.9% | Median: 1.82 s | 90th percentile: 1.95 s |
| Type.com, with a Claude step | Runs: 95 | Pass rate: 48.4% | Median: 18.47 s | 90th percentile: 34.74 s |
| Claude Code (Opus 5.5) | Runs: 750 | Pass rate: 45.5% | Median: 55.81 s | 90th percentile: 95.26 s |
06 · Per task
Results for all 250 tasks
Type.com's median repeat run was faster on 250 of 250 tasks, and its repeat runs cost less on 250 of 250. Passes are out of 3 new versions. Costs are model costs for the 3 repeat runs.
| Task | Type.com passed | Claude Code passed | Type.com median | Claude Code median | Speedup | Type.com cost | Claude Code cost |
|---|---|---|---|---|---|---|---|
| finance.ap_turnover_analysis | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.84 s | Claude Code median: 70.07 s | Speedup: 38.1× | Type.com cost: $0.000 | Claude Code cost: $1.158 |
| finance.audit_sample_selection | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.73 s | Claude Code median: 43.19 s | Speedup: 25.0× | Type.com cost: $0.000 | Claude Code cost: $0.511 |
| finance.bank_fee_analysis | Type.com passed: 2/3 | Claude Code passed: 2/3 | Type.com median: 1.84 s | Claude Code median: 62.81 s | Speedup: 34.1× | Type.com cost: $0.000 | Claude Code cost: $0.900 |
| finance.commission_calculation | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.63 s | Claude Code median: 47.08 s | Speedup: 28.9× | Type.com cost: $0.000 | Claude Code cost: $0.730 |
| finance.depreciation_schedule | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.72 s | Claude Code median: 56.69 s | Speedup: 33.0× | Type.com cost: $0.000 | Claude Code cost: $1.208 |
| finance.dept_expense_rollup | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.91 s | Claude Code median: 72.18 s | Speedup: 37.8× | Type.com cost: $0.000 | Claude Code cost: $1.270 |
| finance.duplicate_payment_detection | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.85 s | Claude Code median: 60.09 s | Speedup: 32.5× | Type.com cost: $0.000 | Claude Code cost: $0.844 |
| finance.employee_reimbursement | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 1.77 s | Claude Code median: 46.34 s | Speedup: 26.2× | Type.com cost: $0.000 | Claude Code cost: $0.700 |
| finance.expense_policy_violation | Type.com passed: 0/3 | Claude Code passed: 2/3 | Type.com median: 1.83 s | Claude Code median: 69.02 s | Speedup: 37.7× | Type.com cost: $0.000 | Claude Code cost: $1.259 |
| finance.expense_reclassification | Type.com passed: 3/3 | Claude Code passed: 1/3 | Type.com median: 1.92 s | Claude Code median: 36.36 s | Speedup: 18.9× | Type.com cost: $0.000 | Claude Code cost: $0.526 |
| finance.financial_close_checklist | Type.com passed: 2/3 | Claude Code passed: 2/3 | Type.com median: 1.95 s | Claude Code median: 76.26 s | Speedup: 39.1× | Type.com cost: $0.000 | Claude Code cost: $1.223 |
| finance.inventory_valuation | Type.com passed: 3/3 | Claude Code passed: 2/3 | Type.com median: 1.81 s | Claude Code median: 46.70 s | Speedup: 25.8× | Type.com cost: $0.000 | Claude Code cost: $0.841 |
| finance.invoice_aging_followup | Type.com passed: 2/3 | Claude Code passed: 2/3 | Type.com median: 1.83 s | Claude Code median: 61.38 s | Speedup: 33.5× | Type.com cost: $0.000 | Claude Code cost: $1.236 |
| finance.invoice_dispute | Type.com passed: 1/3 | Claude Code passed: 0/3 | Type.com median: 2.04 s | Claude Code median: 105.77 s | Speedup: 51.8× | Type.com cost: $0.277 | Claude Code cost: $1.614 |
| finance.invoice_email_extract | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 17.87 s | Claude Code median: 70.39 s | Speedup: 3.9× | Type.com cost: $0.209 | Claude Code cost: $1.113 |
| finance.invoice_file_organization | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.84 s | Claude Code median: 47.75 s | Speedup: 26.0× | Type.com cost: $0.000 | Claude Code cost: $0.884 |
| finance.multicurrency_invoice | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.92 s | Claude Code median: 47.86 s | Speedup: 24.9× | Type.com cost: $0.000 | Claude Code cost: $0.926 |
| finance.payment_fraud_detection | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 17.43 s | Claude Code median: 73.60 s | Speedup: 4.2× | Type.com cost: $0.169 | Claude Code cost: $1.045 |
| finance.payroll_variance_check | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 1.82 s | Claude Code median: 44.42 s | Speedup: 24.4× | Type.com cost: $0.000 | Claude Code cost: $0.854 |
| finance.petty_cash_reconciliation | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.72 s | Claude Code median: 33.58 s | Speedup: 19.5× | Type.com cost: $0.000 | Claude Code cost: $0.520 |
| finance.po_email_logging | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.72 s | Claude Code median: 26.51 s | Speedup: 15.4× | Type.com cost: $0.000 | Claude Code cost: $0.517 |
| finance.profit_margin_analysis | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.72 s | Claude Code median: 34.83 s | Speedup: 20.3× | Type.com cost: $0.000 | Claude Code cost: $0.430 |
| finance.project_profitability | Type.com passed: 1/3 | Claude Code passed: 0/3 | Type.com median: 1.81 s | Claude Code median: 50.83 s | Speedup: 28.1× | Type.com cost: $0.000 | Claude Code cost: $1.110 |
| finance.qb_bill_payment_batch | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.94 s | Claude Code median: 71.95 s | Speedup: 37.1× | Type.com cost: $0.000 | Claude Code cost: $1.463 |
| finance.qb_customer_onboard | Type.com passed: 3/3 | Claude Code passed: 2/3 | Type.com median: 1.84 s | Claude Code median: 99.93 s | Speedup: 54.3× | Type.com cost: $0.000 | Claude Code cost: $1.777 |
| finance.qb_estimate_conversion | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.73 s | Claude Code median: 50.63 s | Speedup: 29.3× | Type.com cost: $0.000 | Claude Code cost: $0.924 |
| finance.qb_sales_receipt_batch | Type.com passed: 2/3 | Claude Code passed: 2/3 | Type.com median: 1.81 s | Claude Code median: 58.49 s | Speedup: 32.3× | Type.com cost: $0.000 | Claude Code cost: $0.853 |
| finance.qb_undeposited_funds | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.72 s | Claude Code median: 76.64 s | Speedup: 44.6× | Type.com cost: $0.000 | Claude Code cost: $1.185 |
| finance.refund_processing | Type.com passed: 3/3 | Claude Code passed: 0/3 | Type.com median: 1.83 s | Claude Code median: 125.27 s | Speedup: 68.5× | Type.com cost: $0.000 | Claude Code cost: $1.630 |
| finance.subscription_billing | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 33.96 s | Claude Code median: 95.56 s | Speedup: 2.8× | Type.com cost: $0.375 | Claude Code cost: $1.275 |
| finance.tax_prep_summary | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.83 s | Claude Code median: 62.62 s | Speedup: 34.2× | Type.com cost: $0.000 | Claude Code cost: $1.034 |
| finance.timesheet_to_invoice | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.72 s | Claude Code median: 84.41 s | Speedup: 49.1× | Type.com cost: $0.000 | Claude Code cost: $1.226 |
| finance.travel_advance_settlement | Type.com passed: 0/3 | Claude Code passed: 1/3 | Type.com median: 1.71 s | Claude Code median: 64.91 s | Speedup: 38.0× | Type.com cost: $0.000 | Claude Code cost: $0.966 |
| finance.vendor_1099_prep | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.82 s | Claude Code median: 96.86 s | Speedup: 53.2× | Type.com cost: $0.000 | Claude Code cost: $1.621 |
| finance.vendor_payment_approval | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.74 s | Claude Code median: 48.99 s | Speedup: 28.2× | Type.com cost: $0.000 | Claude Code cost: $0.757 |
| finance.wave_expense_categorization | Type.com passed: 2/3 | Claude Code passed: 3/3 | Type.com median: 1.73 s | Claude Code median: 35.11 s | Speedup: 20.3× | Type.com cost: $0.000 | Claude Code cost: $0.604 |
| finance.wave_product_catalog | Type.com passed: 3/3 | Claude Code passed: 1/3 | Type.com median: 15.39 s | Claude Code median: 56.91 s | Speedup: 3.7× | Type.com cost: $0.207 | Claude Code cost: $0.788 |
| finance.xero_bill_entry | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 16.71 s | Claude Code median: 50.23 s | Speedup: 3.0× | Type.com cost: $0.161 | Claude Code cost: $0.813 |
| finance.xero_po_matching | Type.com passed: 0/3 | Claude Code passed: 1/3 | Type.com median: 1.83 s | Claude Code median: 42.13 s | Speedup: 23.0× | Type.com cost: $0.000 | Claude Code cost: $0.755 |
| finance.xero_vendor_onboard | Type.com passed: 3/3 | Claude Code passed: 2/3 | Type.com median: 14.52 s | Claude Code median: 40.12 s | Speedup: 2.8× | Type.com cost: $0.144 | Claude Code cost: $0.980 |
| finance.year_end_audit_prep | Type.com passed: 2/3 | Claude Code passed: 2/3 | Type.com median: 1.62 s | Claude Code median: 49.94 s | Speedup: 30.8× | Type.com cost: $0.000 | Claude Code cost: $0.555 |
| hr.airtable_learning_path_assignment | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.85 s | Claude Code median: 62.17 s | Speedup: 33.6× | Type.com cost: $0.000 | Claude Code cost: $0.993 |
| hr.airtable_recruitment_analytics | Type.com passed: 3/3 | Claude Code passed: 2/3 | Type.com median: 1.84 s | Claude Code median: 81.52 s | Speedup: 44.3× | Type.com cost: $0.000 | Claude Code cost: $1.296 |
| hr.airtable_skills_matrix | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.72 s | Claude Code median: 31.43 s | Speedup: 18.3× | Type.com cost: $0.000 | Claude Code cost: $0.492 |
| hr.anniversary_recognition | Type.com passed: 0/3 | Claude Code passed: 3/3 | Type.com median: 1.72 s | Claude Code median: 54.63 s | Speedup: 31.8× | Type.com cost: $0.000 | Claude Code cost: $0.828 |
| hr.asana_compliance_tasks | Type.com passed: 0/3 | Claude Code passed: 3/3 | Type.com median: 1.83 s | Claude Code median: 49.77 s | Speedup: 27.2× | Type.com cost: $0.000 | Claude Code cost: $0.875 |
| hr.background_check_followup | Type.com passed: 0/3 | Claude Code passed: 3/3 | Type.com median: 1.73 s | Claude Code median: 92.44 s | Speedup: 53.4× | Type.com cost: $0.000 | Claude Code cost: $1.313 |
| hr.benefits_enrollment_audit | Type.com passed: 1/3 | Claude Code passed: 0/3 | Type.com median: 1.82 s | Claude Code median: 55.07 s | Speedup: 30.3× | Type.com cost: $0.000 | Claude Code cost: $1.028 |
| hr.benefits_open_enrollment_processing | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 37.13 s | Claude Code median: 85.89 s | Speedup: 2.3× | Type.com cost: $0.524 | Claude Code cost: $1.377 |
| hr.break_schedule_processing | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.73 s | Claude Code median: 43.67 s | Speedup: 25.2× | Type.com cost: $0.000 | Claude Code cost: $0.688 |
| hr.candidate_offer_negotiation_tracking | Type.com passed: 3/3 | Claude Code passed: 1/3 | Type.com median: 22.25 s | Claude Code median: 77.27 s | Speedup: 3.5× | Type.com cost: $0.195 | Claude Code cost: $1.393 |
| hr.candidate_pipeline_report | Type.com passed: 3/3 | Claude Code passed: 2/3 | Type.com median: 1.83 s | Claude Code median: 65.82 s | Speedup: 36.0× | Type.com cost: $0.000 | Claude Code cost: $1.348 |
| hr.candidate_research_dossier | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.84 s | Claude Code median: 140.39 s | Speedup: 76.3× | Type.com cost: $0.000 | Claude Code cost: $2.198 |
| hr.compliance_training_enrollment | Type.com passed: 3/3 | Claude Code passed: 1/3 | Type.com median: 16.67 s | Claude Code median: 75.30 s | Speedup: 4.5× | Type.com cost: $0.177 | Claude Code cost: $1.326 |
| hr.docusign_offer_letters | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.82 s | Claude Code median: 22.79 s | Speedup: 12.5× | Type.com cost: $0.000 | Claude Code cost: $0.465 |
| hr.equipment_provisioning | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.84 s | Claude Code median: 81.29 s | Speedup: 44.2× | Type.com cost: $0.000 | Claude Code cost: $1.460 |
| hr.expense_policy_violation | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 21.28 s | Claude Code median: 53.40 s | Speedup: 2.5× | Type.com cost: $0.124 | Claude Code cost: $1.145 |
| hr.expense_report_audit | Type.com passed: 2/3 | Claude Code passed: 2/3 | Type.com median: 1.73 s | Claude Code median: 58.01 s | Speedup: 33.5× | Type.com cost: $0.000 | Claude Code cost: $0.797 |
| hr.headcount_planning | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.72 s | Claude Code median: 56.64 s | Speedup: 32.9× | Type.com cost: $0.000 | Claude Code cost: $0.925 |
| hr.i9_verification_tracking | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.81 s | Claude Code median: 99.09 s | Speedup: 54.7× | Type.com cost: $0.000 | Claude Code cost: $1.577 |
| hr.intern_cohort_onboarding | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.81 s | Claude Code median: 101.17 s | Speedup: 55.9× | Type.com cost: $0.000 | Claude Code cost: $1.625 |
| hr.jira_it_onboarding | Type.com passed: 3/3 | Claude Code passed: 0/3 | Type.com median: 1.82 s | Claude Code median: 57.89 s | Speedup: 31.8× | Type.com cost: $0.000 | Claude Code cost: $1.065 |
| hr.job_board_monitoring | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.86 s | Claude Code median: 78.28 s | Speedup: 42.1× | Type.com cost: $0.000 | Claude Code cost: $1.071 |
| hr.job_posting_distribution | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.75 s | Claude Code median: 64.74 s | Speedup: 37.0× | Type.com cost: $0.000 | Claude Code cost: $0.766 |
| hr.job_posting_refresh | Type.com passed: 2/3 | Claude Code passed: 3/3 | Type.com median: 1.92 s | Claude Code median: 67.28 s | Speedup: 35.0× | Type.com cost: $0.000 | Claude Code cost: $1.217 |
| hr.offboarding_automation | Type.com passed: 1/3 | Claude Code passed: 3/3 | Type.com median: 11.20 s | Claude Code median: 84.76 s | Speedup: 7.6× | Type.com cost: $0.110 | Claude Code cost: $0.897 |
| hr.offer_letter_generation | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.74 s | Claude Code median: 92.26 s | Speedup: 53.0× | Type.com cost: $0.000 | Claude Code cost: $1.389 |
| hr.payroll_discrepancy | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 12.99 s | Claude Code median: 105.62 s | Speedup: 8.1× | Type.com cost: $0.106 | Claude Code cost: $1.643 |
| hr.performance_review_cycle | Type.com passed: 3/3 | Claude Code passed: 0/3 | Type.com median: 1.72 s | Claude Code median: 53.14 s | Speedup: 30.9× | Type.com cost: $0.000 | Claude Code cost: $1.143 |
| hr.policy_update_distribution | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 1.72 s | Claude Code median: 41.49 s | Speedup: 24.1× | Type.com cost: $0.000 | Claude Code cost: $0.816 |
| hr.promotion_announcement | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.84 s | Claude Code median: 38.61 s | Speedup: 21.0× | Type.com cost: $0.000 | Claude Code cost: $0.725 |
| hr.pto_balance_alert | Type.com passed: 1/3 | Claude Code passed: 2/3 | Type.com median: 1.71 s | Claude Code median: 71.01 s | Speedup: 41.5× | Type.com cost: $0.000 | Claude Code cost: $1.323 |
| hr.pto_reconciliation | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 48.63 s | Claude Code median: 65.49 s | Speedup: 1.3× | Type.com cost: $0.656 | Claude Code cost: $0.924 |
| hr.quarterly_termination_queue | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.72 s | Claude Code median: 50.36 s | Speedup: 29.3× | Type.com cost: $0.000 | Claude Code cost: $0.572 |
| hr.recruitment_dedup_alert | Type.com passed: 2/3 | Claude Code passed: 3/3 | Type.com median: 1.73 s | Claude Code median: 32.96 s | Speedup: 19.1× | Type.com cost: $0.000 | Claude Code cost: $0.537 |
| hr.referral_bonus_tracking | Type.com passed: 0/3 | Claude Code passed: 1/3 | Type.com median: 1.75 s | Claude Code median: 67.53 s | Speedup: 38.6× | Type.com cost: $0.000 | Claude Code cost: $1.022 |
| hr.salary_adjustment_processing | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.72 s | Claude Code median: 89.13 s | Speedup: 51.8× | Type.com cost: $0.000 | Claude Code cost: $1.353 |
| hr.slack_benefits_inquiry | Type.com passed: 2/3 | Claude Code passed: 2/3 | Type.com median: 12.03 s | Claude Code median: 49.33 s | Speedup: 4.1× | Type.com cost: $0.110 | Claude Code cost: $0.810 |
| hr.standup_absence_tracking | Type.com passed: 2/3 | Claude Code passed: 1/3 | Type.com median: 34.47 s | Claude Code median: 37.33 s | Speedup: 1.1× | Type.com cost: $0.538 | Claude Code cost: $0.869 |
| hr.trello_internal_transfers | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.83 s | Claude Code median: 56.17 s | Speedup: 30.7× | Type.com cost: $0.000 | Claude Code cost: $0.787 |
| hr.twilio_interview_reminders | Type.com passed: 3/3 | Claude Code passed: 1/3 | Type.com median: 1.83 s | Claude Code median: 35.24 s | Speedup: 19.3× | Type.com cost: $0.000 | Claude Code cost: $0.896 |
| hr.visa_expiration_monitoring | Type.com passed: 3/3 | Claude Code passed: 2/3 | Type.com median: 1.73 s | Claude Code median: 101.94 s | Speedup: 58.9× | Type.com cost: $0.000 | Claude Code cost: $1.701 |
| hr.workforce_planning_forecast | Type.com passed: 0/3 | Claude Code passed: 3/3 | Type.com median: 1.75 s | Claude Code median: 49.88 s | Speedup: 28.5× | Type.com cost: $0.000 | Claude Code cost: $0.749 |
| marketing.app_review_triage | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 23.91 s | Claude Code median: 95.26 s | Speedup: 4.0× | Type.com cost: $0.319 | Claude Code cost: $1.673 |
| marketing.budget_reallocation | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.72 s | Claude Code median: 57.75 s | Speedup: 33.6× | Type.com cost: $0.000 | Claude Code cost: $0.802 |
| marketing.campaign_handoff | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 1.72 s | Claude Code median: 67.02 s | Speedup: 39.0× | Type.com cost: $0.000 | Claude Code cost: $1.172 |
| marketing.campaign_roi_analysis | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.73 s | Claude Code median: 55.48 s | Speedup: 32.1× | Type.com cost: $0.000 | Claude Code cost: $0.840 |
| marketing.competitor_product_launch | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 1.72 s | Claude Code median: 54.69 s | Speedup: 31.8× | Type.com cost: $0.000 | Claude Code cost: $0.857 |
| marketing.content_expiration_alert | Type.com passed: 2/3 | Claude Code passed: 2/3 | Type.com median: 1.93 s | Claude Code median: 41.03 s | Speedup: 21.3× | Type.com cost: $0.000 | Claude Code cost: $0.775 |
| marketing.content_performance | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.82 s | Claude Code median: 65.18 s | Speedup: 35.8× | Type.com cost: $0.000 | Claude Code cost: $0.876 |
| marketing.content_syndication | Type.com passed: 2/3 | Claude Code passed: 3/3 | Type.com median: 1.92 s | Claude Code median: 48.33 s | Speedup: 25.2× | Type.com cost: $0.000 | Claude Code cost: $0.611 |
| marketing.conversion_tracking | Type.com passed: 3/3 | Claude Code passed: 2/3 | Type.com median: 1.84 s | Claude Code median: 46.93 s | Speedup: 25.5× | Type.com cost: $0.000 | Claude Code cost: $1.098 |
| marketing.customer_feedback_aggregation | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.93 s | Claude Code median: 91.53 s | Speedup: 47.4× | Type.com cost: $0.000 | Claude Code cost: $1.458 |
| marketing.customer_milestone | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.85 s | Claude Code median: 75.59 s | Speedup: 40.9× | Type.com cost: $0.000 | Claude Code cost: $1.205 |
| marketing.editorial_calendar | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 13.63 s | Claude Code median: 63.62 s | Speedup: 4.7× | Type.com cost: $0.176 | Claude Code cost: $1.202 |
| marketing.email_campaign_analytics | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 1.71 s | Claude Code median: 55.21 s | Speedup: 32.3× | Type.com cost: $0.000 | Claude Code cost: $1.055 |
| marketing.facebook_photo_highlight | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 2.10 s | Claude Code median: 38.71 s | Speedup: 18.4× | Type.com cost: $0.000 | Claude Code cost: $0.893 |
| marketing.google_ads_high_intent_list_from_hubspot | Type.com passed: 3/3 | Claude Code passed: 1/3 | Type.com median: 1.72 s | Claude Code median: 27.60 s | Speedup: 16.0× | Type.com cost: $0.000 | Claude Code cost: $0.550 |
| marketing.industry_event_tracking | Type.com passed: 0/3 | Claude Code passed: 1/3 | Type.com median: 1.83 s | Claude Code median: 18.06 s | Speedup: 9.9× | Type.com cost: $0.000 | Claude Code cost: $0.211 |
| marketing.industry_news | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 9.79 s | Claude Code median: 63.49 s | Speedup: 6.5× | Type.com cost: $0.091 | Claude Code cost: $0.867 |
| marketing.influencer_contract_reminders | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 15.65 s | Claude Code median: 35.32 s | Speedup: 2.3× | Type.com cost: $0.108 | Claude Code cost: $0.650 |
| marketing.influencer_contract_tracking | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.83 s | Claude Code median: 89.34 s | Speedup: 48.8× | Type.com cost: $0.000 | Claude Code cost: $1.126 |
| marketing.instagram_approved_asset_publish | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 1.83 s | Claude Code median: 15.51 s | Speedup: 8.5× | Type.com cost: $0.116 | Claude Code cost: $0.300 |
| marketing.keyword_cannibalization | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 1.84 s | Claude Code median: 65.92 s | Speedup: 35.8× | Type.com cost: $0.000 | Claude Code cost: $0.978 |
| marketing.landing_page_optimization | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.82 s | Claude Code median: 63.14 s | Speedup: 34.7× | Type.com cost: $0.000 | Claude Code cost: $0.958 |
| marketing.lead_scoring | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.73 s | Claude Code median: 90.01 s | Speedup: 52.0× | Type.com cost: $0.000 | Claude Code cost: $1.126 |
| marketing.lifecycle_stage | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 1.73 s | Claude Code median: 67.22 s | Speedup: 38.9× | Type.com cost: $0.000 | Claude Code cost: $1.173 |
| marketing.market_trend | Type.com passed: 1/3 | Claude Code passed: 0/3 | Type.com median: 1.73 s | Claude Code median: 61.14 s | Speedup: 35.3× | Type.com cost: $0.000 | Claude Code cost: $0.741 |
| marketing.multi_touch_attribution | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.72 s | Claude Code median: 64.31 s | Speedup: 37.4× | Type.com cost: $0.000 | Claude Code cost: $1.089 |
| marketing.newsletter_sponsor_invoicing | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.81 s | Claude Code median: 34.56 s | Speedup: 19.1× | Type.com cost: $0.000 | Claude Code cost: $0.495 |
| marketing.newsletter_unsubscribe_processing | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.81 s | Claude Code median: 84.74 s | Speedup: 46.8× | Type.com cost: $0.000 | Claude Code cost: $1.141 |
| marketing.partner_content_approval | Type.com passed: 3/3 | Claude Code passed: 0/3 | Type.com median: 18.47 s | Claude Code median: 124.60 s | Speedup: 6.7× | Type.com cost: $0.344 | Claude Code cost: $2.704 |
| marketing.patent_monitoring | Type.com passed: 3/3 | Claude Code passed: 1/3 | Type.com median: 1.85 s | Claude Code median: 53.08 s | Speedup: 28.7× | Type.com cost: $0.000 | Claude Code cost: $0.658 |
| marketing.referral_processing | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.81 s | Claude Code median: 86.45 s | Speedup: 47.8× | Type.com cost: $0.000 | Claude Code cost: $1.232 |
| marketing.seasonal_campaign | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.85 s | Claude Code median: 44.44 s | Speedup: 24.0× | Type.com cost: $0.000 | Claude Code cost: $0.814 |
| marketing.social_audit | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 19.52 s | Claude Code median: 28.41 s | Speedup: 1.5× | Type.com cost: $0.253 | Claude Code cost: $0.543 |
| marketing.social_benchmark | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 26.05 s | Claude Code median: 62.93 s | Speedup: 2.4× | Type.com cost: $0.383 | Claude Code cost: $1.084 |
| marketing.social_contest | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.75 s | Claude Code median: 25.69 s | Speedup: 14.7× | Type.com cost: $0.000 | Claude Code cost: $0.453 |
| marketing.social_mention_response | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 1.92 s | Claude Code median: 45.35 s | Speedup: 23.6× | Type.com cost: $0.000 | Claude Code cost: $0.658 |
| marketing.social_roi | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.82 s | Claude Code median: 53.83 s | Speedup: 29.6× | Type.com cost: $0.000 | Claude Code cost: $0.827 |
| marketing.social_scheduling | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 21.06 s | Claude Code median: 42.27 s | Speedup: 2.0× | Type.com cost: $0.204 | Claude Code cost: $0.664 |
| marketing.tam_analysis | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.84 s | Claude Code median: 56.23 s | Speedup: 30.6× | Type.com cost: $0.000 | Claude Code cost: $0.899 |
| marketing.trending_topic | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 21.22 s | Claude Code median: 48.40 s | Speedup: 2.3× | Type.com cost: $0.211 | Claude Code cost: $0.737 |
| marketing.ugc_moderation | Type.com passed: 3/3 | Claude Code passed: 0/3 | Type.com median: 1.83 s | Claude Code median: 81.19 s | Speedup: 44.4× | Type.com cost: $0.000 | Claude Code cost: $1.056 |
| marketing.webinar_cancellation_workflow | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.72 s | Claude Code median: 185.81 s | Speedup: 108.0× | Type.com cost: $0.000 | Claude Code cost: $2.702 |
| operations.access_request_validation | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.84 s | Claude Code median: 39.89 s | Speedup: 21.7× | Type.com cost: $0.000 | Claude Code cost: $0.687 |
| operations.asana_fire_drill | Type.com passed: 0/3 | Claude Code passed: 3/3 | Type.com median: 1.72 s | Claude Code median: 49.68 s | Speedup: 28.9× | Type.com cost: $0.000 | Claude Code cost: $0.676 |
| operations.asana_slack_sheets_audit | Type.com passed: 0/3 | Claude Code passed: 1/3 | Type.com median: 2.05 s | Claude Code median: 68.13 s | Speedup: 33.2× | Type.com cost: $0.000 | Claude Code cost: $0.921 |
| operations.buffer_engagement_optimization | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.84 s | Claude Code median: 82.31 s | Speedup: 44.7× | Type.com cost: $0.000 | Claude Code cost: $1.347 |
| operations.calendar_airtable_maintenance | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.72 s | Claude Code median: 66.12 s | Speedup: 38.4× | Type.com cost: $0.000 | Claude Code cost: $0.942 |
| operations.calendar_slack_training | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.81 s | Claude Code median: 29.94 s | Speedup: 16.5× | Type.com cost: $0.000 | Claude Code cost: $0.590 |
| operations.calendly_equipment_inspection | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.82 s | Claude Code median: 43.26 s | Speedup: 23.8× | Type.com cost: $0.000 | Claude Code cost: $0.820 |
| operations.canva_asset_management | Type.com passed: 2/3 | Claude Code passed: 1/3 | Type.com median: 1.71 s | Claude Code median: 28.09 s | Speedup: 16.4× | Type.com cost: $0.000 | Claude Code cost: $0.503 |
| operations.chemical_inventory_safety_audit | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 1.72 s | Claude Code median: 69.27 s | Speedup: 40.3× | Type.com cost: $0.000 | Claude Code cost: $1.109 |
| operations.compressed_gas_cylinder_tracking | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 1.81 s | Claude Code median: 63.60 s | Speedup: 35.1× | Type.com cost: $0.000 | Claude Code cost: $0.786 |
| operations.conference_room_booking_conflicts | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.85 s | Claude Code median: 23.85 s | Speedup: 12.9× | Type.com cost: $0.000 | Claude Code cost: $0.533 |
| operations.contract_renewal_pipeline | Type.com passed: 2/3 | Claude Code passed: 2/3 | Type.com median: 1.72 s | Claude Code median: 35.98 s | Speedup: 20.9× | Type.com cost: $0.000 | Claude Code cost: $0.637 |
| operations.contractor_onboarding_workflow | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.74 s | Claude Code median: 45.92 s | Speedup: 26.4× | Type.com cost: $0.000 | Claude Code cost: $0.724 |
| operations.cross_department_budget_reconciliation | Type.com passed: 3/3 | Claude Code passed: 2/3 | Type.com median: 1.84 s | Claude Code median: 66.32 s | Speedup: 36.0× | Type.com cost: $0.000 | Claude Code cost: $0.833 |
| operations.docusign_lease_renewal | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.72 s | Claude Code median: 39.85 s | Speedup: 23.2× | Type.com cost: $0.000 | Claude Code cost: $0.688 |
| operations.docusign_prospect_nda | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.72 s | Claude Code median: 27.65 s | Speedup: 16.1× | Type.com cost: $0.000 | Claude Code cost: $0.493 |
| operations.drive_notion_archive | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.64 s | Claude Code median: 46.01 s | Speedup: 28.1× | Type.com cost: $0.000 | Claude Code cost: $0.521 |
| operations.energy_consumption_anomaly | Type.com passed: 3/3 | Claude Code passed: 2/3 | Type.com median: 1.74 s | Claude Code median: 43.38 s | Speedup: 24.9× | Type.com cost: $0.000 | Claude Code cost: $0.642 |
| operations.facebook_roas_monitoring | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.82 s | Claude Code median: 18.00 s | Speedup: 9.9× | Type.com cost: $0.000 | Claude Code cost: $0.390 |
| operations.generator_load_testing_schedule | Type.com passed: 3/3 | Claude Code passed: 2/3 | Type.com median: 1.82 s | Claude Code median: 43.20 s | Speedup: 23.7× | Type.com cost: $0.000 | Claude Code cost: $0.616 |
| operations.google_ads_budget_alert | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.94 s | Claude Code median: 66.24 s | Speedup: 34.1× | Type.com cost: $0.000 | Claude Code cost: $1.152 |
| operations.hubspot_churn_prediction | Type.com passed: 3/3 | Claude Code passed: 2/3 | Type.com median: 1.62 s | Claude Code median: 60.70 s | Speedup: 37.5× | Type.com cost: $0.000 | Claude Code cost: $1.160 |
| operations.hubspot_lead_qualification | Type.com passed: 3/3 | Claude Code passed: 2/3 | Type.com median: 1.95 s | Claude Code median: 67.51 s | Speedup: 34.6× | Type.com cost: $0.000 | Claude Code cost: $1.111 |
| operations.hubspot_mailchimp_sync | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.72 s | Claude Code median: 46.29 s | Speedup: 26.9× | Type.com cost: $0.000 | Claude Code cost: $0.982 |
| operations.mailchimp_subscriber_sync | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 26.83 s | Claude Code median: 54.30 s | Speedup: 2.0× | Type.com cost: $0.354 | Claude Code cost: $0.901 |
| operations.monday_calendar_emergency_drill | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.74 s | Claude Code median: 36.58 s | Speedup: 21.0× | Type.com cost: $0.000 | Claude Code cost: $0.712 |
| operations.parking_allocation_management | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 11.95 s | Claude Code median: 59.99 s | Speedup: 5.0× | Type.com cost: $0.091 | Claude Code cost: $1.099 |
| operations.pipefy_slack_purchase_request | Type.com passed: 0/3 | Claude Code passed: 1/3 | Type.com median: 1.81 s | Claude Code median: 54.36 s | Speedup: 30.0× | Type.com cost: $0.000 | Claude Code cost: $0.821 |
| operations.pipefy_vendor_onboarding | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.80 s | Claude Code median: 75.54 s | Speedup: 42.0× | Type.com cost: $0.000 | Claude Code cost: $1.070 |
| operations.procurement_card_reconciliation | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.72 s | Claude Code median: 47.32 s | Speedup: 27.5× | Type.com cost: $0.000 | Claude Code cost: $0.764 |
| operations.sheets_asana_approved_request | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.83 s | Claude Code median: 51.04 s | Speedup: 27.9× | Type.com cost: $0.000 | Claude Code cost: $1.028 |
| operations.sheets_monday_maintenance_queue | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.73 s | Claude Code median: 42.10 s | Speedup: 24.3× | Type.com cost: $0.000 | Claude Code cost: $0.691 |
| operations.sla_breach_early_warning | Type.com passed: 3/3 | Claude Code passed: 2/3 | Type.com median: 1.81 s | Claude Code median: 35.41 s | Speedup: 19.6× | Type.com cost: $0.000 | Claude Code cost: $0.692 |
| operations.training_compliance_check | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.62 s | Claude Code median: 49.29 s | Speedup: 30.4× | Type.com cost: $0.000 | Claude Code cost: $0.582 |
| operations.trello_basecamp_compliance | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.72 s | Claude Code median: 33.74 s | Speedup: 19.6× | Type.com cost: $0.000 | Claude Code cost: $0.408 |
| operations.trello_basecamp_slack_compliance_followup | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.72 s | Claude Code median: 33.46 s | Speedup: 19.5× | Type.com cost: $0.000 | Claude Code cost: $0.438 |
| operations.trello_vendor_hold_email | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.72 s | Claude Code median: 33.00 s | Speedup: 19.2× | Type.com cost: $0.000 | Claude Code cost: $0.491 |
| operations.twilio_afterhours_incident | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 2.19 s | Claude Code median: 67.73 s | Speedup: 30.9× | Type.com cost: $0.000 | Claude Code cost: $1.072 |
| operations.utility_cost_allocation | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.72 s | Claude Code median: 44.18 s | Speedup: 25.7× | Type.com cost: $0.000 | Claude Code cost: $0.666 |
| operations.vendor_compliance_audit | Type.com passed: 0/3 | Claude Code passed: 3/3 | Type.com median: 1.83 s | Claude Code median: 33.58 s | Speedup: 18.3× | Type.com cost: $0.000 | Claude Code cost: $0.737 |
| operations.zoom_dr_drill | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.83 s | Claude Code median: 65.46 s | Speedup: 35.8× | Type.com cost: $0.000 | Claude Code cost: $0.856 |
| sales.add_contact_to_campaign | Type.com passed: 3/3 | Claude Code passed: 2/3 | Type.com median: 1.72 s | Claude Code median: 68.89 s | Speedup: 40.1× | Type.com cost: $0.000 | Claude Code cost: $1.305 |
| sales.calendly_capacity_management | Type.com passed: 2/3 | Claude Code passed: 2/3 | Type.com median: 1.64 s | Claude Code median: 53.20 s | Speedup: 32.4× | Type.com cost: $0.000 | Claude Code cost: $1.184 |
| sales.calendly_lead_scoring | Type.com passed: 1/3 | Claude Code passed: 2/3 | Type.com median: 1.92 s | Claude Code median: 67.94 s | Speedup: 35.4× | Type.com cost: $0.000 | Claude Code cost: $1.714 |
| sales.calendly_meeting_prep | Type.com passed: 0/3 | Claude Code passed: 2/3 | Type.com median: 1.73 s | Claude Code median: 56.60 s | Speedup: 32.7× | Type.com cost: $0.000 | Claude Code cost: $1.193 |
| sales.calendly_reschedule_chain | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.84 s | Claude Code median: 44.92 s | Speedup: 24.4× | Type.com cost: $0.000 | Claude Code cost: $0.905 |
| sales.chatgpt_contract_summary | Type.com passed: 2/3 | Claude Code passed: 3/3 | Type.com median: 19.33 s | Claude Code median: 57.36 s | Speedup: 3.0× | Type.com cost: $0.295 | Claude Code cost: $1.582 |
| sales.chatgpt_email_sentiment_routing | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.94 s | Claude Code median: 109.50 s | Speedup: 56.4× | Type.com cost: $0.000 | Claude Code cost: $1.665 |
| sales.chatgpt_lead_classification_pipeline | Type.com passed: 3/3 | Claude Code passed: 2/3 | Type.com median: 2.56 s | Claude Code median: 84.59 s | Speedup: 33.0× | Type.com cost: $0.000 | Claude Code cost: $1.592 |
| sales.competitive_displacement_outreach | Type.com passed: 3/3 | Claude Code passed: 2/3 | Type.com median: 1.84 s | Claude Code median: 77.91 s | Speedup: 42.3× | Type.com cost: $0.000 | Claude Code cost: $1.399 |
| sales.contract_renewal_coordinator | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.82 s | Claude Code median: 61.56 s | Speedup: 33.8× | Type.com cost: $0.000 | Claude Code cost: $1.366 |
| sales.create_contact_for_account | Type.com passed: 3/3 | Claude Code passed: 1/3 | Type.com median: 1.82 s | Claude Code median: 46.68 s | Speedup: 25.6× | Type.com cost: $0.000 | Claude Code cost: $1.122 |
| sales.create_important_draft | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.93 s | Claude Code median: 82.83 s | Speedup: 42.9× | Type.com cost: $0.000 | Claude Code cost: $1.337 |
| sales.cross_platform_account_health_score | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 2.02 s | Claude Code median: 110.71 s | Speedup: 54.8× | Type.com cost: $0.000 | Claude Code cost: $2.802 |
| sales.deal_escalation | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 1.72 s | Claude Code median: 48.38 s | Speedup: 28.1× | Type.com cost: $0.000 | Claude Code cost: $0.688 |
| sales.docusign_compliance_template | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 2.04 s | Claude Code median: 48.04 s | Speedup: 23.5× | Type.com cost: $0.000 | Claude Code cost: $1.281 |
| sales.docusign_conditional_template_selection | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.84 s | Claude Code median: 60.96 s | Speedup: 33.1× | Type.com cost: $0.000 | Claude Code cost: $1.830 |
| sales.docusign_contract_send | Type.com passed: 3/3 | Claude Code passed: 2/3 | Type.com median: 1.71 s | Claude Code median: 50.72 s | Speedup: 29.7× | Type.com cost: $0.000 | Claude Code cost: $1.150 |
| sales.docusign_deal_workspace | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.84 s | Claude Code median: 39.06 s | Speedup: 21.2× | Type.com cost: $0.000 | Claude Code cost: $0.973 |
| sales.docusign_renewal_automation | Type.com passed: 2/3 | Claude Code passed: 0/3 | Type.com median: 1.84 s | Claude Code median: 79.26 s | Speedup: 43.1× | Type.com cost: $0.000 | Claude Code cost: $1.258 |
| sales.docusign_void_resend | Type.com passed: 0/3 | Claude Code passed: 2/3 | Type.com median: 14.26 s | Claude Code median: 93.55 s | Speedup: 6.6× | Type.com cost: $0.147 | Claude Code cost: $1.214 |
| sales.form_to_crm_qualification | Type.com passed: 2/3 | Claude Code passed: 0/3 | Type.com median: 1.72 s | Claude Code median: 47.94 s | Speedup: 27.9× | Type.com cost: $0.000 | Claude Code cost: $0.709 |
| sales.gcal_deal_milestone_tracker | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.93 s | Claude Code median: 32.27 s | Speedup: 16.7× | Type.com cost: $0.000 | Claude Code cost: $0.786 |
| sales.historical_context_pipeline_report | Type.com passed: 3/3 | Claude Code passed: 1/3 | Type.com median: 1.80 s | Claude Code median: 48.98 s | Speedup: 27.2× | Type.com cost: $0.000 | Claude Code cost: $0.977 |
| sales.historical_win_loss_recap | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 1.73 s | Claude Code median: 44.63 s | Speedup: 25.8× | Type.com cost: $0.000 | Claude Code cost: $1.046 |
| sales.invalid_email_cleanup | Type.com passed: 2/3 | Claude Code passed: 2/3 | Type.com median: 1.85 s | Claude Code median: 156.48 s | Speedup: 84.6× | Type.com cost: $0.000 | Claude Code cost: $2.714 |
| sales.job_posting_contact | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.74 s | Claude Code median: 69.95 s | Speedup: 40.2× | Type.com cost: $0.000 | Claude Code cost: $1.671 |
| sales.linkedin_connection_outreach | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 1.95 s | Claude Code median: 45.15 s | Speedup: 23.2× | Type.com cost: $0.000 | Claude Code cost: $0.963 |
| sales.linkedin_content_engagement | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.83 s | Claude Code median: 32.40 s | Speedup: 17.7× | Type.com cost: $0.000 | Claude Code cost: $0.996 |
| sales.linkedin_job_change | Type.com passed: 0/3 | Claude Code passed: 1/3 | Type.com median: 1.84 s | Claude Code median: 73.65 s | Speedup: 40.0× | Type.com cost: $0.000 | Claude Code cost: $1.979 |
| sales.linkedin_signal_monitoring | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.81 s | Claude Code median: 40.76 s | Speedup: 22.5× | Type.com cost: $0.000 | Claude Code cost: $0.918 |
| sales.mark_vip_emails_read | Type.com passed: 3/3 | Claude Code passed: 2/3 | Type.com median: 23.16 s | Claude Code median: 88.93 s | Speedup: 3.8× | Type.com cost: $0.291 | Claude Code cost: $1.613 |
| sales.multi_step_drip_campaign | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.72 s | Claude Code median: 27.24 s | Speedup: 15.8× | Type.com cost: $0.000 | Claude Code cost: $0.658 |
| sales.negative_selection | Type.com passed: 1/3 | Claude Code passed: 0/3 | Type.com median: 1.85 s | Claude Code median: 56.48 s | Speedup: 30.5× | Type.com cost: $0.000 | Claude Code cost: $1.499 |
| sales.qualify_lead | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 14.06 s | Claude Code median: 53.85 s | Speedup: 3.8× | Type.com cost: $0.158 | Claude Code cost: $1.025 |
| sales.sheets_multi_channel_campaign_router | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 2.02 s | Claude Code median: 48.75 s | Speedup: 24.1× | Type.com cost: $0.000 | Claude Code cost: $0.872 |
| sales.sheets_reconciliation | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.83 s | Claude Code median: 41.30 s | Speedup: 22.6× | Type.com cost: $0.000 | Claude Code cost: $0.949 |
| sales.unreliable_label_account_review | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.85 s | Claude Code median: 44.14 s | Speedup: 23.9× | Type.com cost: $0.000 | Claude Code cost: $0.906 |
| sales.vendor_review_prep | Type.com passed: 1/3 | Claude Code passed: 0/3 | Type.com median: 1.73 s | Claude Code median: 51.72 s | Speedup: 29.9× | Type.com cost: $0.000 | Claude Code cost: $0.981 |
| sales.weighted_priority | Type.com passed: 3/3 | Claude Code passed: 0/3 | Type.com median: 1.83 s | Claude Code median: 39.65 s | Speedup: 21.7× | Type.com cost: $0.000 | Claude Code cost: $1.064 |
| sales.zoom_attendance_pipeline | Type.com passed: 1/3 | Claude Code passed: 0/3 | Type.com median: 1.73 s | Claude Code median: 47.81 s | Speedup: 27.6× | Type.com cost: $0.000 | Claude Code cost: $1.043 |
| sales.zoom_recording_distribution | Type.com passed: 1/3 | Claude Code passed: 2/3 | Type.com median: 1.83 s | Claude Code median: 100.69 s | Speedup: 55.0× | Type.com cost: $0.000 | Claude Code cost: $2.127 |
| sales.zoom_recurring_meeting_optimizer | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.73 s | Claude Code median: 44.02 s | Speedup: 25.4× | Type.com cost: $0.000 | Claude Code cost: $0.975 |
| support.freshdesk_auto_merge | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 1.63 s | Claude Code median: 36.53 s | Speedup: 22.4× | Type.com cost: $0.000 | Claude Code cost: $0.566 |
| support.freshdesk_compliance_tagging | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.71 s | Claude Code median: 53.48 s | Speedup: 31.3× | Type.com cost: $0.000 | Claude Code cost: $0.881 |
| support.freshdesk_csat_followup | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.94 s | Claude Code median: 210.24 s | Speedup: 108.4× | Type.com cost: $0.000 | Claude Code cost: $3.230 |
| support.freshdesk_multi_queue_routing | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 1.73 s | Claude Code median: 34.33 s | Speedup: 19.8× | Type.com cost: $0.000 | Claude Code cost: $0.607 |
| support.freshdesk_onboarding_automation | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.83 s | Claude Code median: 69.99 s | Speedup: 38.2× | Type.com cost: $0.000 | Claude Code cost: $1.242 |
| support.freshdesk_zoho_migration | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.83 s | Claude Code median: 54.28 s | Speedup: 29.7× | Type.com cost: $0.000 | Claude Code cost: $0.934 |
| support.gorgias_defect_tracking | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 2.80 s | Claude Code median: 45.36 s | Speedup: 16.2× | Type.com cost: $0.000 | Claude Code cost: $0.737 |
| support.gorgias_fraud_detection | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.82 s | Claude Code median: 76.93 s | Speedup: 42.3× | Type.com cost: $0.000 | Claude Code cost: $1.270 |
| support.gorgias_inventory_routing | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.83 s | Claude Code median: 109.92 s | Speedup: 60.1× | Type.com cost: $0.000 | Claude Code cost: $1.879 |
| support.gorgias_order_lookup | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.72 s | Claude Code median: 43.73 s | Speedup: 25.4× | Type.com cost: $0.000 | Claude Code cost: $0.861 |
| support.gorgias_refund_processing | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.82 s | Claude Code median: 67.91 s | Speedup: 37.3× | Type.com cost: $0.000 | Claude Code cost: $1.350 |
| support.helpcrunch_engagement_scoring | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 1.74 s | Claude Code median: 106.03 s | Speedup: 60.9× | Type.com cost: $0.000 | Claude Code cost: $2.213 |
| support.helpcrunch_satisfaction | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.82 s | Claude Code median: 98.13 s | Speedup: 53.9× | Type.com cost: $0.000 | Claude Code cost: $1.611 |
| support.helpscout_customer_merge | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.86 s | Claude Code median: 91.25 s | Speedup: 49.1× | Type.com cost: $0.000 | Claude Code cost: $1.384 |
| support.helpscout_daily_digest | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.72 s | Claude Code median: 80.19 s | Speedup: 46.6× | Type.com cost: $0.000 | Claude Code cost: $1.226 |
| support.helpscout_response_audit | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.85 s | Claude Code median: 52.12 s | Speedup: 28.2× | Type.com cost: $0.000 | Claude Code cost: $0.770 |
| support.helpscout_weekly_report | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.82 s | Claude Code median: 66.77 s | Speedup: 36.7× | Type.com cost: $0.000 | Claude Code cost: $1.100 |
| support.hiver_cross_team_handoff | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.72 s | Claude Code median: 85.46 s | Speedup: 49.7× | Type.com cost: $0.000 | Claude Code cost: $1.221 |
| support.hiver_escalation_patterns | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.83 s | Claude Code median: 63.31 s | Speedup: 34.6× | Type.com cost: $0.000 | Claude Code cost: $0.939 |
| support.hiver_slack_digest | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.84 s | Claude Code median: 68.89 s | Speedup: 37.4× | Type.com cost: $0.000 | Claude Code cost: $1.089 |
| support.hiver_workload_forecast | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.82 s | Claude Code median: 32.73 s | Speedup: 18.0× | Type.com cost: $0.000 | Claude Code cost: $0.462 |
| support.intercom_conversation_quality | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 2.04 s | Claude Code median: 69.76 s | Speedup: 34.2× | Type.com cost: $0.000 | Claude Code cost: $1.076 |
| support.intercom_feature_request | Type.com passed: 2/3 | Claude Code passed: 1/3 | Type.com median: 1.73 s | Claude Code median: 74.12 s | Speedup: 42.8× | Type.com cost: $0.000 | Claude Code cost: $1.182 |
| support.intercom_freshdesk_escalation | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 2.08 s | Claude Code median: 60.26 s | Speedup: 29.0× | Type.com cost: $0.000 | Claude Code cost: $1.147 |
| support.intercom_multi_product_routing | Type.com passed: 2/3 | Claude Code passed: 2/3 | Type.com median: 1.94 s | Claude Code median: 76.32 s | Speedup: 39.3× | Type.com cost: $0.000 | Claude Code cost: $1.290 |
| support.intercom_winback | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.84 s | Claude Code median: 84.45 s | Speedup: 45.9× | Type.com cost: $0.000 | Claude Code cost: $1.369 |
| support.reamaze_lifecycle_tagging | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.72 s | Claude Code median: 64.81 s | Speedup: 37.7× | Type.com cost: $0.000 | Claude Code cost: $1.214 |
| support.reamaze_multi_channel_dedup | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.71 s | Claude Code median: 33.91 s | Speedup: 19.8× | Type.com cost: $0.000 | Claude Code cost: $0.746 |
| support.zendesk_automated_responses | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.73 s | Claude Code median: 24.57 s | Speedup: 14.2× | Type.com cost: $0.000 | Claude Code cost: $0.597 |
| support.zendesk_cross_org_merge | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.82 s | Claude Code median: 51.02 s | Speedup: 28.0× | Type.com cost: $0.000 | Claude Code cost: $1.020 |
| support.zendesk_customer_sentiment | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.93 s | Claude Code median: 41.43 s | Speedup: 21.5× | Type.com cost: $0.000 | Claude Code cost: $0.763 |
| support.zendesk_data_quality | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.84 s | Claude Code median: 44.19 s | Speedup: 24.0× | Type.com cost: $0.000 | Claude Code cost: $0.735 |
| support.zendesk_freshdesk_reconciliation | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 1.73 s | Claude Code median: 91.51 s | Speedup: 52.9× | Type.com cost: $0.000 | Claude Code cost: $1.314 |
| support.zendesk_gdpr_purge | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.62 s | Claude Code median: 50.00 s | Speedup: 30.9× | Type.com cost: $0.000 | Claude Code cost: $0.738 |
| support.zoho_calendar_callbacks | Type.com passed: 1/3 | Claude Code passed: 2/3 | Type.com median: 1.74 s | Claude Code median: 41.87 s | Speedup: 24.1× | Type.com cost: $0.000 | Claude Code cost: $0.829 |
| support.zoho_desk_capacity_planning | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.74 s | Claude Code median: 67.71 s | Speedup: 38.9× | Type.com cost: $0.000 | Claude Code cost: $1.156 |
| support.zoho_desk_root_cause | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 15.46 s | Claude Code median: 58.81 s | Speedup: 3.8× | Type.com cost: $0.197 | Claude Code cost: $1.138 |
| support.zoho_desk_ticket_categorization | Type.com passed: 1/3 | Claude Code passed: 1/3 | Type.com median: 1.83 s | Claude Code median: 62.64 s | Speedup: 34.2× | Type.com cost: $0.000 | Claude Code cost: $0.933 |
| support.zoho_desk_warranty_processing | Type.com passed: 0/3 | Claude Code passed: 1/3 | Type.com median: 1.73 s | Claude Code median: 47.88 s | Speedup: 27.7× | Type.com cost: $0.000 | Claude Code cost: $0.924 |
| support.zoho_freshdesk_escalation | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 1.72 s | Claude Code median: 56.38 s | Speedup: 32.8× | Type.com cost: $0.000 | Claude Code cost: $0.795 |
| support.zoho_risk_scoring | Type.com passed: 3/3 | Claude Code passed: 3/3 | Type.com median: 1.82 s | Claude Code median: 45.01 s | Speedup: 24.7× | Type.com cost: $0.000 | Claude Code cost: $0.700 |
| support.zoho_sf_enrichment | Type.com passed: 0/3 | Claude Code passed: 0/3 | Type.com median: 30.89 s | Claude Code median: 71.67 s | Speedup: 2.3× | Type.com cost: $0.458 | Claude Code cost: $1.554 |
07 · Methodology
Methodology
We drew 250 tasks from the 533 repeatable tasks in Zapier's AutomationBench (pinned to commit 4a8e106), balanced across finance, HR, marketing, operations, sales and support. The tasks run in simulated versions of Salesforce, Gmail, Google Calendar, Google Sheets, Slack, and accounting and support tools. For each task, Claude solved it once and saved it as a Type.com code automation, written in TypeScript with the Type.com automation SDK, that can hand any step back to Claude. We kept the first program that ran, not the best-scoring one, and the authoring run never saw the grading checks.
We then made 3 new versions of every task with different amounts, dates, statuses, eligibility and branch conditions, and ran each version twice from the same starting state: once with the saved code and once in a fresh Claude Code session, for 750 paired runs. Both setups used Claude Opus 5.5 in Claude Code at medium effort, with a 50-turn and 600-second limit. AutomationBench's deterministic checks, adapted for the new versions, graded the final state of the apps, and a run passed only if every check passed.
Model costs are estimates at Claude list prices and include writing the code, validation, Claude calls and retries; they don't include Type.com's price or compute. Run times exclude grading and include retries. Only infrastructure failures were retried, and every attempt was kept and charged: 1,753 attempts, 0 unresolved grading errors and $487.65 in total model spend. The Type.com setup ran with the Type.com automation SDK in a benchmark harness. This is not an official AutomationBench leaderboard score. AutomationBench is © 2026 Zapier, Inc., under the MIT License; Zapier has not reviewed or endorsed these results. Results verified November 2, 2026.
08 · Error analysis
Error analysis
Saved code makes a correct check repeat every time, and it repeats a mistake every time too. Three tasks where the setups disagreed:
| Task | Type.com | Claude Code | What happened |
|---|---|---|---|
| sales.docusign_contract_send | Type.com: 3/3 | Claude Code: 2/3 | What happened: The most senior signer's Salesforce notes said not to route contracts to them. The saved code read the notes and picked the next eligible signer. Claude Code didn't read them. |
| hr.background_check_followup | Type.com: 0/3 | Claude Code: 3/3 | What happened: The saved code matched the first column containing "id", which was "Candidate", so it used names as IDs and missed a result change. |
| operations.vendor_compliance_audit | Type.com: 0/3 | Claude Code: 3/3 | What happened: The saved code never read the compliance inbox, so it missed an expired certificate and a pending-renewal exemption. |
FAQ
Questions about this benchmark
- Is Type.com faster than Claude Code for recurring automations?
- Yes. On 750 paired repeat runs of 250 AutomationBench tasks, Type.com's median run took 1.82 s and Claude Code's took 55.81 s: 30.6× faster. Type.com was faster on 250 of 250 tasks, and 6.8× faster at the 90th percentile.
- How much cheaper is Type.com than Claude Code for automations that run repeatedly?
- Type.com used 96.8% less model spend on repeat runs: $8.28 vs $259.72 for 750 runs, about $0.011 vs $0.346 per run. Counting the one-time $0.88 per task to write the code, Type.com breaks even after 2.6 runs and was 12.2% cheaper in total after 3 runs.
- Is Type.com less accurate than Claude Code?
- No. Type.com passed 48.8% of runs (366 of 750) and Claude Code passed 45.5% (341 of 750), graded by AutomationBench's strict checks on the final app state. Type.com passed all 3 versions of 96 tasks, against 75 for Claude Code.
- Why are Type.com automations faster and cheaper?
- Type.com has Claude solve a task once and saves the repeatable steps as code. Later runs execute that code and call Claude only for steps that need judgment, so 87.3% of repeat runs in this benchmark finished without a model call.
- What benchmark is this, and can I check the data?
- It uses Zapier's open-source AutomationBench (MIT License), which grades AI agents on business tasks in simulated apps such as Salesforce, Gmail and Slack. The per-task results and every number on this page are available to download as JSON and CSV.
09 · Data
Data and downloads
The JSON file has every number on this page and the results for all 250 tasks. The CSV has the per-task results.
Download all results (JSON) Download per-task results (CSV) AutomationBench on GitHub
All Type.com benchmarks Version 1 · Results from November 2, 2026 · Claude Opus 5.5 in Claude Code
Cite this benchmark
Type.com (2026). Recurring automation benchmark: Type.com vs Claude Code, version 1. https://type.com/benchmarks/recurring-automations
Run your repeat work 30× faster.
Describe the job once. Type.com writes the code, calls Claude when a step needs judgment, and runs it on your team's connections.
Get started for free