Abse - @0hmx https://github.com/tscircuit/tscir...

Shit i guess the timeout number are bit different also found the same issue https://github.com/tscircuit/tscircuit-autorouter/pull/1411#issuecomment-4753328875 Investigatin it now

you were right ``` • You’re wrong that both are using 300s. I checked the PR 1411 benchmark logs: - Run with --sample-timeout 300s: 27836781966 - Command: ./benchmark.sh "--dataset" "18" "--pipeline" "7" "--sample-timeout" "300s" - sample002 timed out at 300.0s - sample012 timed out at 300.0s - Result: 75.0%, 2/16 timed out - Run without --sample-timeout: 27837715698 - Command: ./benchmark.sh "--dataset" "18" "--pipeline" "7" - sample002 timed out at 360.0s - sample012 finished in 273.1s - Result: 81.3%, 1/16 timed out The reason is in scripts/benchmark/index.ts:57: DEFAULT_TASK_TIMEOUT_BASE_MS = 300 * 1000 DEFAULT_TASK_TIMEOUT_PER_EFFORT_MS = 60 * 1000 and scripts/benchmark/index.ts:608: if (sampleTimeoutMs !== undefined) return sampleTimeoutMs return baseTimeoutMs + effortTimeoutMs * getTaskEffort(task) So default timeout is 300s + 60s * effort. These runs are 1x effort, so default is 360s, not 300s. --sample-timeout 300s overrides the default formula and makes the timeout exactly 300s. That’s why sample012 differs: it needs about 273s, but in the 300s run it was still in a later phase and got killed before completing; in the 360s run it had enough time to finish. ``` its basically 300s + 1x effort * 60s so total is 360s and that is why we are at 81% using 360s <@757706909351411845> do you think we should fix this ? or its fine with 360s

Well the sample timeout should be accurate, the exact timeout we use for benchmarking is whatever

i am taking this as a no new fix! because sample timeout is accurate already it gives 300s 75.0% and normal run will use 360s from now getting 81.3%