Correction: we doubled the CBP benchmark to 100 cases, and the confidence rule we published did not survive it

What we said earlier today, at 45 rulings: every correct answer from the HS classifier scored a confidence of 0.7 or above, and none of the 17 answers below 0.7 was correct. We published that as a usable rule — read a low confidence as “do not rely on this”. At 100 rulings it is 2 of 32.