The thread should be marked "unread" when at least one unread message is in the thread.
The code in postbox plugin is quite complex:
Code:
$sql = "SELECT count(id) AS cnt, MIN(toread) AS toread, MAX(flagged) AS flagged, MAX(datum) AS datum, id, displayname FROM (
(SELECT a.toid AS id, 1 AS toread, 0 AS flagged, a.datum AS datum, b.".($config->realnames ? "name" : "username")." AS displayname FROM #__uddeim AS a LEFT JOIN #__users AS b ON a.toid=b.id WHERE a.fromid=".(int)$myself." AND a.totrashoutbox=0 )
UNION
(SELECT a.fromid AS id, toread, flagged, a.datum AS datum, b.".($config->realnames ? "name" : "username")." AS displayname FROM #__uddeim AS a LEFT JOIN #__users AS b ON a.fromid=b.id WHERE a.toid=".(int)$myself." AND a.totrash=0 AND a.archived=0 AND a.delayed=0 )
) AS comb_table".$filter." GROUP BY id".$sort." LIMIT ".(int)$limitstart.", ".(int)$limit;
"MIN(toread) AS toread" is the magic part. It takes all "toread" flags "0=unread, 1=read" and calculates the minimum. When min=0 then there is at least one unread message.
The fact that it doesn't come in the normal installation made it less obvious to me. 
