The former Labour leader is in a downcast mood