Here's what I think is happening...
Your Solaris guest thinks it is talking to a physical SCSI device. It is issuing a MODE SENSE (6) command (Cmd 0x1a) to its virtual disk. The RDM (in physical compatibility mode, which provides the most direct SCSI channel from the guest to the device) tries to pass that command to the host device (the SATA disk). I don't immediately know whether ESXi attempts to translate that SCSI command into something that makes sense to SATA and translate the results back or whether it gives up and the request doesn't even get to the device, but either way, the command returns CHECK CONDITION (D:0x2), sense is ILLEGAL REQUEST (0x5), additional sense is INVALID FIELD IN CDB (0x24 0x0), which is a relatively harmless error message indicating that the device didn't have the information (the SCSI Mode Page) that the requestor asked for.
As long as the guest is happy to use the disks, I believe the message to be benign. It certainly doesn't indicate a physical problem with the disks.
It is possible that putting the RDM into virtual compatibility mode will cause the ESXi to instead provide a fully synthetic reply to the guest's MODE SENSE (6) command, which would silence the warnings... but I am not 100% certain of this.
Cheers,
--
Darius