It depends on where the message is and how you delete it.
In most folders, the Delete key will move the message to Trash. Shift + Delete on the other hand, deletes the message permanently without it going to Trash. Deleting messages in the Junk folder does not send them to Trash; they are deleted permanently. Deleting messages in the Trash is the same; they are permanently deleted.