I mentioned this one back in this Explorer shortcut post, but it deserves being spelled out on its own: When you see a file in a directory folder (such as when opening or saving a file, or when using the Windows Explorer), you can delete a file normally by using the Delete key. It then goes to the Trashcan, where you can undelete it.
If you have a sensitive file or a big file that you don’t want taking up disk space, press Shift+Delete to delete it instead. It won’t appear in the Trashcan, it’s just gone with only one confirmation dialog box to confirm.
Change your mind? You can press Ctrl+Z to Undo the delete operation, but don’t wait too long.