When wearing pants or shorts with back pockets I put in the back right pocket. When wearing shorts or pants without back pockets I put it in the front left pocket. I am right handed.
Think about your movements with your wallet. Being right handed I removed wallet with right hand, transfered to left hand for holding, then dug out what I needed with right hand. Skip a step, remove wallet with left, dig with right.
Comment