I share your thoughts about psychological horror. Usually Japanese influenced horror is my favorite. I'm not scared of monsters of zombies, but ghosts and the unknown are what get under my skin. The only game I ever had a nightmare about in all my years of gaming was FEAR 1. Those are well paced games that have a balance between full throttle FPS shootouts and slowed down scare sequences. The first game, FEAR, and its expansion packs: Extraction and Perseus Mandate are my favorite. The 2nd game is okay, it's good in its own ways but it's a bit different. I haven't played the 3rd game but I'm sure it's okay as well, but in general the best one to play is the first game and its expansions. Only play the 2nd and 3rd games if you really feel the need to. The little girl Alma scares me to death. They use her in some pretty effective ways and they find ways to creep me out with her. So FEAR is the only game I ever had a nightmare about, but it's not actually the scariest game I ever played. Not even close.
The scariest game I've ever played is Fatal Frame 2 Crimson Butterfly Directors Cut. Mouthfull of a name but it's a terrifying game. The ghosts terrify me, I always feel vulnerable, and they do an excellent job with the art, lighting, sound design, music, and pacing. I've had Fatal Frame 2 for seven years and I've never beat it. I've made it like 60% through the game or more but it stresses me out so bad that I take long breaks from it and sometimes I even start over from the beginning to remember how the story starts.
I'm going to beat the game sometime, I really want to finish but I really have to be in the mood to play it. It gets under my skin in every way a great horror game should.