It reminds me of "sanity points system" variant rules from D&D (I think) where smarter characters are more vulnerable to SAN damage.
There are a lot of different D&D SAN systems depending upon the game version (they tend to litter Unearthed Arcana), but the ones I remember tend to mostly use either Wis or else a total of all your "mental stats". The Pathfinder version is even helpfully available online.
The 3e version was especially bad, however, as by the rules as written, you have to roll to lose SAN each time you see even humanoids like orcs or potentially even elves. ("AAAAAH! They're like humans, but have pointy ears! AAGH!") It also has rules for losing SAN each time a character casts a spell, with divine spells taking up an additional 5 SAN per cast, making them vastly less playable than arcane casters.
I checked again, and it wasn't a SAN points system, but a homebrew spell (Terrible Secret from here) that does damage via a mind-shattering revelation, and is harder to resist for characters with high INT.
The 3e version was especially bad, however, as by the rules as written, you have to roll to lose SAN each time you see even humanoids like orcs or potentially even elves. ("AAAAAH! They're like humans, but have pointy ears! AAGH!") It also has rules for losing SAN each time a character casts a spell, with divine spells taking up an additional 5 SAN per cast, making them vastly less playable than arcane casters.
Jesus Christ, that sounds utterly unplayable as written! What the hell happens if you have a mixed party!?
The easiest way is to take a Friend's whole sandstar and use it to create a fissure in empty space but right now Raccoon-san and Fennec are the only ones around so we can't do that no da!First we have to find the Hell Boss that can lead us there no da!We have no choice but to negotiate with the Idiot King that sleeps at the center of the Hell Area no da!I-I-I-I don't want to be here!
H-how do we get out?Unfortunately there's no direct way to exit this place no da!R-r-run, hurry up and let's run, Raccoon-sanThis isn't the first or second time Raccoon-san has been to the Hell Area no da!Why is Raccoon-san calm after seeing the Hell Friend?H-huh, w-why is Raccoon-san so calm?What are you trying to say, Fennec!Ra-Raccoo-coon-san
Tha-that