Alright so in order to play the visual novels I had installed the japanese localization onto my laptop. Windows7 made it easy enough. It added a little 'switch between language inputs' button on my taskbar with an EN and a JP button.
What's even better is that I don't need to switch to the japanese language input to play/install the games, it simply displays the correct characters for Japanese in the installers.
The issue that plagues me now is that my backslash (\) key has been replaced by the symbol for yen in windows explorer.
It only displays as yen, and if I copy / paste the yen symbol into notepad or even in here it becomes a regular backslash.