I just bought one of these things. The thing is, it won't turn off.
Would it be alright for me to just ungplug it from the adapter each time I want to shut it off? Any real damage?
Offline
Turning it off wouldn't really be a problem, The on/off switches on those old synths basically would just disconnect the power anyway.
Turning it on with plugging the adapter in could cause some problem, being connected and then plugging in the adapter could create a voltage spike. It'll probably work tho 
Maybe you can open it and see if the on/off switch just needs some cleaning or soldering? it's probably an easy fix, if the rest of the machine just works.