I'll tell you exactly what is happening here. The positive end of the rechargable (the end that protrudes) doesn't protrude enough to touch the contact inside the Gameboy. The same end on the none-rechargeable battery, however, does protrude enough, which is why it works in that far right slot.
I would suggest opening the Gameboy up and bend that right contact forward so that it can make contact with the rechargeable battery, or just pad the space out with some tin foil or something.
Just tried this, sure enough it was the problem. The little thing on the end is too short to reach, and on that one specifically there's a lot of plastic in the way.
I might add a glob of solder to the metal to see if that sorts it, otherwise i'll need to shave away some plastic. It's an aftermarket case so I'm happy to, but it works perfectly with a bit of tin foil.
Yay, thanks!