Nessy: Play NES as a MIDI synth with USB CopyNES Installed (Page 1) - Nintendo Consoles - Forums

1 Sep 25, 2018 4:30 am

Offline

Vancouver, BC, Canada

Thought I'd share this here. After modding a NES console with a USB CopyNES, I've written a desktop-side MIDI driver that pushes live APU register changes (in real-time!) to the NES with zero latency. The connection speed is fast enough to stream LFO data streams (ie: vibrato, tremolo) and even DMC PCM samples in real-time without the need for storing them in NES CPU memory and triggering. Which opens up a ton of possibilities.

In a nutshell: I wrote a custom desktop MIDI driver that holds all 2A03 register state information and contains all the logic to translate MIDI notes, CCs, etc. to appropriate APU register changes ($4000 ... $4017). The only data being transmitted over USB (currently 460800 baud rate but higher rates may be stable) is the actual register bytes being received on an interrupt in real-time, so it's essentially zero-latency and very responsive. (It's utilizing a USB CopyNES installed in the system as a way to write to the APU registers.)

The goal is to make all features of the 2A03 chip available as MIDI CCs so you can emulate the behavior of trackers in a MIDI-compatible DAW (Logic, Cubase, Pro Tools, Reaper, Ableton Live, etc.) and to aim it towards power-users (surpass the basic feature set of the midiNES / Chip Maestro) and thereby make it available for sequencing in parallel with other synths/samplers and audio in your DAW and not be restricted to working within a tracker.

Current features of note:
- implemented software-driven modulation (vibrato, tremolo, etc. - any APU parameter can be modulated with depth & rate)
- implemented software-driven sequence macros (akin to Plogue Chipsounds' Wave Sequencer) so note triggers can execute complex sets of user-programmed APU events during the attack, hold (loop), and release period of notes. These presets can either be stored in your DAW session as MIDI SysEx data so they are portable and platform-agnostic, or stored in a global settings file that the driver can load from disk when you open your DAW session

In future versions it might be possible to:

- stream NSF files direct from the desktop in real-time without requiring a RAM cart or side-loading/buffering
- have potential to receive real-time direct APU output stream from Famitracker or a modded NES emulator

Here's some preliminary videos demonstrating that it works. I'll get a full video put together eventually when all the features are implemented.

https://twitter.com/orchetect/status/10 … 1755319296

https://www.instagram.com/p/BpEBwqAADaK

Last edited by orchetect (Dec 3, 2018 4:02 am)

2 Dec 3, 2018 3:44 am

runwithb

Offline

Athens, GA

This is awesome!

I have some questions:

1) Is there any other modding besides just installing the CopyNES?
2) Don't know much about CopyNES, but are you doing this with no other cart loaded in the NES?
3) Real-time DMC PCM samples would be absolutely insane. I realize that's not a question.
4) Have you had any luck implementing portamento? I imagine that'll take some work (math).

I stumbled on your post trying to dig up info on the MidiNES (just bought one on ebay). It's a lot of fun, but definitely hard to control in certain areas (vibrato, pitch bend). It looks like you're implementing this in a user friendly manner right from the start!

I followed on instagram, so I'll be keeping up with your progress!

3 Dec 3, 2018 3:53 am

orchetect

Offline

Vancouver, BC, Canada

1) Just a USB CopyNES is needed
2) No cart needed, runs with no cart inserted in the NES. (No video screen needed either, runs without interaction. Just turn on the NES!)
3) Yes - in the coming weeks I am going to do some tests with realtime DMC PCM streaming from the host. I haven't implemented it yet and will take some time to test the maximum stable sample rate. The main goal is to enable pre-loaded samples on the desktop, but eventually it may be possible to stream live audio which would be transcoded in real-time on the host into 7-bit PCM and pushed to the NES DMC channel.
4) Yes - portamento will be done in software (pushing all frequency changes in real time)

Yes, I'm making great effort to ensure it's user-friendly and remains flexible.

Caveats - I'm writing the MIDI driver for Mac only, and it works only with the USB version of CopyNES. I had to settle on this configuration, since my DAW workstations are Macs and there's no parallel port capability on Mac. It's written in Swift so it won't work on PC/Windows or linux without substantial rewriting. The point was to make it as efficient and stable as possible on ONE platform instead of dealing with different platforms and sacrificing stability or features.

I also have a midiNES 1.1. It's fun, but I like mine better

I'm about 75% done - once I'm mostly done all the major features I'll do a YouTube video or something explaining its capabilities and how to use it. And I'll probably release it as a binary at the very least, possibly open-source it at some point.

Last edited by orchetect (Dec 3, 2018 4:12 am)