alexras
Offline
Los Angeles, CA

I'm in the middle of developing a suite of scripts for manipulating LSDj .sav files - merging songs together, extracting instruments from one song and sticking them into another, etc. It's called lsdj-sav-utils, and can be found here.

The code is to the point where it can load and save data from .sav files (not guaranteeing that it's bug free, of course).

I don't want to waste my time writing scripts that nobody's going to use, so I thought I'd try to put this Google Moderator board for you all to post suggestions for script functionality (I've already posted a couple of starters) and to up-vote others' suggestions that you like. I think doing it this way might be easier than a thread with a bunch of "+1!" messages in it.

http://goo.gl/mod/IILw

Looking forward to hearing your suggestions. If you think Google Moderator sucks and you want me to set up with an alternative service, please say so. If enough of you complain, I'll switch it over.

Last edited by alexras (May 19, 2011 7:11 am)

nitro2k01
Offline
Sweeeeeeden

I don't really have any good suggestions, but since I've written my own file manager, I can help you a little with hints about hings you can error cherk when loading files.

All of this can be summarized as, check that the file length is correct. You should get $e0, $ff when you have output exactly 32768 bytes, not more, not less. What's below are just methods to detect errors as early as possible.

1) Check that block switches ($e0) are valid. The parameter must be > (the current block) and < $1f. Or constants $f0/$f1 for default instr/wave, or $ff for EOF.
2) Check that there's actually a block switch within the 512 bytes of a block. If not, the file is corrupted somehow.
3) Check each block against the values in FAT. If you don't have a 1:1 match, either the FAT or the contents of the file are corrupted.
4) Another thing you can check, which is the only thing in this list that I don't check in LittleFM, is that the default instrument/wave commands are at output positions that are actually possible. I.e. the instrument command can only output data at positions $3080, $3090 and following aligned addresses. Likewise, waves must be written to $6000, $6010 and so on.

At the very least, check 1 and 2.

rolemodel
Offline
Stockholm

4) Another thing you can check, which is the only thing in this list that I don't check in LittleFM, is that the default instrument/wave commands are at output positions that are actually possible. I.e. the instrument command can only output data at positions $3080, $3090 and following aligned addresses. Likewise, waves must be written to $6000, $6010 and so on.

It is actually entirely possible that you have this data in other locations aswell.

nitro2k01
Offline
Sweeeeeeden
rolemodel wrote:

It is actually entirely possible that you have this data in other locations aswell.

But as far as I remember, the compressor in LSDj only checks for that data and compresses it when it's in the "right" positions. Under normal circumstances you should never get the $e0 $f0 or $e0 $f1 commands anywhere else, even if the same sequence of bytes as the default wave or instrument exists at some other position in the uncompresssed data. Right?

rolemodel
Offline
Stockholm

It could be that you are right. But this is not part of the official format so... nothing is guaranteed smile

rolemodel
Offline
Stockholm
alexras wrote:

I don't want to waste my time writing scripts that nobody's going to use,

I think maybe the biggest problem is the "script" part. In my experience, most people don't even know how to run a script... some kind of GUI application or web page is usually more appreciated.

alexras
Offline
Los Angeles, CA
nitro2k01 wrote:

I don't really have any good suggestions, but since I've written my own file manager, I can help you a little with hints about hings you can error cherk when loading files.

All of this can be summarized as, check that the file length is correct. You should get $e0, $ff when you have output exactly 32768 bytes, not more, not less. What's below are just methods to detect errors as early as possible.

1) Check that block switches ($e0) are valid. The parameter must be > (the current block) and < $1f. Or constants $f0/$f1 for default instr/wave, or $ff for EOF.
2) Check that there's actually a block switch within the 512 bytes of a block. If not, the file is corrupted somehow.
3) Check each block against the values in FAT. If you don't have a 1:1 match, either the FAT or the contents of the file are corrupted.
4) Another thing you can check, which is the only thing in this list that I don't check in LittleFM, is that the default instrument/wave commands are at output positions that are actually possible. I.e. the instrument command can only output data at positions $3080, $3090 and following aligned addresses. Likewise, waves must be written to $6000, $6010 and so on.

At the very least, check 1 and 2.

Thanks for the tips! I'm definitely asserting correct file length as well as checking for 1 - 3. 4 should be easy enough to check.

alexras
Offline
Los Angeles, CA
rolemodel wrote:

I think maybe the biggest problem is the "script" part. In my experience, most people don't even know how to run a script... some kind of GUI application or web page is usually more appreciated.

I will definitely wrap the scripts in GUIs eventually and provide it in both CLI and GUI form, but I'd like to have them written and working in CLI mode first.

Also, I had intended the library portion of lsdj-sav-utils (everything in common/) to serve as a basis for other applications that either I or others might write in the future. So it serves two purposes, really.

8-Bit-Rex
Offline
Chepachet, Rhode Island
rolemodel wrote:

I think maybe the biggest problem is the "script" part. In my experience, most people don't even know how to run a script... some kind of GUI application or web page is usually more appreciated.

I'm glad someone pointed this out.  Many people would be ALL OVER this system, but would not be able to use it including myself.

This is a fantastic Idea and I'll help spread the word whenever i can.

rolemodel
Offline
Stockholm

> 1) Check that block switches ($e0) are valid. The parameter must be > (the current block) and < $1f. Or constants $f0/$f1 for default instr/wave, or $ff for EOF.

In hindsight, having the block switch command seems totally redundant. Since one already has the list of blocks in the block allocation table, one could just look there for where to decode next... I would consider removing it if it wouldn't break backwards compability.

nitro2k01
Offline
Sweeeeeeden

In hindsight, a checksum of the data would've been nice too. And better compression, like Huffman, or even LZO... On the other hand I personally really like LSDj's compression because it's relatively easy to recover data from it. smile

rolemodel
Offline
Stockholm

I am not sure how meaningful it is to write a checksum. What to do if checksum compare fails? The song is still broken.

I have considered putting in some better compression... (maybe lz77?) ...but in practice I think RLE does a good job enough, it's very fast, consumes almost no memory, and is very simple (= easy to verify there are no errors).

rolemodel
Offline
Stockholm

Actually, when one thinks about it, better compression just means more songs lost in cart crashes smile

undergroundclouds
Offline
Spokompton
rolemodel wrote:

Actually, when one thinks about it, better compression just means more songs lost in cart crashes smile

carts : cart crashes :: ground beef : e-coli

Last edited by undergroundclouds (May 19, 2011 11:01 pm)

alexras
Offline
Los Angeles, CA

OK, so the consensus seems to be:
* Build up a database of instruments, waves and tables
* Patch those instruments, waves and tables into existing songs
* Give it a graphical interface

As it is bidden, so shall it be done.

node
Offline
thieveland ohio

this sounds crazy but i would love to print everything out. i don't trust 1's and 0's i have had them fail too often.