DHE Codecs, Dissecting the games Options

[Raijinili]

Hello all. I've started a GitHub repo for a project Rai and I have essentially been working in for five years very sporadically, the same project that provided a lot of the RiviPSP content on the site.
It's where we nerd out and hack proprietary file formats in St!ng games.

BUT WAIT
You can help too. That's why started a repo. There's a wiki on the repo where I am documenting our efforts in decrypting these files, and a number of python files needing coding. Trust me, you can probably help. Python is super easy to learn (you can practically read it as is, though some of this code will require some studying up), and even if you're not in the mood to code, you can help with the wikis and other stuff. Dunno, there's plenty to be done though.

My only request is that you not upload files from the games, whether directly from the game or decrypted by the tools, to the repo. I don't want legal issues, and these files take a lot of space and make working with the repo slow.

Help out if you can, and if you can't... Learn! Seriously, there's a lot that can be done and a lot of the main stuff can be learned in a weekend.
For example, one thing that needs to be done is the LIM decoder. I even have the spec for the LIM file format up in the wiki, and an algorithm that only needs to be translated into python (although I wrote it with python more or less in mind, so it's pretty simple to figure out). And you don't need to solve everything to contribute! Even adding one line of code or documentation to the wiki helps.

My current goal is to get all of the archive decrypting tools documented and maybe even coded (they are the easiest of all to code) so that it's easier for people to start probing and working with the files we actually want.

[Raijinili]

Huh. A BMP to LIM converter is feasible.

[Raijinili]

I think there's a couple of St!ng formats we could feasibly create. I think the AMP file format is also quite sufficiently dissected (though I haven't put up the spec yet) that we could make custom sprite sheets for RiviPSP with fair ease.

[Raijinili]

For those not following the project, progress has been slow. I've been drifting between six or seven projects, and I'm the only one working on it as far as I know, so this is what is done so far:

-DAR and AFS archives are extricable
-Gungnir TPL files, except for sprites, are extricable

Technically you could use the TPL class to extract other TPL files, even sprite ones, but there are some issues. For non-Gungnir TPL files, the colors are in BGRA format, whereas for Gungnir they're in a more normal RGBA format. I haven't figured out how to determine coloring order yet. As for sprites, with how they work you could extract all of the pixels into an image you could then decipher into the specific sprite you wanted, but sprite construction is still something I'm working on incorporating into the class. I have it all figured out (or so I think), I just haven't yet written out the algorithm.


Edit: MAYBE if you were into GameCube hacking you've encountered TPL files before. It's a texture format used by a lot of GC games, but St!ng likes to do their own thing. I imagine they were introduced to the format when they made Evolution Worlds, and now they use it for some of their disc-based games, but a modified one. (Heavily modified as of Gungnir.)

This post has been edited by Dr Sturm: Jan 16 2013, 01:51 AM

[Raijinili]

Sturm, did you look at my Gungnir dropbox work with picture viewer?

[Raijinili]

I don't know what picture viewer is, and the terms together are far too vague for Google to produce results. My guess is that it is the Windows preview software, in which case I haven't access to such a thing.

[Raijinili]

It's a program I wrote in C++ that views files from Gungnir. displayTPL.exe.

It requires OpenGL and GLUT, unfortunately. Teach me Python displays.

[Raijinili]

You could use PyGame or wxPython to run pyOpenGL through a windowed interface. I'm just making classes that can process the data in the files and then return it in a usable way, using Python because of the readability of the code, simple cross-platform usability and large amount of libraries. Right now all it does is output a png, but it could easily be adjusted to return pixel data that could be ouput to a UI surface. With some of the formats I've been trying to plan ahead for insertion/creation.

Now that I think about it, I have Objective C/GL code from when I was outputting the RiviTPL files to a cocoa GL surface.

[Raijinili]

BAHAHAHA
I have Gungnir sprites. I just went through 330 Julios making sure they were all working, so I think I have the algorithm down (even worked out some surprises*)



* As for surprises, the start read parameter in the assembly blocks, the little short in the middle of the six-byte arrays, is actually two values crammed into one. The most significant six bytes are the row, the least significant ten the column.


Pretty sure the animation information is in the associate .bin files, which I've finally started poking through. But for now I'm going to keep working with TPLs, rip all the Gungnir sprites and see if I can reconcile the program with Sting's different hackings of the TPL files.

Regarding that, the sprite above has a third TPL section that I haven't figured out. There's a 32 byte header followed by the exact same palette, twice. It has the same format marker in the TPL header as sprite assembly info (the 0xFFFF). I imagine as I get to generic enemies it will turn out to be alternate palettes, but I don't know why they needed to include it for Julio if his palette doesn't change. WE SHALL SEE.

[Raijinili]

Regarding bin files (for Gungnir animations)...
Still kind of working on it. The structure as far as I can tell consists of a header of 20 bytes (two addresses, two run lengths and one unknown item), followed by two blocks of offsets, the first block pointing to the second and the second to another block, which contains code of some sort that I'm presuming describes animation frames but I'm still working on deciphering it. Following the second block is some semi-mysterious data, but I think the "code of some sort" references it, and I think it actually references the frames as the constructed frames from the TPL files. This section is only present for TPL files with many sprites. However, I know that in this code:

CODE

0x01 = End Command (0x20 and 0x51 will both be met by this at the end)
0x20 = Animation Start
    0x10 = Frame description (three parameters)
        B1: Unknown
        B2: Frame number (from TPL)
        B3: Unknown
0x51 = Animation Start (larger sprites)
    0x## = First four bytes following 0x51 are always a number, power of 2
    0x50 = Frame Description (four parameters)
        B1: Unknown
        B2: Offset of Frame Info in mysterious section
        B3: Unknown
        B4: Unknown

0x23 and 0x21 are other probable commands

What might make it extra hard: Some of the frames I've been extracting from the TPLs are hands (and feet and other limbs). They're frames intended to be pasted over weapons which are pasted over hands, giving the illusion of holding an item.