First of all a new revision of the algorithm:
http://pastebin.com/E9ghYKj0
CHANGELOG:
-Added partial support for debug files
-Added NPD version check (between 0 and3)
-Added per version flags check
-Added support for FLAG_0x20. (Untested)
-Added support for versions 0 and 1 (ISO.BIN.EDAT). See below
When version is 0 or 1 instead of using the digest and the hash of the NPD for calculating the block key and the apploader's IV a zeroed byte array is used as base
I hope this helps your project Snowy.
@
dsadsadsa
I don't think I could help Kakaroto cause most of the checks are not on current code (They probably added further checking on later firmware version) and my expertise is reading assembly which I don't have. However if I can help then in any form they only need to contact me. In fact the only think that I know that is not already public is that 0x20 bytes are copied from the appldr to main memory just before setting mailbox to 7. Those bytes are at 0x890 and probably is the hash for the whitelist.
@
EXE.trim.ALL
There is no hash for SDAT. Those values are unknow and not checked(not confirmed). The only function as you have seen is to be xored to generate the key. To get the exacts values I'll need to check the SDK as those are generated there.
@
Octopus
I haven't look for the second part of the act.dat. My hyphotesis is that the second part of the file is used when debug is enabled and is common for all the consoles (will explain the COD trick). About the digest as I said there is no checked (is a hash of the original data which is unknown until the whole file is read so it can not be used as check). Geohot zeroed it on his code, An analysis of make package will obtain then (or wait until Kakaroto has the SELF fix and hope that it included the algorithm)