Package repacking issues

Sony introduced the PlayGo system into the PS4, which allows games to use chunk read. This allows convenient features for the user. For example, if you want to play the multi-player aspect of a game, you don't need to wait for the single-player aspect to download/install, or you don't need to waste time by looking at the screen whilst game tries to load chinese language if you're not living in China.

Developers may specify which files belong to what chunks, and which chunks should be loaded by one or more scenarios. This information is used by the Orbis Publishing Tools to build a package where files are stored in a specific order. And game could use an API to control PlayGo system (for example, which scenario should be loaded at some specific event, etc). Some information about used scenarios in a game are stored inside a package file, also there is some information about which languages are used in specific stuff (however, there is a loss of some information about them during a build process).

So, this is a problem for those of us who want to repack an original package. We MUST know this information or we MUST be able to recover this information somehow if we want to rebuild a package that will work. Otherwise we may see glitches, crashes or some other weird bugs, but some games could just work normally if they don't rely on this. I believe this is the main reason why a lot of games are not working (or working with some troubles) using the previous PlayRoom-based method, because PlayRoom uses a simple file layout. So, to workaround it we need to make some guesses based on package file's structure and it works, you could even do it semi-automatically (this is what my tool does). But to do this properly you need to get the inner PFS image (called pfs_image.dat). This is basically for games that use complex scenarios, and don't use just one chunk and one default scenario. I have seen a lot of games that uses only one chunk/one scenario and they will just work as is without the need of doing something else, so just repack them and they will work fine (one of such game is DriveClub). But I want to discuss other games that brings these problems I'm talking about.

Here I should note that Orbis OS does handling of package decryption and decompression (yes, packages may be also compressed if there is a need for it, developers could specify that too) transparently, and OS will mount game contents into a specific directory inside the file system and also «stores» an inner PFS image in another directory. But as I've mentioned already, it's a decrypted and decompressed image file. However, if you want to get information about chunk placements inside a package file, you need to know their real offsets there, such offsets that are used by Orbis OS internally when it does decryption/decompression of package file to locate file data. Later, using these offsets you may recover chunks information and build a proper layout of package file. But you can't get these real offsets if you're operating on an inner PFS file that was already decompressed for us by the system. So, decryption IS okay but decompression IS bad, thus you need a way to get a decrypted raw (compressed) pfs_image.dat file. It could be doable with custom kernel code that uses some buil-tin PFS functions from the kernel, but you also need to write custom code that will skip decompression. I think it could also be doable by logging file offsets to the original .pkg file when you try to load all files from it, then you may use this information to arrange chunks properly. Both methods should be made as PS4 applications and most likely, will require some kernel payloads.

I'm not going to implement that tool because, naturally speaking, it's a game's dumper that does a slightly more than just copy pfs_image.dat whilst some game is running, so I don't want legal troubles, but it's not a rocket science, so someone with the knowledge and skills could implement that. However, I could help by releasing a part of code from my tool that does SFO file parsing, PlayGo chunk builder and finally, GP4 project generation. Someone may use it to implement a similar application.
SFO file parsing

System file object (SFO) is used to store some parameters related to the application/game that are used by the system, so this code is used to parse it.

sfo.h
sfo.c
PlayGo chunk builder

Extracting and recovering chunks (and other) information about files stored inside package file.

playgo.h
playgo.c
GP4 project generation

This code is used to generate .gp4 project file from package file by doing some guesses and using its structure format.

gp4.h
gp4.c
Some of PKG/PFS stuff

Here I've uploaded some structures, definitions and functions related to PKG file format, you may see some references to them from GP4 code.

pkg.h
pkg.c