Guest User

Untitled

a guest
Nov 6th, 2013
831
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. As of the beta of Battlefield 4, all individual files are compressed with an LZ77 algorithm.
  2.  
  3. A compressed file consists of several blocks, with no global metadata.
  4. The blocks are set to have a size of 0x010000 when decompressed, except for the last one which is usually smaller.
  5.  
  6. Structure of a compressed block (big endian):
  7. 4 bytes: decompressed size (0x10000 or less)
  8. 2 bytes: compression type (0970 for LZ77, 0070/0071 for uncompressed data, 0000 for empty payload)
  9. 2 bytes: compressed size (null for type 0071 and type 0000) of the payload (i.e. without the header)
  10. compressed payload
  11.  
  12. Decompress each block and glue the decompressed parts together to obtain the file.
  13.  
  14. The compression is an LZ77 variant. It requires 3 parameters:
  15. Copy offset: Move backwards by this amount of bytes and start copying a certain number of bytes following that position.
  16. Copy length: How many bytes to copy. If the length is larger than the offset, start at the offset again and copy the same values again.
  17. Proceed length: The number of bytes that were not compressed and can be read directly.
  18.  
  19. Note that the offset is defined in regards to the already decompressed data which e.g. does not contain any compression metadata.
  20.  
  21. The three values are split up however; while the copy length and proceed length are
  22. stated together in a single byte, before an uncompressed section, the relevant offset
  23. is given after the uncompressed section:
  24. Use the proceed length to read the uncompressed data, at which point you arrive at the start of the offset value.
  25. Read this value, then move to the offset and copy a number of bytes (given by copy length)
  26. to the decompressed data. Afterwards, the next copy and proceed length are given and the process starts anew.
  27.  
  28. The offset has a constant size of 2 bytes, in little endian.
  29.  
  30. The two lengths share the same byte. The first half of the byte belongs to the proceed length,
  31. whereas the second half belongs to the copy length.
  32.  
  33. When the half-byte of the proceed length is f, then the length is extended by another byte,
  34. which is placed directly after the byte that contains both lengths. The value of that byte
  35. is added to the value of the proceed length (i.e. f). However, if the extra byte is ff, one more
  36. byte is read (and so on) and all values are added together.
  37.  
  38. The copy length can be extended in the same manner. However, the possible extra bytes are
  39. located at the end, right after the offset.
  40. Additionally, a constant value of 4 is added to obtain the actual copy length.
  41.  
  42. Finally, it is possible that a file ends without specifying an offset (as the last few bytes
  43. in the file were not compressed). The proceed length is not affected by that (and the copy
  44. length is of no relevance).
  45.  
  46. As an example, consider the length byte B2:
  47. Proceed length: B
  48. Copy length: 2 + 4 = 6
  49.  
  50. Another example, F23C:
  51. Proceed length: F + 3C = 4B
  52. Copy length: 2 + 4 = 6
  53.  
  54. A full example (the whitespace is there to separate hex from ascii; it doesn't count):
  55. 0000001a 0970 0018 80 minimap. 0800 51 ature 0a00 40 mize
  56.  
  57. Header:
  58. Decompressed size 1a
  59. LZ77 compression (due to 0970)
  60. Compressed size 18
  61.  
  62. Payload:
  63. Compressed stream: 80 minimap. 0800 51 ature 0a00 40 mize
  64. Decompressed stream: *empty*
  65.  
  66. The decompression is sequential, start with the left part:
  67. 80 minimap. 0800
  68.  
  69. Read 8 uncompressed bytes into the decompressed stream.
  70. Decompressed stream: minimap.
  71.  
  72. Move back by 8 bytes in the decompressed stream (to the start)
  73. and copy 4 bytes (mini) to the decompressed stream.
  74.  
  75. Compressed stream: 51 ature 0a00 40 mize
  76. Decompressed stream: minimap.mini
  77.  
  78. Perform the same step again:
  79. 51 ature 0a00
  80.  
  81. Read 5 uncompressed bytes into the decompressed stream.
  82. Decompressed stream: minimap.miniature
  83.  
  84. Move back by 0a bytes in the decompressed stream
  85. and copy 5 bytes (.mini) to the decompressed stream.
  86.  
  87.  
  88. Compressed stream: 40 mize
  89. Decompressed stream: minimap.miniature.mini
  90.  
  91. Read 4 uncompressed bytes into the decompressed stream (with no offset specified).
  92.  
  93. Decompressed stream: minimap.miniature.minimize
RAW Paste Data