Identify pickle files

[kam193]

Is your feature request related to a problem? Please describe.
Recently, I came across Python pickle files used in a suspicious package, and I'd like to eventually use some analysis for pickles. To implement any in an AL service, AssemblyLine would need to correctly identify pickle files. Currently, the identification is left as unknown.

Describe the solution you'd like
I looked at the definition of the protocol, and when it looks impossible to detect every pickle version, newer versions (since Python 2.3) have specific start bytes indicating protocol version, and later (since Python 3.4, default in 3.8) also uses frames what seems to introduce another static byte.

See following Python source:

• https://github.com/python/cpython/blob/dcfe198867e4fbfeb7a1661adce45ae345fbe5bc/Lib/pickletools.py#L2123-L2133 - PROTO opcode
• https://peps.python.org/pep-3154/#framing - PEP 3154 explaining structure of pickle in protocol version 4
• https://github.com/python/cpython/blob/dcfe198867e4fbfeb7a1661adce45ae345fbe5bc/Lib/pickletools.py#L2365-L2369 - pickletools enforces frames on optimization in version >= 4
• https://github.com/python/cpython/blob/dcfe198867e4fbfeb7a1661adce45ae345fbe5bc/Lib/pickletools.py#L2150-L2160 - FRAME opcode

Thus, every object pickled in protocol v4 starts with 80 04 95, and in v5 80 05 95. So far, I haven't found any other file format using those bytes as header. Given that AssemblyLine doesn't assign any type to pickle files, it looks safe to introduce those bytes as identifiers.

I'd suggest adding them to the file identification.

Describe alternatives you've considered

• It looks like 80 02 and 80 03 may also be unique, but I feel unsure about using just them. In addition, v4 has been the default for years, so I don't think it's necessary.
• Every pickle should end with STOP opcode; however, I don't think it makes sense to check this.

Additional context
For some reason, identifying pickles isn't popular yet. The python module does not perform any special verification - it just breaks once an unknown opcode is found.