Please add support for Physical Drives

[BlindWanderer]

The filter on "input" file discards physical drives.
https://stackoverflow.com/questions/2108313/how-to-access-specific-raw-data-on-disk-from-java

Additionally some indirection/buffering would need to be added to FileSeeker.kt so that when it reads from a HD it reads entire sectors at a time (limitation of java.io.RandomAccessFile when dealing with physical drives).

I'm suggesting this because the drive I'm interested in is a 4TB usb disk and I'm not keen on buying another drive just so I can first backup it up so that I can run this tool.

[manuelsc]

Unfortunately I don't have much spare time to maintain this project anymore. Feel free to create a PR though

[BlindWanderer]

_ I may have spent too many hours trying to learn Kotlin and getting things to build. I did eventually get it to build and played around with it.

Things to know: Not everything is the same for external drives/different models. The file names are different. Additionally, the working block size is adjusted so that the number of blocks can fit into a 32 bit integer, the block size goes from 512 to 4096. This imposes a theoretical max drive size of 16 TB, but good luck using all that space since there is a 2000 titles. BTW, you can download the FW, so there is always the possibility of reverse engineering it. Back on topic, adding support for 4k blocks to the Kotlin code is Annoying. Doable but annoying. I kinda gave up, too much structure was being thrown at something I was pretty sure could be done in many fewer lines of code. So I wrote myself a python script. It doesn't handle MPG extraction yet but it does dump all* the files.

Anyway, turns out I didn't need to recover my data after all, so I've stopped working on this.

---

For the next person trying to make sense of the layout and structure, here is some code and comments I wrote.

In the script I'm abusing the fact that python buffers everything in 4k chunks. It does file reads in 4k chunks. When using the windows PhysicalDrive# interface, reads must be done in increments of the drives block size. If you need to seek to a position that isn't block aligned and in the current buffer, you may need to call BlockSeek (which handles the double seeking for you). I don't use it because I'm doing block aligned seeking or inter block seeking.

The next step is to reverse engineer the TI000_0#.IFO files. BTW the numbered files indicate active and backup, this way if you loose power during a record the filesystem isn't foobar. I'm pretty sure you don't need to worry about the MPG_SEG#.DAT files.+

I have no idea about the first 128 bytes of FB000_00.LST except for the first three fields (number of blocks, free blocks and allocated blocks). No idea what the deal is with the 0x2E.

import os
import sys
import errno

DEFAULT_SECTOR_SIZE = 512
blockSize = None
directory = "C:\\temp\\found\\"

#TI000_01.IFO & TI000_02.IFO
# file contains a header of 0x1F48 bytes?
# the two files are nearly identical
# first 4byte looks like a size or count
# second 4byte is the last value in the allocated portion of the header
# might be describing some sort of tree?
# data section after the header has Date/Times in it as ascii strings, this is the title name.
# each macro data block is 314 bytes
# no numerological analysis has been attempted yet
# The header is mostly pointers/indexes.

#VBI_DATA.DAT
# some sort of tree? very overkill 
# 0x40 bytes at 0x0000
# 0x30 bytes at 0x1000
# 0x05 bytes at 0x2000
# no numerological analysis has been atttempted yet 

#MPG_SEG1.DAT & MPG_SEG2.DAT
# Some sort of space allocation table. 
# The file is too small. The blocks being allocated would be huge.
# There do appear to be null gaps in the AV001_00.MPG so they may relate.

#Empty Files:
# FONT_DAT.DAT
# DGT_INFO.DAT
# CM__INFO.DAT
# BU_LOG_D.DAT
# BU_DVDFS.DAT

def main():
    #Open it! Hope you have admin because it won't open if you don't!
    disk = open('\\\\.\\PhysicalDrive1', 'rb')
    #Is it one of our drives?
    if not has(disk, b'HDD REMAIN'): 
        print("Not sure this is one of our drives!")
        exit(1)

    global blockSize
    #Now lets determine the block size
    #In an ideal world block == sector size but we don't live in that world.
    #They decided to make is so the number of blocks always fits in a 32 bit integer.
    for x in range(0, 5):    #512, 1024, 2048, 4096, 8192, 16384
        blockSize = DEFAULT_SECTOR_SIZE * (2 ** x)
        disk.seek(blockSize)
        disk.seek(16, 1)
        if has(disk, b'HDDFs 00.07 '):
            break
    else:
        print("Could not determine block size!")
        exit(2)
    
    print("Block size is: " + str(blockSize))
    print("Drive size is at most " + str(blockSize * 0x100000000) )

    #seek to the start of the file system!
    disk.seek(blockSize * 2)
    numberOfBlocks = readInt(disk)
    allocatedBlocks = readInt(disk)
    freeBlocks = readInt(disk)
    
    #assume the table is only one block wide.
    FB000_00_end = blockSize * 3
    files = []

    #seek to the start of the table
    i = blockSize * 2 + 0x80
    while i < FB000_00_end:
        disk.seek(i)
        firstBlock = readInt(disk)
        sizeInBlocks = readInt(disk)
        if firstBlock != 0:
            disk.seek(18, 1)#I know nothing about these fields
            nature = readString(disk, 5)
            name = readString(disk, 12)
            print(name + " " + nature + "\t" + str(firstBlock) + "\t" + str(sizeInBlocks) )
            files.append(FileDescriptor(sizeInBlocks, firstBlock, name, nature))
            if name == "FB000_00.LST":
                FB000_00_end = (sizeInBlocks + firstBlock) * blockSize
        i+= 0x40
    
    for fd in files:
        #do something smarter, like with a loop and do it in chunks.
        if fd.size < 200 * 1024 * 1024 :
            MakeDir(directory + fd.directory)
            ff = open(directory + fd.path, "wb")
            fd.seek(disk)
            ff.write(disk.read(fd.size))
            ff.close()

    disk.close()

def MakeDir(path):
    try:
        os.makedirs(path)
    except OSError as e:
        if e.errno != errno.EEXIST:
            raise

def BlockSeek(disk, offset):
    diff = offset & (blockSize - 1)
    disk.seek(offset - diff, 0)
    if diff != 0:
        disk.seek(diff, 1)
    
class FileDescriptor:
    def __init__(self, sizeInBlocks, firstBlock, name, nature):
        super().__init__()
        self.sizeInBlocks = sizeInBlocks
        self.firstBlock = firstBlock
        self.name = name
        self.nature = nature
        self.size = sizeInBlocks * blockSize
        self.directory = (nature + "\\") if len(nature) > 0 else ""
        self.path = self.directory + name
    def read(self, disk):
        disk.seek(self.firstBlock * blockSize, 0)
        return disk.read(sizeInBlocks * blockSize)
    def seek(self, disk, offset = 0):
        BlockSeek(disk, self.firstBlock * blockSize + offset)
    
def readString(disk, length) :
    data = disk.read(length)
    for i in range(0, length - 1):
        if data[i] == 0:
            return data[0:i].decode(encoding="ascii")
    return data.decode(encoding="ascii")

def readInt(disk) :
    return int.from_bytes(disk.read(4), byteorder='little')

def has(disk, data) :
    return disk.read(len(data)) == data

if __name__ == '__main__':
    main()