cgsecurity / testdisk Goto Github PK

Like this guy on Askubuntu I'm trying to recover files from a external hard disk drive with some bad sectors:

In my case my 1TB HDD connected with USB 3.0 will finish in... 5 years. If this is sadly true I suspect the user should receibe a warning. I also have a superfast SSD drive on my main computer, I wonder why photorec doesn't just make an image and work on the ssd.

This is a great library! I recently tried to restore a JSON file and it wasn't working. After looking at the code, I realized that testdisk only restores JSON files starting with a '{'. It would be nice to also have supports for those starting with '['. Looking forward to a response :-)

The maximum size for a single file in a FAT32 filesystem is 4294967295 bytes. It seems that if you are recovering files to a location formatted with this filesystem (such as a USB thumb drive), Photorec will not properly detect the maximum filesize limit.

I have noticed two things that seem to happen when a Photorec output file reaches the maximum FAT filesize:

The free space on the destination will continue to shrink even though the file has been maxed out, as if photorec continues allocating data to the filesystem without checking if the current file allocation is available and can still be written.

In some cases it seems as though Photorec will no longer find any more files to recover until the existing maxed file is deleted from the destination , even though there are still other files of that format to be recovered.

There should probably be a check done if the recovery destination is on a FAT32 filesystem so that a recovery stream stops when it reaches the maximum file size, and a warning is shown.

Dealing with a large drive I noticed the sector number drop in the recovered filename. Checking with hdparm the data at the sector didn't match but earlier files did. but adding a multiple of 2^32 located the sector.

Looking (quickly) at the code it uses 'unsigned long' for the file locations but then casts to 'unsigned int' when creating the filename.

During

Pass 1 - Reading sector

when the list of recovered files gets to 11 lines, the 11th ("others") makes disappear the "STOP" label.

At least this is what I see on testdisk-7.1-WIP running on W7. A minor issue but puzzling at first.

During partition "Deep Search", if it takes too long, one can abort the process by hitting Enter.
However, there seem to be no way of stopping data copping from detected partition, other then killing the testdisk process.

Title basically says it all. To select any of my normal drives, I would need to use qphotorec with sudo, though doing so leads to a blank window. Atleast this is the problem, installing qphotorec on a Fedora 23 live usb-stick, using the official package. No such problem exists (of course) with the command line tool photorec, which I then used.

It's commonly used partition type must be supported with this great tool.

Hi All
I am just trying to compile the testdisk on my Linux server, for now I don't want QPhotoRec and QT is not installed on my server either, so I just wonder is there any means to disable the QPhotoRec during compile ?
Thanks in advance

During a recovery run on an NTFS volume to recover .txt, .tx? and .vdi, about halfway through an old Claws Mail mailbox (MH format I think) embedded in an old, possibly even deleted .vdi (Virtualbox Virtual Machine, probably ext4) was misidentified as a .emlx file. I don't care about this mailbox, but because there was no "</plist>" end marker the file grew to many gigabytes and made further recovery impossible. This happened reliably with both photorec_win.exe and qphotorec_win.exe (7.0 Sat Apr 18 13:02:01 CEST 2015).

I resolved this by upx decompressing photorec_win.exe and hexediting the strings "Return-Path: " and "Received: from" to start with a "D" instead of an "R". This is a dirty hack but it works.

I'm just letting you know so you could make this great piece of software even better. Maybe by making it possible to disable .emlx files in some way or making the detection more robust?

Relevant piece of code: https://git.cgsecurity.org/cgit/testdisk/tree/src/file_txt.c?id=ae341302369a4a07feda0e94b4ff432217ee3916#n996

My friend has an external hdd with different media on it. It can no longer be accessed via windows (windows wants to format the drive before being useable). The hdd is 1TB and not manually partitioned. Gparted found ~950GB unallocated memory and ~100mb metadata partition

Therefore I tried to help and flashed SystemRescueCD onto a usb drive.

After consulting the wiki I tried to follow step by step. When I needed to select a partition table type I followed through with the assumed one (Intel) because the wiki states that this would be most likely correct.

After writing this to drive I was advised to reboot by the program which I did. Thereafter I was unable to read the drive anymore. It was still shown in lsusb and /var/log/messages with its metadata and write protection was apparently off (stated in message log). But after loading the metadata it states that the drive has a input/output error and read error.

Is there any hint what could be wrong on my side? Or could this be a locked resource because of the reboot?

I just came back from my friend and unfortunately cannot provide any screenshots.

Kaitai Struct is a declarative language to declare file formats. Kaitai Struct compiler generates parsers for the formats provided with ksy definition.

1 signatures can be harvested from ksy files. Just find the field with "contents" property with fixed position (Kaitai Struct compiler precomputes offsets for every field).
2 ksy descriptions can be compiled into some interpreted language (lua, js, python, or byte-code (which is yet to be developed) ), which can be used by photorec to check format of files.
a) for now we can check if enum fields have valid values and that sizes of nested structures are in aggreement with each other.
b) checksum verification is to be developed yet

bunzip2 testdisk-7.0.mac_intel.tar.bz2
tar -xf testdisk-7.0.mac_intel.tar
cp testdisk-7.0 [usbstick]

OS X - El Capitane - recovery boot

cd [usbstick]/testdisk-7.0
./testdisk

results in:

Killed: 9

On my old OS X SnowLeopard, it works.

compile.sh is downloading HTML into progsreiserfs-0.3.1-rc8.tar.gz

I tried the URL with both curl and links.

Here's the beginning of it:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <meta name="google-signin-scope" content="profile email https://www.googleapis.com/auth/plus.me">
    <meta name="google-signin-client_id" content="113750670845-7qn835ic6lh4csohe8o1gvco5d5psfrs.apps.googleusercontent.com">

    <title>OSDN Conference 2016</title>
    <link rel="icon" type="image/png" href="/assets/images/logo_32x32.png" />

    <!-- css -->

I asked on askubuntu.com about warnings of geometry mismatches, and apparently these warnings don't make sense for partitions over 8GB in size.

I tried to build to see if the warnings still appear on the latest version, but I got stuck at Issue #10.

For example, in file_prd.c, specific values are tested at fixed offset but buffer_size is not used to test if offset 0x17 is reachable.

static int header_check_prd(const unsigned char *buffer, const unsigned int buffer_size, const unsigned int safe_header_only, const file_recovery_t *file_recovery, file_recovery_t *file_recovery_new)
{
if( buffer[0x0d]!=0xdb || buffer[0x0e]!=0xe4 || buffer[0x0f]!=0x40 ||
buffer[0x15]!=0xdb || buffer[0x16]!=0xe4 || buffer[0x17]!=0x40)
return 0;
reset_file_recovery(file_recovery_new);
file_recovery_new->extension=file_hint_prd.extension;
return 1;
}

Hi,
In some .E01 encase files (and direct disk access) I cannot see $MFT file, while Encase imager shows it without problems.
Can you please check this one?
Maybe you need to update the libewf library or there is something else?

I am using TestDisk 7.1-WIP from April 2018.

Photorec is just a fast binwalk without features useful for re but with features useful for recovery. I guess combining all the features in a single app may be beneficial.

Unable to create error when recovery path contains non-Latin symbols.

Windows 10 (1803)

QPhotoRec 7.1 to be precise.

Unable to set start/stop sector more than UINT_MAX on creating partition (4294967295)

Because now it silently starts without any output, and it can be hard to not to confuse it with hang or glitch.

Is there a chance to get testdisk functionallity in a library for dynamic binding for the future?
With that it would be possible to include the functions in a nice GUI with translations and so on or simply extend the functionallity of tools like gparted. It would enable people with less knowledge about partitions and filesystems to recover their data from defect media.

Seems like the binary for MacOS still in 32 bit. This cause the binary failed to run since MacOS Catalina removing support for 32 bit application.

sh: ./photorec: Bad CPU type in executable

Ubuntu 18.04 Bionic
testdisk (7.0-3build2)
TestDisk 7.0
GCC 7.2

Selecting "No Log" option, logging option is asked twice.

To reproduce:

• testdisk, select "No Log"
  After selecting No Log, the same logging menu reappears again a second time. Selecting No Log the second time moves on to the program menu.

I need to call testdisk recovery mobile images by scripts, could you give some advices?

TestDisk sometimes gets confused between .h and .sh files. I have a shell file, starting with:
#! /bin/sh

However, it also contains the line:
for f in /etc/apache2/sites-enabled/* \

which causes file_txt.c line 1302-1303 to mark it as ".h" instead. I would expect the shell-header to take precedence...

Hi, sorry for stupid question.
Can I search in the drive/image open with testdisk?
Let's say I want to search for file name in the current view or recursively through partition. Also, including deleted files.
Thanks.

When you select to copy one or more files there is no indication of current progress or estimated time. Would be nice to have a progress bar like in the wget.

At first I want to thank you for that nice tool, it really saved my life by recovering a lot of photos and videos, I lost (due to my own stupidity -.-).

Unfortunately it seems like all my RW2 files are broken, despite others like CR2 or JPG just work perfectly.

Filenames are like "f15399472.rw2", so from the documentation, the "f" indicates that it's a recovered file (not "b" for broken or whatever), if I got this right.

The broken files all look something like the following screenshot, so you can basically see the photo, but just totally messed up.

According to the files list here, RW2 seems to be supported:
https://www.cgsecurity.org/wiki/File_Formats_Recovered_By_PhotoRec

The Photos were taken with a Panasonic Lumix FZ-1000, and some with a G81 (similar to G80/85).
Of course I can send/attach one or more RW2 files, if you like/need to investigate this any further.
Unfortunately I could not use this tool to check the files, as they are around 20MB in size: https://www.cgsecurity.org/photorec/

Is there anything else I can do, to make this work or help you?

Please update install/compile instructions.

./configure not found

logs
I'm on W10 CU trying to scan a dynamic disk.
Until the 350 cylinder, disk is being actually read and all. Then I still get 100% disk activity from Windows, but 0MB/s throughput.

Besides, I'm not sure if that filewin32_setfilepointer(\\.\G:) function is even supposed to be there, since I have no G:\ disk attached in the first place.

You cannot guess where recup_dir must be situated: you don't know on which drives a user is going to make recivery procedure. So make a user to make a choice about recup_dir

I am trying to add a custom formats to photorec.sig for common bioinformatic files (see http://samtools.github.io/hts-specs/ for more information on some of them), but it is quite difficult because some of the formats are compressed with bgzip (an extension of gzip based on blocks).

It looks like some compressed signatures are directly implemented in the .gz format (e.g., xml.gz) and thus a new compressed format can be added by modifying https://github.com/cgsecurity/testdisk/blob/master/src/file_gz.c
Nevertheless, this does not allow to identify any kind of file compressed (with .gz or other algorithm) and thus is difficult to extend. In addition, it might be difficult for users to create a photorec.sig for a compressed extension where the signature is a string.

It will be nice if the photorec.sig can have some kind of mechanism to indicate compressed signatures with certain algorithms, or to add better extensibility of compressed formats (e.g., a boolean field for the file_hint_t struct to indicate that it might use first a compressed format, and then identify the signature after decompression).

Thanks in advance!

I am trying to recover a specific text file that was overwritten by Windows. When I use the program it gives me something like this... D:\recup_dir.1\f0280112.txt

All recovered files have these weird names and the modification dates are the same. There are over 6000 text files with no way to easily find what I am looking for.

Did I do something wrong here?

Several version of testdisk (include the GIT version) die 😀

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7982cf5 in termattrs_sp () from /lib64/libncursesw.so.6
(gdb) where
#0 0x00007ffff7982cf5 in termattrs_sp () from /lib64/libncursesw.so.6
#1 0x00007ffff797ffe8 in _nc_setupscreen_sp () from /lib64/libncursesw.so.6
#2 0x00007ffff797b56f in newterm_sp () from /lib64/libncursesw.so.6
#3 0x00007ffff797ba58 in newterm () from /lib64/libncursesw.so.6
#4 0x000055555558b5d9 in get_newterm_aux ()
#5 0x000055555558da35 in start_ncurses ()
#6 0x000055555555b30b in main ()

I have ncurses-6.1 installed here (Gentoo Linux)

I added for example Czech translation on transifex, long before current release, but it wasn't merged :(

The Nikon SLR cameras have an NEF format file type. Is there any chance that you could add support for these files?

Thanks in advance

Awhile ago, I used photorec to recover the files from someone's failing hard drive. He had some engineering program that created files with extensions I had never heard of before. photorec did a great job of recovering files, and I ended up with a folder that had hundreds of thousands of files in it. I found this annoying, since most of them were useless (.exe, .dll, etc.). If photorec recovered the files he needed, I wouldn't know. So I wrote a script that creates a folder for each file type that photorec recovers, and moves all files with that extension into that folder. It handles files with no extension by moving them into their own folder. I then gave him the external drive he had provided for me to put recovered files on and he was able to find what he needed at his leisure.

Would you guys like to include it with testdisk and photorec for systems that have bash available? If so, where in the source tree should I upload it?

A high level overview of how they a structured is located here: https://www.goprorecovery.co.uk/manual/GoProVideoFileStructures.html

This product has the highest success rate. Would be nice to have these video formats to be recoverable.

Recently I tried to recover some files from .img file(that I grabbed from a 8GB SD card, which has corrupted partition table) and I found this issue:

QPhotoRec does not load .img files(or other images) from a folder/file that has any Japanese characters in its path.

I got this issue with the following environment:

• Windows 10 Home, build 17763.504 64 bit
• CPU: Intel, i7-4712HQ and MEM: 7.89GB of usable RAM
• Language: Japanese
• The file system of the storage that my .img file was on: exFAT
  also I got the same issue with NTFS.

How to reproduce:

1. Put your .img file in a folder that has some Japanese characters
   (In my environment: "D:\作業フォルダ\Data.img" )
   Note: You can include some Japanese characters in your file name instead of folder's name to reproduce this issue.
2. Start QPhotoRec, choose "Add a raw disk image..." in the image selection, then select the file of step 1 (make sure to check the path to the file has at least 1 or more Japanese characters)
3. Done! Nothing changes on GUI.

I also confirmed this issue with some other paths: "C:\ほげ\ABC.img" "C:\Hoge\ぷよ.img".

I got a bit worried but anyway I could recover my files from the image by moving it to D:\ . I must thank for this good software, anyway.

I'm running an Analyse, and the percentage doesn't seem to be updating correctly:

Disk /dev/disk2s1 - 8001 GB / 7452 GiB - 1953503744 sectors
Analyse sector   101449728/1953503743: 00%

101449728/1953503743 = 0.0519, so it should say 5% instead of 0%

I can't see any problem in the associated line:

Could you help me with the command to delete mbr partition table via script please.

Trying the following commands did not work for me.

<PATH_TO_TESTDISK> /cmd <DEV_ENUMERATION> delete -> Asks for (Y/N)
<PATH_TO_TESTDISK> /cmd <DEV_ENUMERATION> noconfirm,delete -> Does not ask, but is not working

As a reference I used your pdf document: https://www.cgsecurity.org/testdisk.pdf

The OS is Windows 7

It looks like photorec makes no attempt at calculating the correct length of ELF files, instead it just assumes an ELF to extend until the next start position of the next file it recognizes.

This seems wrong, as the extracted ELF files can be too large, can have garbage at the end.

It seems as if you hardcoded your token->secure to the .travis.yml file.
You might want to change this to a setting or some non public place.

GPT has no concepts corresponding to CHS disk geometry; it's exclusively LBA. I'm not aware of any GPT partitioning utility using cylinder alignment (maybe some early Itanium tools did, but encountering that is extremely unlikely; the GPT specification in fact was released long after IDE hard drives with virtual geometry became common), and have never in my practice seen anything other than power-of-2 alignment (usually 1MB) on GPT disks.

TestDisk should default to searching on megabyte boundaries on GPT disks (perhaps with an option for CHS, in case someone does come across an old Itanium HDD), and not display CHS-based messages, like "Analyse cylinder X/Y".

Version: 7.0
Compiler: GCC 5.3
ext2fs lib: 1.42.13, ntfs lib: libntfs-3g, ewf lib: none, libjpeg: libjpeg-turbo-1.4.2, curses lib: ncurses 6.0
OS: Linux, kernel 4.4.0-53-generic (#74-Ubuntu SMP Fri Dec 2 15:59:10 UTC 2016) x86_64

While scanning an external drive mounted as /dev/sdc (Whole disk), the drive got detached from the operating system and the device path was removed.
The Photorec output continued on as if nothing had happened even though the target disk and its device path were removed.

Expected it to have shown an error and halted the scan upon the target drive being detached from the computer.

According to the instructions you need to do
./configure && make && make install
./configure - not available!

─[root@Https]─[/home/User/testdisk-master]
└──╼ #./configure
bash: ./configure: Нет такого файла или каталога

When recovering from a large disk, one might want to omit some files which are too big, or known to exist elsewhere, etc. Conversely, one might want to compare recoveries with different parameters, different tools, etc.

Currently, one still needs to have enough free disk space to have all these redundant files, or to have all versions of recovery. I suggest to have an option where photorec performs all the analysis, but doesn't actually write the recovered data to disk. It is simple enough, afterwards, to recover the actual files one wants by reading the xml, and most likely much less time consumptive than the original analysis. (Of course, there can also be a feature for processing a report.xml and recovering by its spec, so one can simply delete the items which are unwanted, but I assume this is a much larger feature.)