Data Recovery with Open Source Software
Overview and SMART data
There are some cases when customer can't afford data recovery and it does not hurt if you fail or make it worse. Following procedure should not be considered as any sort of sollution for important data recovery - if data worth recovery lab(400-600€ ++ ) - DO NOT TRY IT.
One of the most important thing to check before even trying to access data is SMART report of the hard drive. on Ubuntu/Debian based Linux distribution you can access it directly through standard disk utility. There are plenty of programs for MacOS / Windows, the most important thing you need to know is that many of them do not interpret "pending sectors" value and might show almost dead drive as something like "80% health" which might be absolutely wrong, so ignore this "summary" information.
If you connect it via SATA, be sure SMART reporting is enabled in Bios settings. There are many values in list, but there are few which are most important:
Current Pending Sectors Count: This value represents the number of sectors which are marked as "unstable" and will be transfered by HDD internal firmware as soon as they are successfully read. The problem is, that if you actually have that value with a number more than zero - you have already lost some data and your drive is already failed. Since reading of one sector and moving it to another place takes just a tiny amount of time, you should not ever catch it in SMART. As soon as unstable sector is found, drive tries to read it multiple times untill it is being transferred to another potentially good block and it adds its position to the Translator(map of the LBA sectors on drive) and G-List(Growing list of bad sectors on drive). The problem is, that higher than zero value is almost always indicates major reading problem: either scratch or (much worse) head damage. If you see this value growing rapidly while transferring data - STOP immediately. First of all, it might switch your drive into a "busy" mode and drives stops being recognised by the OS. Second of all, constant writing to the service area (updating G-List + Smart values) might lead to firmware corruption which will also render drive unmountable.
Reallocated Sector Count: once unstable sector is successfully transferred, CPSC value is decreased, RSC value is increased. Having 100-500 bad sectors is not as bad as it seem to be (Note that many programs show hex values, so you might need to convert it to decimal because "F00" would indicate 3840 bad sectors).
Uncorrectable Error Count: in theory, CPS which were failed to read will be marked as lost and moved to UEC. In practice this value might stuck on zero and drive might stop being recognized by the OS because firmware might be configured to figure pending sectors before going into ready state (during this process it might as well hit some bad sectors in the service area and after you power-reset the drive, it will not be able to boot/initialize at all)
High Fly Writes: this value is a common indication of failed head. 1-100 writes are mostly fine, but you will see this value increasing very rapidly if you have bad head.
G-Sense Error Rate: this value indicates how many times drive indicated some mechanical steress and disabled operation. For example if you turn your laptop too fast or drop it while its on - this value will increase.
Ultra ATA CRC Error Count: this value might indicate bad sata connection(for example on MacBook Pro 13" 2012 Mid this is a clear indication of broken cable)
So long story short: while transferring your data it is really important to check and update SMART values every 5-10 seconds and keep an eye on Pending / Reallocated sectors. You should not try to clone drive with increasing pending sectors, maximum what you could do is to backup most important files. Not VERY important though because it might, as well, make professional data recovery impossible(for example, if due to multiple retries head crashes to the surface and scratches it... exactly on the location of your most important data).
If you have a HDD drive with bad sectors, NEVER use any sort of file recovery software. Most basic rule of data recovery - work with an image/clone, not source.
Software/Hardware:
- Linux (Ubuntu/Mint/etc)
- gddrescue (sudo apt install gddrescue). Read its manual at least briefly.
- smartmontools and ddrescueview
- APFS Fuse
- SSD Adapter. It does not really matter if you use USB-SATA or direct SATA connection. For macbook drives use usb-apple adapter, thats totally fine (there are gen.4 PCI-E cards, but they are difficult to find / too expensive. Check Apple SSD page for some links
- KVM switch or USB with on/off button
IMPORTANT NOTE: you should disable SSD trim on data recovery device especially if you use some file signature-search software like Diskdrill or Photorec.
Ubuntu/Mint: sudo rm /etc/cron.weekly/fstrim
Windows: fsutil behavior set disabledeletenotify 1
MacOS: sudo trimforce disable
Recovery(Terminal commands):
Theory:
- If you have failing/freezing SSD drive, you should try to make an image of it as soon as possible. You may try to copy 1-2 most important files if you can mount it but never try to transfer big amounts of data. Obviously Photo Library is actually a huge folder with thousands of files. After you cloned the drive you may try data recovery software, disc utility repair, etc. DO NOT ATTEMPT TO FIX ORIGINAL DRIVE.
- Note on Apple-Samsung SSDs: they tend to freeze once they hit a bad block. If it is frozen, you need to power-reset the drive and continue data recovery. This might skip few block of data, but in most cases it is 0.1-5% of whole drive so it might not be critical for most of the data
sudo lsblk
this will show you list of block devices. if you cant understand it, simply open Disk utility or gparted to see dist identities in graphical manner. Linux disks look like that: /dev/sdXY where X is disk number and Y is a partition.
my normal command looks like that:
sudo ddrescue -df -r0 /dev/sdX /dev/sdZ customername.log
X is a SOURCE ID , Z is TARGET ID. -df uses forced overwrite (to write on other drive) -r0 means it will not retry failed block and go further. Log is necessary so if you re-run the command it will continue from the point it was stopped.
I usually use a physical drive as target, brand new clean drive or pre-wiped(owerwritten with zeroes) one.
now you should see whats going on with the cloning process: if it goes through with couple of errors - you are lucky. If it freezes for more than 10-20 seconds, disconnect the drive ( or use KVM switch ad a reset button).
Example bash script for such cases:
#!/bin/bash while : do sudo ddrescue -df -r0 /dev/sdX /dev/sdZ customername.log sleep 5 done
if device disappears, ddrescue quits and script sleeps a little bit. After you restart it it starts with the next block and goes further. if you have 100-300Mb/sec transfer rate, it might take you 1-2 hours to clone a drive.
after you have a 90%+ clone, you can try to mount APFS partition with APFS Fuse(link above) or use known good data recovery tools like Diskwarrior (HFS-only) iBoySoftware or Diskdrill. In some cases you might be able to mount cloned drive or fix it with MacOS disk utility
Some interesting articles for further research
HFS file system repair/recovery in Linux https://photobyte.org/recover-damaged-mac-hfs-disk-linux/
Images and corrections pending, by questions you can reach me on Rossmann's discord as @inwerp