Where do i write the log file (map file) […] Can I also write that to the new target disk?
Create a filesystem on the target disk, mount it and tell ddrescue to write to a regular file in the filesystem, not to the device (/dev/sdc in your commands). Let the mapfile be another regular file in the same filesystem (sane idea: in the same directory).
This will work well if the target disk is bigger than the source one, so even after the filesystem takes some space for its structures there will be enough room for the image (of the size of the source disk) and the mapfile. But even if the target disk is not bigger, a filesystem with compression and/or the -S/--sparse option of ddrescue may be enough to squeeze the image into the filesystem. However there is no easy way to tell in advance if data can be compressed/sparsified enough. If the source drive was healthy, you could do this: Clone only space in use from hard disk. But in your case of a failing drive this approach is not recommended.
Fortunately you said (in a comment) that "the target drive is double the size of the failing one". Create a filesystem in the target drive, even without compression, and the image and the mapfile will fit. The filesystem may be in a partition (/dev/sdc1) or on the entire device (/dev/sdc). Please see Uses of single-partition disk configuration. But read the whole current answer before you decide.
I use Btrfs for this even when I don't need compression, because Btrfs supports copy-on-write. After ddrescue finishes I remove write permission from the image, create its copy (cp --reflink=always …) which initially takes virtually no additional space. All operations that modify the image (e.g. fsck) are performed on the copy. In case of any trouble I still have the original file, I can always start anew. I believe ZFS is similarly useful but I have no experience with it.
Having an image of the entire disk as a regular file, you can examine its partition table, if any (assuming ddrescue managed to read it), with gdisk -l /path/to/image or some equivalent command. And you can mount filesystem(s) from it (assuming ddrescue managed to read enough data for the filesystem(s) to be mountable). Useful commands: losetup, kpartx or just mount -o offset=…. Reading files is thus possible. And you can copy them to the very same filesystem that holds the image.
There are at least two scenarios when copying directly to /dev/sdc is justified:
- You need to boot from the copy like you used to boot from the failing disk.
- You want to use a tool (e.g. recovery tool) that cannot use a regular file; a tool that insists on working with an actual disk. Note such limitation is not a Unix-y design. The tool may be some Windows tool or the Windows itself.
How big is the log file typically?
The mapfile consists of a header and such (about 350 bytes) and a list of data blocks: one line (about 30 bytes) per continuous block of sectors that hold the same status. A sane worse case scenario is when each physical sector of the drive has different status than its neighboring sectors. Then it would be one line per physical sector, i.e. about 30 bytes of mapfile per 512 or 4096 bytes of the source disk; so the size of the mapfile should not exceed 6% or 1% of the size of the source disk.
So in theory it can reach gigabytes; but reaching this size (i.e. testing all sectors of a drive whose every other sector is bad) would take ages. In practice expect more fortunate layout of faulty sectors. Expect mapfile taking kilobytes, maybe several megabytes.
If you need (or simply choose) to copy the source disk directly to the target one (not to a filesystem as advised above) and the target one is substantially bigger, then you will be able to save the mapfile on the target drive. A possible approach is like this:
On the target drive create a partition that starts far enough, so even after ddrescue overwrites a large fragment (of the size of the source disk) starting from the very beginning of the target disk, the content of the partition will be untouched. Make the partition big enough to hold a filesystem able to hold a mapfile of the expected size. But leave room (1 MiB is enough) at the very end of the disk in case it's going to use GPT. Create a filesystem in the partition.
Make sure you can mount the filesystem with mount -o offset=… /dev/sdc … (as opposed to mount /dev/sdc1 … or so). Leave it mounted. Note the offset on a piece of paper.
Run ddrescue and let it write to /dev/sdc, but place the mapfile in the mounted filesystem. This will overwrite the partition table of sdc; but you will still be able to mount the filesystem holding the mapfile, because you know the offset.
After ddrescue does its job (possibly over multiple sessions), check the partition table on /dev/sdc. DOS partition table in MBR or the primary GPT will originate from the source disk (unless ddrescue failed to read this part).
(Note: there may be a problem with the logical sector size, I will get to it later. For now I assume there is no problem, but please read the whole answer before you act.)
If the copied partition table is a DOS partition table in MBR then it should be OK.
If it's GPT then you will need to fix the secondary GPT. The copy of the secondary GPT from the source disk is now somewhere in the middle of the target disk; normally it should be at the end. And it may be that at the and there is an old secondary GPT from /dev/sdc that has nothing to do with the copy. gdisk /dev/sdc should detect the discrepancy and offer you an option to fix the secondary GPT basing on the primary one (manual way: r for recovery options, then d to rebuild backup; see "Manual Recovery Procedures" here).
You will still be able to mount the filesystem holding the mapfile (with offset=…), but note according to the partition table this part of the disk is now unused. You can create an entry in the partition table to access the filesystem more easily (compare this answer of mine) or expend the filesystem as if it was never there.
I read it's not a good idea to connect the hard drive to a USB enclosure? What alternative do I have though if I only have one SATA connection?
Another computer in the network that will either
- mount the target disk and let you create a file on it via NFS or similar protocol; or
- expose the target disk for you to use as a block device.
But a USB enclosure may be fine. It will probably be fine if you know how to deal with its possible quirk (we will get to it).
If I had to choose, is it better to have the new drive in the SATA port and the failing drive in the enclosure or vice versa?
The enclosure is an additional layer that may misbehave or introduce unexpected phenomena if the disk inside misbehaves. Therefore I would prefer to use it with the healthy target disk. There are other aspects though (mainly the quirk, we will get to it).
Do I have to ensure the drives are not mounted before running ddrescue?
The source drive must not be altered. You cannot read all the data at once, you read in chunks. If the content changes between two readings, there's a risk you will get an incoherent image. Compare "panorama fail" in photography: different parts of the image are taken in different moments, while the world (source) is not perfectly still.
The source drive may be mounted read-only. But since the drive is failing, any reading may worsen its state; so it's better not to read unnecessarily. Keep the source not mounted.
If ddrescue is about to write to a regular file in a filesystem on the target drive, the filesystem must be mounted. If ddrescue is about to write to the target drive, the fragment that is going to be altered must not be mounted; but some further part may if you have a good reason (like storing the mapfile in the procedure described above).
Do I have to check if the disk sector sizes are the same and pass any parameters to ddrescue if they're not?
Yes, you do have to check. But no, ddrescue itself should work; although there is a parameter that can be adjusted to the physical sector size of the source disk (but not because the sectors differ). Different sector sizes may be an issue later.
Additionally some USB enclosures introduce a quirk that can interfere.
First, get familiar with the concept of "physical sector size" vs "logical sector size". Useful links:
In short: you talk to a drive using its logical sectors, but internally it reads/writes data using its physical sectors. The OS can request as little as one logical sector, but if it happens to be smaller than the physical sector size then a whole physical sector will be read anyway, but only its part (the requested logical sector) will be returned.
While invoking ddrescue, you can specify the sector size in bytes (-b, default value 512) and cluster size (-c, sectors to copy at a time, default value 128). At first (copying phase) the tool reads whole clusters, many sectors in each read; but later (trimming, scrapping phase) it tries single sectors, one at a time. Well, not "sectors", rather "what it thinks are sectors".
If you specify -b smaller than the actual physical sector size of the device, then in case of a read error ddrescue will eventually try and retry to read parts of a physical sector. Internally the drive will try to read the whole physical sector each time; and if by chance it succeeds then some data will be discarded anyway, despite the fact it could fill the neighboring what-ddrescue-thinks-are-sectors in the image. Each of these neighboring fragments will require its own tries. If the disk is faulty, you really want to get as much data in as little number of reads as possible because any additional operation may damage the drive further; so -b being too small can be quite bad.
If you specify -b larger than the actual physical sector size of the device, then in case of a read error ddrescue will eventually try and retry to read more than one physical sector at a time. If it never succeeds, a gap larger than a physical sector will remain in the image. It's possible some fragments of it could be read without problems if only the tool used more adequate sector size.
I have no idea what happens when you specify a totally insane sector size (e.g. 511, 513 or 4444). I haven't even tested.
The default value of -b is 512. It is suboptimal for a drive that uses 4096-byte physical sector size. This is the parameter you should adjust (regardless if sector sizes differ between your two disks).
I think the sector size of the healthy target does not matter that much, ddrescue just writes to a seekable file (regular file or a block device).
Then your enclosure may interfere. Let's suppose you decided to use it with the faulty disk. Some enclosures change the logical sector size which is not crucial if you know the physical sector size. But I own an adapter that lied to me about the physical sector size! It reported 512 for a disk that really uses 4096. It was a faulty disk, I used ddrescue -b 512 …, then all bad sectors appeared in bunches of 8. This got me thinking. The true value of 4096 was reported after I connected via SATA. The disk died in the process and I suspect I could have recovered more data if I used -b 4096 from the start.
Earlier I wrote "I would prefer to use the enclosure with the healthy target disk". True, but if your enclosure can change the logical sector size then these are possible if you use it with the target disk:
- You decide to create a filesystem in a partition, then you write an image as a regular file. All works. Then you scrap the faulty disk and move the target disk from the enclosure to the SATA connection. The logical sector size changes and you experience exactly this: Drive can't mount due to wrong fs type.
- You decide to write to the device but then the partition table in the copy does not correspond to the logical sector size used by the target disk.
Note the latter case may occur even without the enclosure; or the enclosure may fix the problem (as long as it's in use). It all depends on logical sector sizes of the two disks and on how (if ever) the enclosure interferes.
Good news: with mount -o offset=… you should be able to mount filesystems even if the partition table makes no sense. Follow the last link, my answer there explains the details.
But if you copy to the target device in order to boot from it and the partition table turns out to be invalid because the logical sector size is now different, then you will need to fix the partition table. Fixing may or may not be possible.
