Why my 3TB disk shows an actual size of ~800 GB with several tools except smart tools?

Question

System disk started to fail and hence I ran out to buy a new HDD 10 minutes before closing time :/ This is for a 32-bit system disk. From earlier, relatively long time ago, I had checked out Seagate Barracuda 2TB, but got convinced by the salesman to buy a 3TB WD Red NAS Harddrive as a lot of customers had returned Seagate disks with errors over the last months. Seagate in various colors. Went from 2TB to 3TB due to cost and my inkling to dive into that league – and force myself into GPT. NAS disks should be OK as OS disks, e.g. for this box that is up running 24/7 with a lot of activity as coding etc. (as in low I/O), no gaming, some media processing as converting cam cordings, rendering (though that is mostly done by cluster).

My main question is regarding size of disk using various tools.

(f)disk:

#fdisk -l
#disk -l

Disk /dev/sdc: 801.6 GB, 801569726464 bytes
255 heads, 63 sectors/track, 97451 cylinders, total 1565565872 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

lsblk:

#lsblk
sdc      8:32   0 746.5G  0 disk

lshw:

#lshw -class disk
*-disk
     description: SCSI Disk
     physical id: 0.0.0
     bus info: scsi@2:0.0.0
     logical name: /dev/sdc
     size: 746GiB (801GB)
     configuration: sectorsize=512

Why are they all yielding 764 GiB? Could there be any issues?
As this would require GPT; any things I should be aware of / take extra caution with? E.g. any issues with 32-bit?

S.M.A.R.T:

Have done smartctl -t <conveyance|short> which both yield OK.

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%         0         -
# 2  Conveyance offline  Completed without error       00%         0         -

S.M.A.R.T also yield correct size (obviously):

# smartctl --info /dev/sdc
smartctl 5.43 2012-06-30 r3573 [i686-linux-3.5.0-51-generic] (local build)

=== START OF INFORMATION SECTION ===
Device Model:     WDC WD30EFRX-68EUZN0
Serial Number:    WD-XXXXXXXXXXXX
LU WWN Device Id: X XXXXXX XXXXXXXXX
Firmware Version: 82.00A82
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   8
ATA Standard is:  ACS-2 (revision not indicated)
Local Time is:    Sat Jan 10 18:40:58 2015 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

Not much relevance but 1 hour usage of various tests etc. yield:

Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   100   253   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   100   253   021    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       1
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   100   253   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   100   100   000    Old_age   Always       -       1
 10 Spin_Retry_Count        0x0032   100   253   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   253   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       1
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       0
193 Load_Cycle_Count        0x0032   200   200   000    Old_age   Always       -       6
194 Temperature_Celsius     0x0022   131   125   000    Old_age   Always       -       19
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   253   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   253   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   100   253   000    Old_age   Offline      -       0

Sub question:

If above OK, any other things that is smart to do before installing OS on disk?

Note that by > 2TB I meant above 2TB, in original title. Sorry for mess. — user129107, Jan 10 '15 at 19:15
@CristianCiupitu: Intel 82801H (ICH8 Family) 4 port SATA IDE Controller. — user129107, Jan 10 '15 at 20:06
The "Beyond 2TB" article from Seagate mentions: "If your system uses the Intel RST driver, your 3TB disk drive may appear to be 800GB. See our support article about the Intel RST driver.". — Cristian Ciupitu, Jan 10 '15 at 20:13
@CristianCiupitu: Though this is WD and not Seagate it could perhaps be the/an issue with drivers. Then of course the question becomes if Linux is up to date on this. The articles I found from Seagate was for Windoze (e.g. http://knowledge.seagate.com/articles/en_US/FAQ/218615en?language=en_US) — user129107, Jan 10 '15 at 20:21
I know you don't have RAID, but I found this gem on the Intel forum, "RAID array with 3TB disks": "You need RST OROM 10.5 or later to support physical disks >= 2TB. Check with your motherboard manufacturer for a BIOS update if your RST OROM is lower than 10.5. Also, it's not supported on controllers older than ICH9." — Cristian Ciupitu, Jan 10 '15 at 20:59
@CristianCiupitu: Oh, dang, that does not look to promising ... as i have ICH8. But as I have already connected it and run SMART tests there is no return for a 2TB. — user129107, Jan 10 '15 at 21:06
If the motherboard does not really supports your disk, you could install it in an external (USB) enclosure or buy an additional SATA controller. If it's a simple one it shouldn't be very expensive. — Cristian Ciupitu, Jan 10 '15 at 21:09
I'd like to know what kernel version you are running. Nevermind - I see 3.5 - local build. I wonder if you've enabled EFI GUID Partition Support? — mikeserv, Jan 10 '15 at 21:41
cross-site duplicate: Why would a 3TB disk show as being 746GiB — phuclv, Jun 17 '17 at 13:54

score 4 · Accepted Answer · edited Jun 11 '20 at 12:04

If you're using a Debian-Ubuntu based system (like Linux Mint), then fdisk (util-linux 2.20.1) does not understand GPT.

You should try parted -l and have you partitioned / formatted the drive yet? Using gparted (if you're using a GUI) would probably be easier.

I'm not sure about lsblk, it says it reads from sysfs. It's possible there's only one small partition on the drive, or a strange MBR/GPT.

But Cristian Ciupitu's comment and link from Seagate sound very promising, but it appears to be about Windows drivers...

Searching for "parted Error: /dev/sdc: unrecognised disk label" brought up this link that appears to do what you'd like: How to Create Partition on Linux for >2TB Size using Parted GPT. I've bold-ed (embiggened?) the error message in question, looks like it should go away after creating a GPT. (I'll bet using gparted would be similar/easier.)

Use parted’s mklabel command to set disk label to GPT as shown below.

# parted /dev/sdb
GNU Parted 2.1
Using /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print

Error: /dev/sdb: unrecognised disk label

(parted) mklabel gpt
(parted) print
Model: Unknown (unknown)
Disk /dev/sdb: 5909GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number  Start  End  Size  File system  Name  Flags

[Then it continues making a partition, but the above commands look the most relevant]

Create >2TB Partition using Parted mkpart

Use parted’s mkpart command as shown below to create partition that is greater than 2TB. In this example, we are creating a partition that is roughly of 6TB in size.

# parted /dev/sdb
(parted) mkpart primary 0GB 5909GB
(parted) print
Model: Unknown (unknown)
Disk /dev/sdb: 5909GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number  Start   End     Size    File system  Name     Flags
 1      1049kB  5909GB  5909GB               primary

If trying to create a GPT fails, there should be some good error messages to search. Or if it still only reports the size as 800GB, I'd begin to suspect there might be something wrong with the disk, but since SMART thinks it's ok, that leaves the linux drivers/kernel...

And you're not using some sort of external USB enclosure are you? It's connected directly to the computer, like an internal HD?

No, I have not done anything to the disk yet. It is "blank". I wanted to make sure it is OK before I started the work on populating all my installs and quirks on it ... :/ I tried parted -l earlier but it simply yielded "Error: /dev/sdc: unrecognised disk label" — user129107, Jan 10 '15 at 20:18
Found a few promising looking links from searching for "parted Error: /dev/sdc: unrecognised disk label"... I'll add some info from one — Xen2050, Jan 10 '15 at 20:28
Hmm. I thought one had to use GPT on > 2TB disks (>2.2). Do I read it correctly if one can mix it up? Old MBR / DOS on boot partition and some way get the rest by other means. I am going to split the disk up in system and storage. — user129107, Jan 10 '15 at 20:52
I think you could have GPT and also a MBR (for backwards compatibility, but may just cause more problems). MBR might be ok on big disks, but might not be able to make a single partition >2TB. — Xen2050, Jan 10 '15 at 21:10
@user129107 - one has to use GPT on those disks if one desires to boot from them. Otherwise you can partition the disk to MBR-recognizable chunks if you must, but GPT is a far better format anyway. I will second Cristian in a gdisk recommendation - it is a far more simple-to-use tool than is parted in my opinion. I once had the same problem as you - booting in UEFI-mode solved it for me, but probably that is not necessary - even if you should do it anyway. There is almost definitely no issue with the disk - just get gdisk and do gdisk /dev/sdc to have a look at the real disk. — mikeserv, Jan 10 '15 at 21:38
The problem isn't with fdisk or GPT; the kernel thinks the drive is only 800gb. — psusi, Jan 10 '15 at 23:43
Yes, HDD is connected directly to MOBO. I'll look more at this. Thanks or all help. — user129107, Jan 11 '15 at 07:52

score 2 · Answer 2 · edited Jan 11 '15 at 18:13

2

You seem to have built your own kernel, and it is only 32 bit. For it to have access to > 2 TB disks, you have to enable CONFIG_LBDAF (support for large (2TB+) block devices and files), though I could have sworn that without it the size just was clamped to 2 TB rather than wrapping around to the 32 bit modulus.

edited Jan 11 '15 at 18:13

Cristian Ciupitu

2,510

answered Jan 10 '15 at 23:45

psusi

17,303

Thanks. Promising, but I have CONFIG_LBDAF=y in config for kernel. – user129107 Jan 11 '15 at 06:56
@user129107, hrm.. then there's some other kind of bug in your kernel. Try using a stock distribution kernel instead of a custom built one. – psusi Jan 11 '15 at 18:20

user129107 · Answer 3 · 2015-01-11T07:53:24.480

1

Posting an "answer", which is more of an observation, as the question would become somewhat cluttered and long with this information there.

Did some more tests. One with simple C-code using ioctl() and lseek().

ioctl BLKGETSIZE64  : 801569726464 (Same value as the tools)
ioctl BLKGETSIZE    :   1565565872 (Same as sectors reported by fdisk)
lseek Size          : 801569726464

A quick look at source for lsblk, fdisk and lshw they all use ioctl() or sysfs to get the size data. (Take "they use" light here as I have only skimmed some online source.)

lsblk uses sectors, (/sys/block/sdc/size), and left shift by 9. (AKA × 512, or × 2^9)

Sectors: 1565565872                      (From printout in Q)
         1565565872 << 9 = 801569726464

Looking closer at the size reported and switching my brain to binary instead of decimal one also quickly recognize that:

3000592982016 = 00101011101010100001010001110110000000000000 (44-bits (48))
 801569726464 =     1011101010100001010001110110000000000000 (40-bits)

Further more:

Size by SMART - Size by tools (in effect kernel)
3000592982016 - 801569726464 = 2199023255552 == 2^32 * 512 or 2^32 << 9

fdisk also reports physical sector size as 512 opposite to SMART's 4096 which perhaps indicates some controller issue.

edited Jan 11 '15 at 07:53

answered Jan 11 '15 at 07:48

user129107

241

It isn't a controller issue. It's bit-depth. Just look at the actual partition table on the disk - use gdisk - and make sure you're working with a kernel configured to handle EFI disks - as I linked to in the other comment. If you want to see it actually work - just boot a recent live disc for a popular distribution and see for yourself. – mikeserv Jan 11 '15 at 08:00
I've had the same problem before - I even mentioned it here last summer. Your disk is fine. – mikeserv Jan 11 '15 at 08:05
You could have also used the blockdev command from util-linux, e.g. blockdev --getsz /dev/sda or blockdev --getsize64 /dev/sda. – Cristian Ciupitu Jan 11 '15 at 18:18
1

The disk is addressed in sectors, so the truncation isn't 40 bits vs. 44, but rather 36 bits vs. 32. – psusi Jan 11 '15 at 18:22

score 1 · Answer 4 · answered Jun 07 '18 at 17:58

For me, the issue was that I was using it (same drive model) with an external USB enclosure. Once I plugged it directly to a SATA connector, it started being recognized as 3TB. If you're having the same problem and using a USB interface for the drive, try connecting the drive directly to a SATA connection.

As @phuclv noted, there is a great explanation of this on Superuser at https://superuser.com/questions/1101839/why-would-a-3tb-disk-show-as-being-746gib

Why my 3TB disk shows an actual size of ~800 GB with several tools except smart tools?

S.M.A.R.T:

Sub question:

4 Answers4