0

Somehow, something is eating up around 80GB of HDD space and I cannot figure out what. This is starting to become a major problem for me as I only have 100GB to play with and if I find something to delete, something else will just fill that space, until I can barely use the computer.

I have tried running ncdu -x -q to clear up as much as possible and it says that I am down to about 22G usage. I get the same with du, but I guess ncdu uses du

du -x -d1 -h /
4,0K    /srv
28K     /media
4,0K    /mnt
4,0K    /lib64
16K     /lost+found
3,3G    /home
588M    /opt
360M    /lib
23M     /build
9,7M    /bin
48M     /boot
27M     /etc
1,4G    /root
4,0K    /cdrom
6,9G    /usr
16M     /sbin
9,0G    /var
22G     /

And if I run df:

 df
Filesystem      1K-blocks       Used Available Use% Mounted on
udev              2748080          4   2748076   1% /dev
tmpfs              559400       1568    557832   1% /run
/dev/sdc2       109481872  104180864         0 100% /
none                    4          0         4   0% /sys/fs/cgroup
none                 5120          0      5120   0% /run/lock
none              2796996        688   2796308   1% /run/shm
none               102400          8    102392   1% /run/user
overflow             1024        236       788  24% /tmp
/dev/sdd2      1953382140 1874376664  79005476  96% /media/munhunger/Shaman
/dev/sda1      1953382140 1926088316  27293824  99% /media/munhunger/Holy
/dev/sdb1      2930264060 2930264060         0 100% /media/munhunger/Glory

df -aTh
Filesystem     Type             Size  Used Avail Use% Mounted on
sysfs          sysfs               0     0     0    - /sys
proc           proc                0     0     0    - /proc
udev           devtmpfs         2,7G  4,0K  2,7G   1% /dev
devpts         devpts              0     0     0    - /dev/pts
tmpfs          tmpfs            547M  1,6M  545M   1% /run
/dev/sdc2      ext4             105G  103G     0 100% /
none           tmpfs            4,0K     0  4,0K   0% /sys/fs/cgroup
none           fusectl             0     0     0    - /sys/fs/fuse/connections
none           debugfs             0     0     0    - /sys/kernel/debug
none           securityfs          0     0     0    - /sys/kernel/security
none           tmpfs            5,0M     0  5,0M   0% /run/lock
none           tmpfs            2,7G  688K  2,7G   1% /run/shm
none           tmpfs            100M  8,0K  100M   1% /run/user
none           pstore              0     0     0    - /sys/fs/pstore
overflow       tmpfs            1,0M  236K  788K  24% /tmp
binfmt_misc    binfmt_misc         0     0     0    - /proc/sys/fs/binfmt_misc
systemd        cgroup              0     0     0    - /sys/fs/cgroup/systemd
gvfsd-fuse     fuse.gvfsd-fuse  0,0K  0,0K  0,0K    - /run/user/1000/gvfs
/dev/sdd2      fuseblk          1,9T  1,8T   76G  96% /media/munhunger/Shaman
/dev/sda1      fuseblk          1,9T  1,8T   27G  99% /media/munhunger/Holy
/dev/sdb1      fuseblk          2,8T  2,8T     0 100% /media/munhunger/Glory

Running du with sh flag(and x because I do not want to scan mounted disks)

du -xsh /
22G     /

What could be causing this, and what can I do to mitigate the problem? Running on a linux mint 17 from uname -a: 3.16.0-38-generic #52~14.04.1-Ubuntu

From running lsof | grep -Fw deleted:

lsof | grep -Fw deleted
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs
      Output information may be incomplete.
munHunger
  • 191
  • What OS are you using? – jesse_b Aug 03 '17 at 19:45
  • Opps, missed the most important part :D It is a linux mint 17 I believe. 3.16.0-38-generic #52~14.04.1-Ubuntu from running uname -a – munHunger Aug 03 '17 at 19:47
  • 2
    I guess this is a huge deleted file (possibly in /var/log) that is still open by a process which keeps writing into it. You would be able to identify it with lsof | grep -Fw deleted. – xhienne Aug 03 '17 at 20:08
  • Can you do df -aTh? – jesse_b Aug 03 '17 at 20:15
  • https://community.hpe.com/t5/System-Administration/difference-between-filesystem-and-mountpoint/m-p/5260291#M52653 . Not pretending I fully understand this but I would still like to see your du -sh / without the x argument. – jesse_b Aug 03 '17 at 20:26
  • Yeah, it has been running for the past 15 minutes and it seems to be taking awhile as it has to scan through /dev/sdd2, /dev/sda1 and /dev/sdb1 – munHunger Aug 03 '17 at 20:30
  • The file may not be deleted but hidden under a mount point. Look for very big files with a command like lsof -s | awk 'int($(NF-2)) > 100000000' – xhienne Aug 03 '17 at 20:33

2 Answers2

1

Very similar to Jesse_b's answer, what has worked well for me over the years is simply:

df -xk / | sort -n | tail -40

The -x option to df instructs it not to traverse other mount points. Using -k keeps all the sizes in 1024 byte blocks. The output gets sorted and I tend to like looking at the last 40 lines, but this is purely arbitrary. This should point you to a directory containing the files taking up the most space. If you want files in the output, change the df command to df -axk before piping to sort and tail.

Deathgrip
  • 2,566
0

You can try this to find where the large file(s) may be but beware this command may be very long running depending on the number of files in your system:

du -k / | sort -nr | head -n25

Here are my results:

[root@JBCLAMP001 ~]# du -x -d1 -h /
512     /boot
512     /system
81M     /var
2.9M    /root
1.0K    /opt
512     /srv
512     /mnt
512     /media
3.0K    /tmp
20M     /home
15M     /etc
5.0K    /native
981M    /usr
1.1G    /

[root@JBCLAMP001 ~]# du -sh /
du: cannot access ‘/proc/45795/task/45795/fd/3’: No such file or directory
du: cannot access ‘/proc/45795/fd/3’: No such file or directory
du: cannot access ‘/native/proc/45795/fd/3’: No such file or directory
du: cannot access ‘/native/proc/45795/path/3’: No such file or directory
8.7G    /
[root@JBCLAMP001 ~]# du -xsh /
1.1G    /

As you can see the du -x -d1 -h / does not find all files.

jesse_b
  • 37,005
  • well, sorting is out of the question... no space left on device(and I have a hard time finding anything to delete at this point) – munHunger Aug 03 '17 at 19:50
  • Although, isn't that pretty much the same as running du -x -d1 -h which did not help me find anything? – munHunger Aug 03 '17 at 19:52
  • No, I'm not exactly sure what du -x -d1 -h does but my command will list the 25 largest files in your root file system. – jesse_b Aug 03 '17 at 19:53
  • As far as I am aware du -x -d1 -h lists the disk usage of all folders in current directory(depth 1) and when I run it on root, there is nothing that takes up full disk storage. i.e. there is 80gb unaccounted for – munHunger Aug 03 '17 at 19:55
  • Also based on your original post it looks like you have 3+GB in /home and 9GB in /var. I'm sure some of those files can safely be removed. – jesse_b Aug 03 '17 at 19:55
  • That's because it's not searching recursively (like you said, just going 1 dir deep). So it wont find nested files. Unfortunately *nix directories don't represent the size of the files within them. – jesse_b Aug 03 '17 at 19:56
  • Sure, but it is avoiding the root problem. As you can see I am only using 22GB of the full 100GB disk, but it still says that I don't have any disk space available. And I have observed that everytime I delete something something else takes it's place that I cannot find – munHunger Aug 03 '17 at 19:57
  • False. The files it found (depth 1) are only using 22GB, but since it's only checking 1 dir deep it's not finding MOST of your files. – jesse_b Aug 03 '17 at 19:58
  • 1
    Try doing this and see what it finds: du -sh / – jesse_b Aug 03 '17 at 20:01
  • from man du -d, --max-depth=N print the total for a directory (or file, with --all) only if it is N or fewer levels below the command line argument; --max-depth=0 is the same as --summarize... it is only printing on first level, it is however searching further down. it says for example that my /home is 3,3 GB and it is only folders in that folder, i.e. it is searching recursively – munHunger Aug 03 '17 at 20:01
  • @munHunger I've made edits to my post. – jesse_b Aug 03 '17 at 20:03
  • Try running it as sudo? either way I updated mt questiong with the results of du -xsh – munHunger Aug 03 '17 at 20:08
  • take the x out. – jesse_b Aug 03 '17 at 20:09