====== Filesystem Full ======
My filesystem / (slash) is full and when I compare df on / and size of all directories on / (using du) informations are not the same size. df give me a / used 100%, while I don't find same directory sum size.
[root@monitor]/ # df -m
Filesystem 1024-blocks Free %Used Iused %Iused Mounted on
/dev/hd4 327680 1640 100% 10612 24% /
...
/dev/auditlv 2097152 2096500 1% 8 1% /audit
[root@monitor]/ #
[root@monitor]/ # ls -l
total 96
drwxr-xr-x 4 root system 256 Mar 30 15:14 admin
drwxr-xr-x 3 root system 256 Apr 3 10:14 audit
lrwxrwxrwx 1 bin bin 8 Mar 30 15:11 bin -> /usr/bin
drwxrwxr-x 5 root system 4096 Apr 3 10:14 dev
drwxr-xr-x 16 root system 4096 Mar 30 15:20 esa
drwxr-xr-x 38 root system 8192 Mar 31 11:04 etc
drwxr-xr-x 7 bin bin 256 Mar 30 16:28 home
lrwxrwxrwx 1 bin bin 8 Mar 30 15:11 lib -> /usr/lib
drwx------ 2 root system 256 Mar 30 15:11 lost+found
drwxr-xr-x 173 bin bin 12288 Mar 30 15:38 lpp
drwxrwxr-x 28 root system 4096 Mar 25 09:36 mnt
drwxr-xr-x 15 root system 4096 Mar 30 15:30 opt
drwxr-xr-x 4 pconsole pconsole 256 Mar 30 15:26 pconsole
dr-xr-xr-x 1 root system 0 Apr 3 10:15 proc
drwxr-xr-x 5 root system 256 Mar 30 16:22 root
drwxr-xr-x 3 bin bin 256 Mar 31 11:01 sbin
drwxr-xr-x 2 root system 256 Mar 30 15:11 tftpboot
drwxrwxrwt 10 bin bin 4096 Apr 3 10:15 tmp
lrwxrwxrwx 1 bin bin 5 Mar 30 15:11 u -> /home
lrwxrwxrwx 1 root system 21 Mar 30 15:32 unix -> /usr/lib/boot/unix_64
drwxr-xr-x 48 bin bin 4096 Mar 30 16:08 usr
drwxr-xr-x 33 bin bin 4096 Mar 30 15:59 var
[root@monitor]/ # for i in admin dev esa etc lpp mnt pconsole root sbin tftpboot
do
du -sm $i
done
0.00 admin
0.02 dev
54.27 esa
19.86 etc
7.04 lpp
0.00 mnt
92.21 pconsole
0.76 root
10.07 sbin
0.00 tftpboot
When I stop all processes, and umount the filesystems, I check all directories except filesystems mount points (I add audit in my directories sum), and now I see /audit (the directory) with 130MB
[root@monitor]/ # for i in audit admin dev esa etc lpp mnt pconsole root sbin tftpboot
do
du -sm $i
done
130.00 audit
0.00 admin
0.02 dev
54.27 esa
19.86 etc
7.04 lpp
0.00 mnt
92.21 pconsole
0.76 root
10.07 sbin
0.00 tftpboot
[root@monitor]/ # ls -l /audit
total 120
-rw------- 1 root system 0 Apr 3 10:14 auditb
-rw-rw---- 1 root system 0 Apr 3 10:17 bin1
-rw-rw---- 1 root system 2961 Apr 3 10:20 bin2
drwxr-xr-x 2 root system 256 Apr 3 10:14 lost+found
-rw-r----- 1 root system 136314880 Apr 3 10:17 trail
[root@monitor]/root # du -sm /audit/*
0.00 /audit/auditb
0.01 /audit/bin1
0.00 /audit/bin2
130.00 /audit/trail
=== Conclusion: ===
The file in use by audit are located in the directory /audit, and not in the filesystem with the same name. This is an overmount !!! The audit process was started before the filesystem was mounted, and it hangs on the files inode located in the directory /audit.\\
Delete all files in the /audit directory after having stop the audit process, and mount the filesystem /audit. Then you can start audit process again.