Adept Information Technology û SysAdmin Tools

Home

Adept IT SysAdmin Tools

This is the beginning (and hopefully not the end) of tools I shall write to help us (myself and other SysAdmins) do the day to day system administration jobs, quicker and easier.

These Tools will be targetted at running on Solaris primarily, but will run on other systems as appropriate (The beauty of Java).

This starts with the "findUsedSpace" tool, but shall grow in time.

Find Used Space

I've found that on enough occasions, I've run into the problem of a system's disk being full (or surprisingly over-utilised), and I need to find out quickly whats chewing up all the space. As opposed to mucking around with a stack of du's, I've written a java program that does all the work for you, and finds out where the space above a threshold you specify, is being used. This threshold *can* be as granular as bytes right through to gigabytes. The software performs about as quickly as a du on Solaris (its slightly slower, but not much); but this software provides more information and can also be run on Windows, Linux, etc.

Examples

So you can see where this might come in handy, some examples are provided

Example 1: We wish to find all files/directories with 1 Gigabyte or more in them (this is the default, but we'll specify the params):

[root@vir /]$findUsedSpace -t 1 -g
findUsedSpace - Version 1.0-RC5
(C) Copyright 2006-2007 Geoff Ongley
http://sysadmin.AdeptIT.com.au

INFO: Running On Solaris, Excluding mounted Proc File Systems
INFO: Excluding /proc
INFO: Excluding /export/zones/ldapm1/root/proc
INFO: Excluding /export/zones/proxy/root/proc

Threshold: 1G
Searching from /
2.48G /usr
2.73G /

Example 2: We want to fine all files/directories with over 100 meg, but we also want to stop ourselves from hitting our NFS mounts (/net) and home dirs (/home).
We specify a threshold of 100 (-t 100), we specify we want things in Megabytes (-m, this includes our threshold) and we exclude /net (-x /net) and /home (-x /home)

[root@vir /]$findUsedSpace -t 100 -m -x /net -x /home
findUsedSpace - Version 1.0-RC5
(C) Copyright 2006-2007 Geoff Ongley
http://sysadmin.AdeptIT.com.au

INFO: Running On Solaris, Excluding mounted Proc File Systems
INFO: Excluding /proc
INFO: Excluding /export/zones/ldapm1/root/proc
INFO: Excluding /export/zones/proxy/root/proc

Threshold: 100M
Searching from /
214.72M /usr/share
441.67M /usr/lib
130.09M /usr/openwin/lib
190.0M /usr/openwin
199.64M /usr/sfw/lib
436.37M /usr/sfw
112.61M /usr/jdk/instances/jdk1.5.0/jre/lib
114.55M /usr/jdk/instances/jdk1.5.0/jre
159.66M /usr/jdk/instances/jdk1.5.0
159.66M /usr/jdk/instances
181.38M /usr/jdk
137.95M /usr/staroffice7/help
249.98M /usr/staroffice7/program
146.2M /usr/staroffice7/share
535.28M /usr/staroffice7
2536.24M /usr
2796.86M /

Example 3: We want to find all files/directories with over 5G in /home. We specify our root directory (-d /home) and our threshold of 5 (-t 5) and gigs (-g)

root@vir> findUsedSpace -d /home -t 5 -g
findUsedSpace - Version 1.0-RC5
(C) Copyright 2006-2007 Geoff Ongley
http://sysadmin.AdeptIT.com.au

INFO: Running On Solaris, Excluding mounted Proc File Systems
INFO: Excluding /proc
INFO: Excluding /export/zones/ldapm1/root/proc
INFO: Excluding /export/zones/proxy/root/proc

Threshold: 5G
Searching from /home

(we found nothing, we have good users, don't we! Well, no, this is just a virtual machine for testing...)

Usage:

java findUsedSpace [-d <dir>] [-t <threshold>] [-b] [-k] [-m] [-g] [-x <dir>] [-v]
-d <dir> - the root directory to start the search for used space in
-t <threshold> - the root directory to start the search for used space in
-b - the threshold specified is in bytes
-k - the threshold specified is in kbytes
-m - the threshold specified is in mbytes
-g - (default) the threshold is specified in gbytes
-x <dir> - exclude <dir> from search, this IS case sensitive, even on Windows
use multiple -x's for multiple excludes
-v - Version information

Defaults are: Search from this directory, exclude proc filesystem (UNIX/Linux/BSD), 1gigabyte threshold.

It calculates "real" gigs, megs, kbytes, etc, (ie 1k is 1024 bytes, 1m is 1024k, etc).

The output is significantly more accurate (to 2 decimal places) in RC2, and the code is significantly lighter (code was totally stripped back, cleared up and many bugs were fixed).

Download:

Below are some "Release Candidate 5" versions of the initial AdeptTools package, which contains sample batch and/or shell scripts.

AdeptTools-1.0-RC5.pkg.tar.gz - Solaris Package (Architecture independent, x86 and SPARC)
AdeptTools-1.0-RC5.zip - General BSD/Linux/Windows (or other) package

This is written on Java 1.6, but compiled for a JVM >= 1.2 for compatibility with Solaris 8, for those who can't be bothered or afford the space to put a new JVM on.

ChangeLog

RC5 (3/Sep/2007)

Long time since last release, meant to be fixing this one annoying bug for ages... finally got to it!

BugFix: On Solaris 10 with Zones, findUsedSpace would continue to loop in Zone PROC filesystems - fixed.
New Feature Request: On Solaris, a flag to skip autofs,ctfs,objfs,dev and other filesystems that aren't really relevant. This will go into RC6, which will probably be the final release of 1.0

RC4 (23/Jul/2006)

Added functionality to show not only directories, but files over threshold
Modified output layout and added method to handle this
modified version and startup output

RC3 (22/Jul/2006)

Fixed symlink detection algorithm for files (this was calculating directory links correctly, but not individual files)
Updated version information, added website
Cleaned up code further, removed unused routines

RC2 (21/Jul/2006)

Improved accuracy of output (output is now to 2 decimal places, instead of being rounded to the nearest whole number
Fixed bugs in checking for the proc file system
Significant clean up of code, removed duplication of work
Fixed Usage output, to display threshold information
Reduced internal Array sizes which were over-sized (directory arrays which were the size of directories and files in a certain directory, only required to be the size of the directories)
Fixed Accuracy on Solaris (as per above bug)
Reduced calculation code, for converting between bytes, kbytes, mbytes, gbytes
Fixed output error, when bytes selected, the software would output "bbytes"
Changed to Solaris Package and .ZIP offering only (UNIX/Linux guys can use unzip, no big deal).
Improved Sample shell and batch scripts
Improved Solaris Package

RC1 (21/Jul/2006)

Initial public release
Added exclusions (to exclude, for example, NFS directories)
Added symlink detection
Fixed bugs relating to drive letters on Windows sometimes being presented as lower case, sometimes as upper, from the getCanonicalPath() and getAbsolutePath() methods.
Added basic shell and batch scripts
Created Solaris Package, generic UNIX/Linux (.tar.gz) and Generic platform (.zip)
Fixed too many bugs to mention!

Any questions can be sent to geoff at adeptit.com.au.

Disclaimer: These tools are free for use; but are not to be reverse engineered, re-packaged, mirrored, sold, re-sold or anything else of the likes without the Author's written permission. Also, I don't guarantee they will work for you and so on.