######################################################################
HANGFG (Hang file generator) is a series of unix shell scripts used
to automate the generation and collection of hanganalyze and 
systemstate trace files. HANGFG generates and collects hang trace
files based on the impact of taking diagnostic traces on a system 
which is already in a degraded state. The overall decision on what 
level of impact the user can afford is left up to the user when he 
runs HANGFG, as the level of impact is passed in as an argument to 
the tool. HANGFG is also capable of making this decision for the user
if the user selects light impact (option 1) as an argument to the tool.
HANGFG is RAC aware and can run in either a RAC or non RAC environment.
######################################################################
INSTALLATION:

Once the tool has been downloaded to the directory you wish to install,
untar the hangfg.tar file. This will create a directory called hangfg.
The hangfg files are then untared into this new directory. Next, make 
sure to change the file permissions on these files to execute by using
the chmod command.

######################################################################
RUNNING HANGFG:

If running hangfg on a RAC cluster you must run hangfg as the oracle 
user. To run the HANGFG utility, execute the hangfg.sh shell script. 
This script takes 1 argument which controls the number and level of 
detail of the hang trace files generated. 

ARG1 = level of impact (1, 2 or 3).

1) Light impact on system. This option collects 2 hanganlyze level 3 
traces and then determines whether it can also collect 1 hanganalyze
level 4 trace with minimal impact to the system. If so, it collects
the hanganalyze level 4 trace. If not, it does not collect an 
additional trace file.

2) Medium impact on system (default value). This option collects 1
hanganlyze level 3 trace and then determines whether it can also 
collect 2 hanganalyze level 4 traces with minimal impact to the 
system. If so, it collects the 2 additional hanganalyze level 4 
traces. If not, it collects 1 additional hanganalyze level 3 trace.
This option also collects 1 systemstate level 258 trace.

3) Heavy impact on system. This option collects 2 hanganalyze level 4
traces and 2 systemstate level 258 traces.


If you do not enter any argument the script runs with a default value 
of 2 meaning collect trace files with a medium impact on the system.

Example 1:

./hangfg.sh 3

This would start the tool and collect traces with the heaviest impact
on the system ( 2 hanganalyze level 4 traces and 2 systemstate dumps).

Example 2:

./hangfg.sh

This would use the default values of 2 and collect traces with a 
medium impact on the system.

######################################################################
RAC ENVIRONMENTS:

HANGFG is rac aware. If the node you are running on is part of a RAC
cluster the hanganalyze and systemstate dump commands are issued with
the -g option. If the current node where you are running hangfg is 
hung you will be given a notification and asked to run hangfg from
another node in the cluster. HANGFG finds all files on all nodes in 
the cluster and lists these files along with nodename in the file
hangfiles.out. HANGFG bundles all the trace files from all the nodes
and puts them into one compressed tar file hfiles.tar.Z. 

######################################################################
OUTPUT FILES:

HANGFG creates hang trace files in the respective udump and bdump 
directories. In addition the following files are also generated in the
directory where you are running hangfg.

1) hangfiles.out. This file contains a list of all files generated 
during the hang. This is the list of files you need to send into 
support for analysis. 

2) hangfg.log.  This file contains a log of the hangfg program. This 
file is useful for debugging any issues they may arise along with
logging all actions associated with the hangfg program. 

3) hfiles.tar.Z  This file contains a compressed tarball of all hang 
traces collected by the hangfg program. It also contains the file 
hangfiles.out.
