scc-log(1)



NAME


	scc-log - create a SCC-snapshot and compare with previous snapshot

RELEASE


	scc	1.23.185

SYNOPSIS


	scc-log [ -c|--comment <remark> ]
		[ -e|--extra <mod> ]
	        [ -S|--selection <mod> ]
		[ -f|--fqdn ]
		[ -h|--help ]
		[ -l|--label <label> ]
		[ -m|--max_age <max> ]
		[ --pretrans ]
		[ --preinstall ]
		[ --postinstall ]
		[ --preremove ]
		[ -r|--reset ]
		[ -v|--virtual <host> ]

DESCRIPTION


	This program calls scc-collect to create a SCC-snapshot and compares
	it with a previous snapshot. All differences are added to a logbook.
	Finally, both the new snapshot and the logbook are also written in
	HTML-format. Refer to scc(4) for a detailed description of the format
	of the snapshots and logbooks.

	scc-collect runs system and user modules. Each module collects
	specific configuration data. You can extend the "reach" of scc by
	writing and scheduling your own user modules. Refer to scc-collect(1)
	for a detailed description of modules and how to schedule and maintain
	them.

	The data in the snapshot can be split into several, partial snapshots. 
	This is done before comparing it with a previous snapshot. The mechanism
	can be used to provide Oracle DBA's with a snapshot for each Oracle 
	instance. It can also be used to separate each package in a ServiceGuard
	cluster. The partial snapshots of these packages can be compared on the
	scc-srv. This avoids many changes in the logbooks of a cluster, when a
	package "switches".
	Refer to the file /etc/opt/scc/newconfig/scc-split.conf for more details.

	The contents of the data generated by the modules are tested to 
	check their format. Lines with unknown formats are prefixed with the
	classification: "fix:messages::". The philosophy of scc is to collect
	data, not to interpret it. scc signals when it is unable to collect
	configuration data. Any interpretation of the correctness of the
	data can be done by the program scc-rules of scc-srv.

	In rare occasions, scc-collect "hangs". Use the following command
	to find the child processes of scc:
		UNIX95= ps -Hef
	This will show you the entire process-tree. On Solaris, you can use
	the ptree command from the proctools. Find the "deepest" descendant
	from scc-collect and kill this process. Then issue the command again
	to monitor the progress of scc-collect.

	On abnormal termination of scc-collect, the new snapshot probably is
	incomplete. When the cause of the problem is removed, a new run of
	scc can be started. This leads to large differences with the new,
	complete snapshot. To avoid this, use the -r option of scc-log to
	remove the current snapshot before making a new one.

	When the abnormal termination of scc-collect was not detected, the
	next run of scc will add many differences to the logbook. To reduce
	the size of the logbook, the log file can be edited by hand and the
	differences can be deleted.

LIMITATIONS


	As the rpm-database is locked during the install of scc, the snapshot 
	made during the install does not contain any rpm-data. To avoid massive
	changes in the logbook during the pre-install run and the run after the
	install, the rpm data is taken from a "keep" file. Each run of scc that
	is not performed during install, records all rpm data not only in the
	snapshot, but also in the keep file. This mechanism does not work for
	the initial install of scc as the keep file does not yet exist. In that case
	all rpm changes are ignored and this is recorded with a specific remark
	in the logbook.

	This mechanism is also used for packages on BSD systems.

	On AIX systems, the installed rpm packages are also listed as AIX
	installp packages. To avoid changes in the logbook for each install of
	scc, the above mechanism is extended to AIX packages when rpm packages
	have been installed.

ARGUMENTS


	None.

OPTIONS


	Note that the --pretrans, --preinstall, --postinstall and --preremove options 
	are used by package management scripts. Please use with care!

	-c|--comment <remark>   Add a remark to the logbook, indicating a "specific" 
	                        reason to run scc. For instance to mark the successful
	                        configuration/change of a system. An empty <remark> means
	                        no comment.
	-e|--extra <mod>        scc-collect runs all system modules and matching <mod> user modules.
	                        Where <mod> can contain shell filename wildcards. When used in 
	                        combination with the -v option, make sure that each 
	                        invocation activates the same set of user modules.
	-S|--selection <mod>    Run selected (system or user) module(s) for virtual host (-v).
	                        Where <mod> can contain shell filename wildcards.
	                        To detect and process layout version changes, use the -r option
	                        on the next invocation or include module general in the selection.
	-f|--fqdn               Use the full qualified name, not the short
	                        hostname, when sending files to the SCC server.
	                        On the SCC client, all filenames contain the 
	-h|--help               Display the syntax and exit.
	-l|--label <label>      Label to indicate the function of the system. Use <label>
	                        in the webinterface of scc-srv to select/group systems.
	                        An empty string means: no label.
	-m|--max_age <max>      Entries in the log-file, that are older than <max>
	                        months, are deleted
	--pretrans            	Check whether an already installed instance of scc is active.
	--preinstall            Run before an install, sets SCC_INSTALL_PHASE.
	--postinstall           Run new version after install, implies -r option and sets
	                        upgrade comment and SCC_INSTALL_PHASE.
	--preremove             Remove data directory prior to removal of the software.
	-r|--reset              Remove the current snapshot to avoid comparing it with
	                        the new (incompatible/incomplete) snapshot.
	-v|--virtual <host>     Do not use the hostname of the system, as base for all
	                        scc-files, use <host> instead. This option should be
	                        used to run specific modules via -e and/or -S option.

DIAGNOSTICS


	This program writes the following messages to stderr:

	Syntax error, use: scc-log [ -c|--comment <remark> ]
	        [ -e|--extra <mod> ] [ -S|--selection <mod> ] [ -f|--fqdn ]
	        [ -h|--help ] [ -l|--label <label> ] [ -m|--max_age <max> ]
	        [ --pretrans | --preinstall | --postinstall | --preremove ]
	        [ -r|--reset ]
	        [ -v|--virtual <host> ]
	A syntax error has been detected and reported.

	scc-log: Syntax error, missing argument for option <option>
	The indicated option is missing argument(s).

	scc-log: another instance is active, check process ID in <lock>
	This program compares a new snapshot with a previous one. When more
	than one instance of this program is active, they use the same files
	to store their new version of the snapshot. Therefore only one 
	instance of scc-log should be active.
	In rare occasions, a system program, called by scc-collect, might hang.
	When that is the case, try to kill the child processes of scc-collect.
	The lock file contains the PID of scc-log. When the lock-file exists and
	the process with the PID does not exist, the lock file is removed and 
	scc-log continues to run. Remove the lock file only when you are
	certain that scc-collect is not running.

	scc-log: non-numeric argument for -m option: <max>
	Use only numeric arguments for -m option.

	scc-log: cannot use -S option without -v option
	Limiting system modules only makes sense for a virtual host run.

	scc-log: use -e and/or -S option with -v option
	The -v option can only be used when the -e and/or -S option is used.

	scc-log: insufficient permissions to write in data directory: <data_dir>
	To run this program, log in as the owner of the directory.

	scc-log: no class actions found for host: <host>
	The split file specifies host <host>, but did not find any data for it.

	scc-log: scc-collect failed.
	Check the temporary files of scc-collect to determine the cause of the failure or
	set environment variable SCC_DEBUG and call scc-collect with the -i option.

EXTERNAL INFLUENCES


	During the pre- and post-install, the existence of the lock-file
	scc.lock is checked to find out whether scc is running.

	When diff is implemented via busybox, the logbook will be left empty
	and all compares should be done on the scc-srv.

	Use environment variable SCC_DATA to specify an alternative directory
	for the SCC data files. Should be an absolute path.

RETURN VALUE


	Upon completion, the program returns one of the following values:

		0 successful completion
		1 Syntax error
		2 Runtime error

COPYRIGHT


	scc-log is free software under the terms of the GNU General Public 
	License. Copyright (C) 2001-2004 Open Challenge B.V.,
	2004-2005 OpenEyeT Professional Services, 2005-2015 QNH.

FILES


	/tmp - directory for temporary files
		scc.lock - lock file to indicate that SCC runs
	/var/opt/scc/data - directory for data files
		scc.<hostname>.cur - current SCC-snapshot
		scc.<hostname>.html - current SCC-snapshot in HTML-format
		scc.<hostname>.old - previous SCC-snapshot
		scc.<hostname>.new - new (temporary) SCC-snapshot
		scc.<hostname>.log - logbook for changes in SCC-snapshots
		scc.<hostname>.log.html - logbook in HTML-format
		scc.<hostname>.keep - data kept from a previous run
		cfg.log - symbolic link to scc.<hostname>.log
			This symbolic link is used to detect whether the
			name of the system has changed. When this is the
			case, all scc.<hostname>.* files are renamed.
		index.html - compatibility for snapshot on scc-srv

	/etc/opt/scc/conf/scc-split.conf - specification to split the
		data of the snapshot into several files of pseudo hosts.
		This file can be used to provide each Oracle instance with
		its own snapshot. The file is installed in the subdirectory
		newconfig to avoid that your changes are erased by an upgrade.
		This file is ignored when the -v option is used.

	/etc/opt/scc/conf/scc_local_* - specification for module scc_0640_s_local

SEE ALSO


	scc(1), scc-cmp(1), scc-collect(1), scc-log(1), scc-log2html(1),
	scc-plugin(1), scc-snap2html(1), scc(4), scc(5)

VERSION


	$Revision: 5917 $

)	;;
	echo "${ProgName}: alternative SCC_DATA (${SCC_DATA}) should be an absolute path" >&2
xit 2;;
ac

Perform the security settings before calling any program.
TH=/sbin:/usr/sbin:/usr/bin:/bin:${SCC_BIN};	export PATH

port TMPDIR=${SCC_TMP}
port TMP=${SCC_TMP}

port SHELL=/bin/sh

Avoid too strict permissions. Otherwise scc-srv will not work correctly when pulling client data.
ask 022
! -d ${SCC_DATA} ] && mkdir -p ${SCC_DATA} 2>/dev/null
! -d ${SCC_DATA}/transfer ] && mkdir -p ${SCC_DATA}/transfer 2>/dev/null
ask 077
! -d ${SCC_TMP} ] && mkdir -p ${SCC_TMP} 2>/dev/null

[ ! -w ${SCC_DATA} ]
en
cho "${ProgName}: insufficient permissions to write in data directory: ${SCC_DATA}" >&2
xit 2


${SCC_TMP}

-r ${SCC_CONF}/scc-localize ] && . ${SCC_CONF}/scc-localize

${SCC_BIN}/scc_modules/scc_utils

PuppyLinux does not provide nice.
exe="$(which nice 2>/dev/null)"
[ ! -x "${n_exe}" ]
en
lias nice=""


stname_exe="$(which hostname 2>/dev/null)"
[ -x "${hostname_exe}" ]
en
ame=$(hostname 2>/dev/null)
se
ame=${HOSTNAME:-}

me=${name%%.*}
[ -z "${name}" ]
en
ame="empty"			# same value as in scc and scc-collect


dex=${SCC_DATA}/index.html
ck_cfg=${SCC_TMP}/scc.lock
o_log=${SCC_DATA}/cfg.log

lit_file=${SCC_CONF}/scc-split.conf

ndom="$(get_RANDOM)"
port TMP1_FILE=${SCC_TMP}/scc_log1_$$_${random}
port TMP2_FILE=${SCC_TMP}/scc_log2_$$_${random}
port TMP3_FILE=${SCC_TMP}/scc_log3_$$_${random}

port PROC_FILE=${SCC_TMP}/scc_log_ps_$$_${random}

Produce head for html-file on stdout.
Argument: title
c_html_head( )
{
cho '<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">'
cho "<HTML lang=\"en\">"
cho "<HEAD>"
cho '	<LINK HREF="style.css" REL="stylesheet" TYPE="text/css">'
cho "	<TITLE>Index for ${1}</TITLE>"
cho "</HEAD>"
}

Check the correctness of the snapshot format:
c_check_snapshot()
{
wk '/^fix:messages:/	{
			# When a module signals an error, the syntax of the message-line is correct
			visited[ "fix:messages::" ] = 1;
			compare[ ":messages:" ] = 1;
	}
^fix:|^var:|^hlp:|^stats:/	{
		class=$0;
		sub( /::.*/, "::", class );
		cnt=split( class, part, ":" );

		if ( visited[ class ] )			# Each visited class already has been checked.
		{
			print $0;			# print, but do not check again.
			next;
		}

		# line with classification with minimal number of elements:
		#	fix:general::			# cnt is 4
		# line with classification with maximal number of elements:
		#	fix:1:2:3:4:5:6:7:8:9:10::	# cnt is 13
		if ( cnt < 4 )
		{
			print "fix:messages::not enough fields in classification:", $0;
		}
		else if ( cnt > 13 )
		{
			print "fix:messages::too many fields in classification:", $0;
		}
		else if ( ( class == $0 ) && ( class !~ "::$" ) )
		{
			print "fix:messages::no end of classification:", $0;
		}
		else
		{
			visited[ class ]=1;

			# A class conflict occurs in the following case:
			# fix:a:b::data
			# fix:a:b:c::data
			# Either a:b is the classification of configuration data,
			# or it is the entry-point for a sub-menu.
			# Help-info can be available for a sub-menu.
			# Therefore no class conflicts for help-info.
			if ( class !~ "^hlp:" )
			{
				sub( /^.../, "", class );	# erase fix or var or hlp
				sub( /::$/, ":", class );	# map :a:: to :a: to check for :a:b:
				for ( c in compare )
				{
					if ( c == class )
					{
						break;
					}
					if ( index( c, class ) == 1 )
					{
						print "fix:messages::class conflict for \"" class "\" and \"" c "\"";
						break;
					}
					if ( index( class, c ) == 1 )
					{
						print "fix:messages::class conflict for \"" c "\" and \"" class "\"";
						break;
					}
				}
				compare[ class ]=1;
			}
			print $0;
		}
		next;
	}
	{
		print "fix:messages::unknown prefix:" $0;
	}'
eturn 0;
}

Check for the conditions in a split-file.
ARGUMENTS
<condition>	"absent" or "present"
<arg>		possible formats are:
		<file>
		<user>|<process>
eck_split_condition()
{
f [ "${2}" != "${2#*|}" ]
hen
# This is a very simple process checker.
user="${2%|*}"		# Split $2 into <user> and <proc>
proc="${2#*|}"
awk	'{
		if ( $1 != u )
		{
			next;
		}
		if ( $0 ~ p )
		{
			exit 2;
		}
	}' u="${user}" p="${proc}" ${PROC_FILE}
ret=$?
if [ "${1}" = "present" -a "${ret}" -eq 2 ]
then
	return 1;
fi
if [ "${1}" = "absent" -a "${ret}" -ne 2 ]
then
	return 1;
fi
lse
if [ "${1}" = "present" -a -f "${2}" ]
then
	return 1;
fi
if [ "${1}" = "absent" -a ! -f "${2}" ]
then
	return 1;
fi
i

eturn 0
}

nerate_class_counts()
{

Input is regular snapshot data.


Count lines starting with "fix" or "var".


Count the number and percentage of lines with the first and the second classification.

wk -F:	'/^fix/		{ fix_cnt+=1; }
^var/		{ var_cnt+=1; }
^stats/	{ next }		# stats-info is excluded from statistics.
^hlp/		{ next }		# hlp-info is excluded from statistics.
		{
			c=sprintf( "%s:%s", $2, $3 );
			main_cnt[ $2 ] += 1;
			sub_cnt[ c ] += 1;
		}
ND	{
	total = fix_cnt + var_cnt;

	# Display the counters for fix, var and total number of lines.
	printf( "stats:fix_var::fix:%d:%d\n", fix_cnt, 0.5 + ( 100 * fix_cnt ) / total );
	printf( "stats:fix_var::var:%d:%d\n", var_cnt, 0.5 + ( 100 * var_cnt ) / total );
	printf( "stats:fix_var::total:%d:%d\n", total, 100 );

	for ( s in sub_cnt )
	{
		split( s, part, ":" );
		main_counter = main_cnt[ part[ 1 ] ];

		# Use a separate, temporary file for the statistics as the process generating the
		# statistics requires first the "fix" and then the "var" statistics.

		# Use broad, fixed width numbers to enable usage of busybox sort to order the data.
		printf( "%010.10d:%010.10d:%s:%d:%d\n",
			main_counter, sub_cnt[ s ], s,
			0.5 + ( 100 * main_counter ) / total,
			0.5 + ( 100 * sub_cnt[ s ]) / main_counter ) >>tmp;
	}
}' tmp=${TMP3_FILE}

ort -r ${TMP3_FILE}					|
wk -F:	'{
	# Input: <main_class_cnt>:<sub_class_cnt>:<main_class>:<sub_class>:<main_perc>:<sub_class>
	printf( "stats:classes::%s:%s:%d:%d:%d:%d\n", $3, $4, $1, $5, $2, $6 ); 
}'

m -f ${TMP3_FILE}

Format of the output:

fix/var::<total_line_cnt>:100:<nr_fix/var_lines>:<fix/var_percentage>
<main_line_cnt>:<sub_line_cnt>:<main_class>:<sub_class>:<main_perc>:<sub_perc>
}

D_LINE="${ProgName}	[ -c|--comment <remark> ]
	[ -e|--extra <mod> ]
	[ -S|--selection <mod> ]
	[ -f|--fqdn ]
	[ -h|--help ]
	[ -l|--label <label> ]
	[ -m|--max_age <max> ]
	[ --pretrans | --preinstall | --postinstall | --preremove ]
	[ -r|--reset ]
	[ -v|--virtual <host> ]"
NTAX_ERROR="Syntax error, use: ${CMD_LINE}"

SSING_OPT_ARG="${ProgName}: Syntax error, missing argument for option:"
mark=""
x_months=0			# No limit
er_modules=""
lect_modules=""
start=0
dn=0
rtual_host=0
bel=""
ile [ $# -gt 0 ]

ase ${1} in
c|--comment)	if [ "${2:-}" ]			# Empty remark means: no comment
	then
		remark=$(echo "${2}" | tr -d "\012")	# be sure that the remark is single-line
	fi
	shift 1		# make sure that scc-log with a trailing -c option, without argument, finishes.
	shift 1;;

e|--extra)	# Empty is permitted, missing not
	[ -z "${2:-}" ] && [ "${2+defined}" != "defined" ] && echo "${MISSING_OPT_ARG}: ${1}" >&2 && exit 1
	user_modules="${2:-}"
	shift 2;;

S|--selection)	# Empty is permitted, missing not
	[ -z "${2:-}" ] && [ "${2+defined}" != "defined" ] && echo "${MISSING_OPT_ARG}: ${1}" >&2 && exit 1
	select_modules="${2:-}"
	shift 2;;

f|--fqdn)	fqdn="1"
	shift 1;;

h|--help)	echo "${CMD_LINE}"
	exit 0;;

l|--label)	if [ "${2:-}" ]			# Empty label means: no label
	then
		label="${2}"
	fi
	shift 1		# make sure that scc-log with a trailing -l option, without argument, finishes.
	shift 1;;

m|--max_age)	[ -z "${2:-}" ] && echo "${MISSING_OPT_ARG}: ${1}" >&2 && exit 1
	case "${2}" in
	[0-9]*)	max_months=${2};;
	*)	echo "${ProgName}: non-numeric argument for -m option: ${2}" >&2
		exit 1;;
	esac
	shift 2;;

-pretrans)	if [ -f ${lock_cfg} ]
	then
		echo "ERROR:   install aborted, another instance of scc is active."
		exit 5
	fi
	exit 0;;

-preinstall)	SCC_INSTALL_PHASE=preinstall

	echo "NOTE:    Running scc to create a snapshot with the current"
	echo "NOTE:    version of the software. This may take some minutes."

	if [ -f ${lock_cfg} ]
	then
		echo "ERROR:   install aborted, another instance of scc is active."
		exit 2
	fi

	shift 1;;

-postinstall)	SCC_INSTALL_PHASE=postinstall
	restart=1
	remark="install of SCC release 1.23.185"

	# The correct situation is that the lock file does not exist (no preinstall hook)
	# or that the lock file is empty because of the prein(stall) script.
	if [ -s ${lock_cfg} ]
	then
		echo "ERROR:   install aborted, another instance of scc is active."
		exit 2
	fi

	echo "NOTE:    Running scc to create a snapshot with the new version"
	echo "NOTE:    of the software. This may take some minutes."

	shift 1;;

-preremove)	rm -rf ${SCC_DATA} ${lock_cfg}
	exit 0;;			# We are done, no more arguments checking

r|--reset)	restart=1
	shift 1;;

v|--virtual)	[ -z "${2:-}" ] && echo "${MISSING_OPT_ARG}: ${1}" >&2 && exit 1
	virtual_host=1
	name="${2}"		# Act as data is originating from virtual host.
	shift 2;;

)		echo "${SYNTAX_ERROR}" >&2
	exit 1;;
sac
ne

[ ${virtual_host} -eq 0 -a "${select_modules}" ]
en
cho "${ProgName}: cannot use -S option without -v option" >&2
xit 2


[ "${SCC_INSTALL_PHASE:-}" ]
en

During installs on HP-UX, stm often hangs; do not run it.

CC_IGNORE_STM="yes";			export SCC_IGNORE_STM
se
CC_INSTALL_PHASE=""


To avoid that scc is run (by cron) between the pre- and the post-install,
an empty lock-file is created in the pre-install after running scc.
[ -f ${lock_cfg} -a ! -s ${lock_cfg} -a -n "${SCC_INSTALL_PHASE:-}" ]
en
m -f ${lock_cfg}	# make sure that scc-log will continue


Some commands might "hang", use a lock-file to avoid that more and more
invocations of Scc start and swamp the system.
[ -f ${lock_cfg} ]
en
unning=1
id=$(<${lock_cfg})
f [ -n "${pid}" ]
hen
# Check in a non-destructive way whether the process is still running.
kill -0 ${pid} 2>/dev/null
if [ $? -ne 0 ]
then
	# Unable to signal the process: it is no longer active.
	# Maybe hard-killed during a shutdown.
	running=0
fi
i
f [ ${running} -eq 1 ]
hen
echo "${ProgName}: another instance is active, check process ID in ${lock_cfg}" >&2
exit 2
i

m -f ${lock_cfg}


Do not install the traps earlier, as the above exit will remove the
lock file from the other, running invocation of this program.
ap 'rm -f ${TMP1_FILE} ${TMP2_FILE} ${TMP3_FILE} ${lock_cfg} ${PROC_FILE} 2>/dev/null' 0
ap "exit 2" 1 2 3 15

Record our PID in the lock-file
ho "$$" > ${lock_cfg}
[ ${virtual_host} -eq 1 -a -z "${select_modules}" -a -z "${user_modules}" ]
en
cho "${ProgName}: use -e and/or -S option with -v option" >&2
xit 1


Do not overwrite an existing stylesheet.
[ ! -f ${SCC_DATA}/style.css ]
en
p -p ${SCC_NEWCONF}/style.css ${SCC_DATA}
hmod 400 ${SCC_DATA}/style.css


g_cfg=${SCC_DATA}/scc.${name}.log
ep=${SCC_DATA}/scc.${name}.keep
[ "${label}" ]
en

Add the label to the keep file and the general module will pick it up

and preserve it for next runs. Class should be identical with code in general module.
cho "fix:general::label:${label}" >>${keep}


lit_hosts=""
[ -s "${split_file}" -a ${virtual_host} -eq 0 ]
en
plit_hosts="$(awk -F"|" '/^host\|/ { print $2 }' ${split_file} | sort -u)"


[ ${virtual_host} -eq 0 ]
en

For some programs that inspect the logbook, "fixed" filenames are easier.


Use a symbolic link to save disk-space.

f [ -h ${ito_log} ]
hen
# Check whether the target of the symlink corresponds to ${log_cfg}.
# When this is not the case, the hostname of the system changed.
prev_host="$(ls -l ${ito_log}	|
	awk '{ sub( /\.log$/, "", $NF ); sub( /.*scc\./, "", $NF ); print $NF }')"
if [ "${prev_host}" != "${name}" ]
then
	# We have to move the files to reflect the new hostname.
	for suffix in cur html log log.html keep old
	do
		mv ${SCC_DATA}/scc.${prev_host}.${suffix} ${SCC_DATA}/scc.${name}.${suffix} 2>/dev/null
	done

	# Rename also the files in data/transfer
	for file in ${SCC_DATA}/transfer/scc.${prev_host}.*
	do
		if [ -f "${file}" ]
		then
			new_name="$(echo ${file} | sed -e "s|scc.${prev_host}|scc.${name}|")"
			mv -f "${file}" "${new_name}"
		fi
	done

	if [ -z "${remark}" ]		# Do not change remark supplied with -c option.
	then
		remark="Hostname changed from ${prev_host} to ${name}"
	fi
fi
i
m -f ${ito_log}
ouch ${log_cfg}
n -s ${log_cfg} ${ito_log}


Reduce the log file to the specified number of months.
[ ${max_months} -gt 0 ]
en

Remove leading 0 from current month to avoid illegal octal numbers 08 and 09. Indicated by Eduardo Alvarenga.

ur_month=$(date '+%m')
ur_month=${cur_month#0}	# avoid 'octal' arithmetic
ear="$(date '+%Y')"
imit=$(( ( year * 12 ) + cur_month - max_months ))

or h in ${name} ${split_hosts}
o
echo "${SCC_DATA}/scc.${h}.log"
one				|
ort -u				|
hile read logfile
o
first=$(head -n 1 ${logfile} | awk -F- '{ print ( ( $1 * 12 ) + $2 ) }' )
if [ -n "${first}" -a ${first} -lt ${limit} ]
then
	# Some of the log-entries are too old.
	awk -F- '{
			if ( ( ( $1 * 12 ) + $2 ) >= l )
			{
				print;
			}
	}' l="${limit}" ${logfile} >${TMP1_FILE}

	# The html-version of the log file is generated from scratch out of the shorter log file.
	mv ${TMP1_FILE} ${logfile}
fi
one
	# if [ ${max_months} -gt 0 ]

Record the start of collecting.
n=$(date '+%Y-%m-%d')
n_b=$(date '+%H.%M.%S')
${t_n_b#*.*.}		# seconds
${s#0}		# strip leading "0"
${t_n_b%.*.*}		# hours
${h#0}		# strip leading "0"
${t_n_b#*.}		# minutes and seconds
${m%.*}		# minutes
${m#0}		# strip leading "0"
ck_start=$(( s + ( 60 * m ) + ( 3600 * h ) ))

When scc is installed via installp, it runs with a non-default ODM environment. 
Correct this (we only read data from ODM).
[ -d /etc/objrepos ]
en
xport ODMDIR=/etc/objrepos


v="$(which env 2>/dev/null)"
[ -x "${env}" ]
en

Run scc-collect in a "clean" environment.


It needs TZ to use the correct time-settings.


It needs ODMDIR to use the correct object database for AIX.


It uses SCC_PROFILING to determine whether to add performance data to the snapshot.


It uses SCC_IGNORE_STM to determine whether to run stm or not.


It uses SCC_INSTALL_PHASE to determine whether SCC is called during installation.


It uses SCC_INSTALL_METHOD to know whether source installs should be recorded.


It uses SCC_DATA to know whether a user indicated an alternative data hierarchy

ice env -i						\
TZ="${TZ:-}"					\
ODMDIR="${ODMDIR:-}"				\
SCC_PROFILING="${SCC_PROFILING:-yes}"		\
SCC_IGNORE_STM="${SCC_IGNORE_STM:-}"		\
SCC_INSTALL_PHASE="${SCC_INSTALL_PHASE:-}"	\
SCC_INSTALL_METHOD="${SCC_INSTALL_METHOD:-}"	\
SCC_DATA="${SCC_DATA}"				\
SCC_KEEP_CONFIG="${SCC_DATA}/scc.${name}.keep"	\
LOGNAME="root"					\
	${SCC_BIN}/scc-collect -e "${user_modules}" -S "${select_modules}" </dev/null >${TMP3_FILE} 2>&1 
se

Some systems do not provide env. Do not run in a clean environment.

xport SCC_KEEP_CONFIG="${SCC_DATA}/scc.${name}.keep"
ice ${SCC_BIN}/scc-collect -e "${user_modules}" -S "${select_modules}" </dev/null >${TMP3_FILE} 2>&1 


[ $? -ne 0 ]
en
cho "${ProgName}: scc-collect failed." >&2
rap "" 0
xit 2


Record the end of collecting.
n_e=$(date '+%H.%M.%S')

${t_n_e#*.*.}		# seconds
${s#0}		# strip leading "0"
${t_n_e%.*.*}		# hours
${h#0}		# strip leading "0"
${t_n_e#*.}		# minutes and seconds
${m%.*}		# minutes
${m#0}		# strip leading "0"
ck_end=$(( s + ( 60 * m ) + ( 3600 * h ) ))
[ ${tick_end} -lt ${tick_start} ]
en
ick_end=$(( tick_end + ( 24 * 3600 ) ))

ntime="$(( tick_end - tick_start ))"

[ ${restart} -eq 0 -a -f ${SCC_DATA}/scc.${name}.cur ]
en

Check for other layout-version in old (current) snapshot without -r option being used to indicate this.

rev_version="$(awk -F: '/fix:general::layout-version:/ { print $5; exit 0 }' ${SCC_DATA}/scc.${name}.cur)"
f [ "${prev_version}" != "1.23.185" ]
hen
restart=1
remark="Version change detected, from '${prev_version}' to '1.23.185'"
i


[ ${restart} -eq 1 ]
en

Erase the current snapshot(s) to indicate the restart.

m -f ${SCC_DATA}/scc.${name}.cur
f [ ${virtual_host} -eq 0 ]
hen
for h in ${split_hosts}
do
	rm -f ${SCC_DATA}/scc.${h}.cur
done
i


Check whether some classifications should be changed (via scc-localize):
Could be used to switch from fix: to var: or move some data to a separate class in the snapshot.
[ "${SCC_CLASS_MAP:-}" ]
en
cho "${SCC_CLASS_MAP}"		|
ed	-e '/^#/d'	\
-e '/^ *$/d'		|
hile read mapping
o
# mapping contains: <src>|<dest>

src="${mapping%|*}"
if [ "${src}" = "${mapping}" ]
then
	echo "fix:messages::missing class separator '|' in class mapping '${mapping}'" >>${TMP3_FILE}
	continue
fi

dest="${mapping#*|}"
if [ "${src}|${dest}" != "${mapping}" ]
then
	echo "fix:messages::more than one class separator '|' in class mapping '${mapping}'" >>${TMP3_FILE}
	continue
fi

sed -e "s|${mapping}|" <${TMP3_FILE} >${TMP3_FILE}.tmp 2>${TMP3_FILE}.err
if [ -s ${TMP3_FILE}.err ]
then
	echo "fix:messages::regular expression error in class mapping '${mapping}'" >>${TMP3_FILE}
	sed -e 's/^/fix:messages::/' <${TMP3_FILE}.err >>${TMP3_FILE}
else
	mv -f ${TMP3_FILE}.tmp ${TMP3_FILE}
fi
rm -f ${TMP3_FILE}.err ${TMP3_FILE}.tmp
one


Check the syntax of the new snapshot.
-f ${TMP1_FILE}
d	-e 's/[ 	][ 	]*$//'	\
e '/^$/d' ${TMP3_FILE}				|
c_check_snapshot >${TMP2_FILE}

-f ${TMP1_FILE} ${TMP3_FILE}

Do not use the split-file when -v option is used.
[ -s ${split_file} -a ${virtual_host} -eq 0 ]
en

The file ${PROC_FILE} only exists during the collection of data.


Extract its contents from the new snapshot using the same classification

used in scc-collect.
ed -n -e "s/^var:system:processes:://p" <${TMP2_FILE} >${PROC_FILE}

Split the new snapshot according to the split-file.


Produce the new snapshots for all pseudo hosts in the splitfile.

${TMP2_FILE} contains the collected data.
or host in ${split_hosts}
o
rm -f ${SCC_DATA}/scc.${host}.new

# Check whether pseudo host is active/present.
check="$(sed -n -e "s/^host|${host}|present|//p" ${split_file})"
if [ -n "${check}" ]
then
	check_split_condition present "${check}"
	if [ $? -eq 0 ]
	then
		continue
	fi
fi
check="$(sed -n -e "s/^host|${host}|absent|//p" ${split_file})"
if [ -n "${check}" ]
then
	check_split_condition absent "${check}"
	if [ $? -eq 0 ]
	then
		continue
	fi
fi

# Collect all the classifications that have to be moved/copied.
cp_file=${SCC_TMP}/${host}.cp.$$_${random}
del_file=${SCC_TMP}/${host}.del.$$_${random}
rm -f ${cp_file} ${del_file}

awk -F"|"	'/^[ 	]*#/	{ next }
		/^[ 	]*$/	{ next }
				{
					if ( $2 != h )
					{
						next;
					}
					data = "";
					n_class = "";
					del = 0;
					new = 0;
				}
		/^class\|.*\|rm\|/	{
					data = $4;
					del = 1;
				}
		/^class\|.*\|cp\|/	{
					data = $4;
					n_class = $4;
					if ( NF == 5 )
					{
						n_class = $5;
					}
					new = 1;
				}
		/^class\|.*\|mv\|/	{
					data = $4;
					n_class = $4;
					if ( NF == 5 )
					{
						n_class = $5;
					}
					del = 1;
					new = 1;
				}
				{
					if ( new == 0 && del == 0 )
					{
						next;
					}

					if ( length( n_class ) == 0 )
					{
						n_class = ":"
					}

					if ( data ~ "^fix:" || data ~ "^var:" )
					{
						# Do not extend.
						if ( new )
						{
							# n_class should also start with "fix:" or "var:"
							printf( "s%s^%s%s%s%sp\n",
								sep, data, sep,
								n_class, sep ) >>cp_file
						}
						if ( del )
						{
							print data >>del_file;
						}
					}
					else
					{
						# Extend optional with fix, var and hlp.
						col = ":"
						if ( data ~ "^:" )
						{
							col = "";
						}
						if ( new )
						{
							printf( "s%s^fix%s%s%sfix%s%s%sp\n",
								sep, col, data, sep,
								col, n_class, sep ) >>cp_file
							printf( "s%s^var%s%s%svar%s%s%sp\n",
								sep, col, data, sep,
								col, n_class, sep ) >>cp_file
							printf( "s%s^hlp%s%s%shlp%s%s%sp\n",
								sep, col, data, sep,
								col, n_class, sep ) >>cp_file
						}

						if ( del )
						{
							print "fix" col data >>del_file;
							print "var" col data >>del_file;
							print "hlp" col data >>del_file;
						}
					}
					next;
				}'					\
					cp_file="${cp_file}"		\
					del_file="${del_file}"		\
					h="${host}"			\
					sep='\001'			\
						${split_file}

if [ ! -s ${cp_file} -a ! -s ${del_file} ]
then
	echo "${ProgName}: no class actions found for host: ${host}" >&2
fi

if [ -s ${cp_file} ]		# Something to copy?
then
	sed -n -f ${cp_file} ${TMP2_FILE}		|
	scc_check_snapshot >${TMP1_FILE}

	{
		# Some messages are variable to avoid frequent changes. However, the presence of messages is fix!
		grep -l "^[fv][ia][xr]:messages::" ${TMP1_FILE} >/dev/null 2>/dev/null
		if [ $? -eq 0 ]
		then
			echo "fix:messages::inspect scc.${host}.cur to determine cause of messages in system/user-modules"
		fi
		echo "var:general::date:${d_n}"
		echo "var:general::start time:${t_n_b}"
		echo "var:general::stop time :${t_n_e}"
		echo "var:general::runtime:${runtime}"
		echo "fix:general::hostname:${host}"
		echo "var:general::remark:${remark}"
		wc -c <${TMP1_FILE} | awk '{ printf( "var:general::size (MB):%.3f\n", $1  / ( 1024 * 1024) ); }'
		if [ ${restart} -eq 1 ]
		then
			echo "var:general::restart:true"
		fi

		# Get the specification for this host from the split-file.
		# Put them in the general part of the snapshot.
		awk -F"|"	'/^[ 	]*#/	{ next }
						{
							if ( $2 != h )
							{
								next;
							}
							print "fix:general::split-file:" $0;
						}' h="${host}" ${split_file}

		cat ${TMP1_FILE}
		rm -f ${TMP1_FILE}
	} >${SCC_DATA}/scc.${host}.new
fi
if [ -s ${del_file} ]		# Something to remove?
then
	grep -v -f ${del_file} ${TMP2_FILE} >${TMP2_FILE}.tmp
	mv ${TMP2_FILE}.tmp ${TMP2_FILE}
fi

rm -f ${cp_file} ${del_file}

one	# while read host
	# if [ -s ${split_file} ]

The data of the pseudo hosts has been split off from the entire scc-data of the host.
Now we can generate the new snapshot for the local host.
{

Some messages are variable to avoid frequent changes. However, the presence of messages is fix!

rep -l "^[fv][ia][xr]:messages::" ${TMP2_FILE} >/dev/null 2>/dev/null
f [ $? -eq 0 ]
hen
echo "fix:messages::inspect scc.${name}.cur to determine cause of messages in system/user-modules"
i
cho "var:general::date:${d_n}"
cho "var:general::start time:${t_n_b}"
cho "var:general::stop time :${t_n_e}"
cho "var:general::runtime:${runtime}"
cho "var:general::remark:${remark}"
c -c <${TMP2_FILE} | awk '{ printf( "var:general::size (MB):%.3f\n", $1  / ( 1024 * 1024) ); }'
f [ ${restart} -eq 1 ]
hen
echo "var:general::restart:true"
i
at ${TMP2_FILE}
m -f ${TMP2_FILE}
>${SCC_DATA}/scc.${name}.new

When diff is implemented by BusyBox, we cannot parse the differences for the logbook.
Changing the order of two lines and deleting a line only signals the deleted line, not the swap of the lines.
In that case, all compares should be done on the server (which should provide a non-busybox diff).
port REMOTE_COMPARE=""
sybox_check diff
[ $? -ne 0 ]
en
xport REMOTE_COMPARE=yes
se

In case the distro does not provide diff:

iff_exe="$(which diff 2>/dev/null)"
f [ ! -x "${diff_exe}" ]
hen
export REMOTE_COMPARE=yes
i


Compare with previous snapshot and generate html-files.
{
cho "${name} local"
f [ -s ${split_file} -a ${virtual_host} -eq 0 ]
hen
awk -F"|" '/^host\|/ { if ( $3 == "local" || $3 == "remote" ) print $2, $3 }' ${split_file}
i
			|
rt -u				|
ile read host compare

d ${SCC_TMP}

f [ "${REMOTE_COMPARE}" ]
hen
compare=remote
i

ew_snapshot=${SCC_DATA}/scc.${host}.new
rev_snapshot=${SCC_DATA}/scc.${host}.cur
ld_snapshot=${SCC_DATA}/scc.${host}.old
ogfile=${SCC_DATA}/scc.${host}.log

f [ ! -f ${SCC_DATA}/scc.${host}.new ]
hen
# No data split off.
if [ "${compare}" = "local" ]
then
	# No new data available, do nothing. Previous data will be sent to server.
	:
else
	# Remove all files, to avoid that anything is sent to the server.
	rm -f	${SCC_DATA}/scc.${host}.cur	\
		${SCC_DATA}/scc.${host}.log	\
		${SCC_DATA}/scc.${host}.html	\
		${SCC_DATA}/scc.${host}.log.html
fi
continue
i

f [ "${compare}" = "local" ]
hen
# During the post-install run of each install, rpm and pkg_info cannot be used and
# the snapshot does not contain any rpm- and pkg_info data. During runs after the
# installs, rpm- and pkg_info data is collected and copied to the keep-file. During 
# post-install runs of upgrades, the data (of the previous run) is copied from the
# keep file. This strategy does not work immediately after the initial install as
# the keep-file is empty in this case. Ignore corresponding log-messages, but signal
# this in the remark in the logfile (when the -c option was not used).
log_remark="${remark}"

# To avoid some superfluous changes, we ignore some data.
# We use a sed file to remove data from the snapshot prior to comparing.
# Paul te Vaanholt indicated problems with the previous construction.
>${TMP1_FILE} 

# Make sure that the pkgmngt module did run in the previous run.
cnt=0
if [ -f "${prev_snapshot}" ]
then
	cnt="$(sed -n -e "/:scc_...._s_pkgmngt:start of module/,/:scc_...._s_pkgmngt:end of module/p" "${prev_snapshot}" | wc -l)"
fi
if [ "${cnt}" -ne 2 ]
then
	# During install, the RPM-database is locked and cannot return
	# the installed rpm's. After the installation, the new snapshot
	# contains rpm-data, but the old snapshot does not contain
	# rpm-data. Avoid messages in the log file, by ignoring all
	# rpm-data when the old (current) snapshot does not contain
	# any rpm-data. 
	# Ignore this when rpm is implemented by busybox.
	busybox_check rpm
	busybox_rpm=$?
	rpm_exe="/bin/rpm"
	if [ -x /usr/bin/rpm ]
	then
		rpm_exe="/usr/bin/rpm"
	fi
	if [ ${busybox_rpm} -eq 0 -a -x ${rpm_exe} -a -f "${prev_snapshot}" -a -z "${SCC_INSTALL_PHASE:-}" ]
	then
		# No busybox rpm, there is a previous snapshot and we are NOT in the installation process.
		if [ "$(${rpm_exe} -qa 2>/dev/null | wc -l)" -ne 0 ]
		then
			grep -l "^fix:software:installed-rpms:" "${prev_snapshot}" >/dev/null 2>/dev/null
			if [ $? -ne 0 ]
			then
				# No rpm-data in current (previous) snapshot. So this is the first run after the initial install.
				echo "/^new::software:installed-rpms:/d" >>${TMP1_FILE}

				[ -z "${log_remark}" ] && log_remark="rpm changes are ignored on the first run after the initial install"
			fi

			# AIX lists all rpms also as native installp packages. Ignore these too.
			if [ "$(uname -s)" = "AIX" ]
			then
				grep -l "^fix:software:installed-aix-filesets:" "${prev_snapshot}" >/dev/null 2>/dev/null
				if [ $? -ne 0 ]
				then
					echo "/^new::software:installed-aix-filesets:/d" >>${TMP1_FILE}
				fi
			fi

			grep -l "^fix:software:rpm:imported keys:" "${prev_snapshot}" >/dev/null 2>/dev/null
			if [ $? -ne 0 ]
			then
				# RPM is being used, but imported keys have to be checked separate.
				grep -l "^fix:software:rpm:imported keys:" "${new_snapshot}" >/dev/null 2>/dev/null
				if [ $? -eq 0 ]
				then
					# No rpm-data in current (previous) snapshot and data present in new snapshot.
					echo "/^new::software:rpm:imported keys:/d" >>${TMP1_FILE}

					[ -z "${log_remark}" ] && log_remark="rpm imported keys changes are ignored on the first run after the initial install"
				fi
			fi
		fi
	fi

	# During install by yum from a repository, yum is locked. After the installation
	# the new snapshot contains yum repolist data, but the old snapshot does not.
	# Avoid entries in the logbook by ignoring all yum data when the old (current)
	# snapshot does not contain any yum data.
	if [ -x /usr/bin/yum -a ! -x /usr/sbin/rhn-channel ]
	then
		if [ -f "${prev_snapshot}" ]
		then
			grep -l "^fix:software:yum:repolist::" "${prev_snapshot}" >/dev/null 2>/dev/null
			if [ $? -ne 0 ]
			then
				# No yum-data in current (previous) snapshot. So this is the first run after the initial install.
				echo "/^new::software:yum:repolist:/d" >>${TMP1_FILE}

				[ -z "${log_remark}" ] && log_remark="yum changes are ignored on the first run after the initial install"
			fi
		fi
	fi

	# During install, the BSD package database is locked and cannot return
	# the installed packages. After the installation, the new snapshot
	# contains package-data, but the old snapshot does not contain
	# package-data. Avoid messages in the log file, by ignoring all
	# package-data when the old (current) snapshot does not contain
	# any package-data. 
	if [ -x /usr/sbin/pkg_info -o -x /usr/pkg/sbin/pkg_info ]
	then
		if [ -f "${prev_snapshot}" ]
		then
			grep -l "^fix:software:installed-BSD-packages:" "${prev_snapshot}" >/dev/null 2>/dev/null
			if [ $? -ne 0 ]
			then
				# No package-data in current (previous) snapshot. So this is the first run after the initial install.
				echo "/^new::software:installed-BSD-packages:/d" >>${TMP1_FILE}

				[ -z "${log_remark}" ] && log_remark="pkginfo changes are ignored on the first run after the initial install"
			fi
		fi
	fi
fi

# Moving data from the main-snapshot through the split-file should not
# add to the differences when the split-file is created or extended.
# Only appplicable for the "main"-snapshot.
if [ "${host}" = "${name}" -a -f ${split_file} -a -f "${prev_snapshot}" ]
then
	# Retrieve the classifications of data that is moved.
	# Adjust them to delete-statements in a sed-file.
	# Replace all '/' by '\/' to avoid syntax errors.
	sed	-n					\
		-e 's/|$//'				\
		-e 's@^class|.*|mv|\(.*\)|.*@\1@p'	\
		-e 's@^class|.*|mv|\(.*\)@\1@p'		\
			<${split_file}				|
	sed	-e 's@/@\/@g'	\
		-e 's@^@/@'	\
		-e 's@$@/d@'	\
			>>${TMP1_FILE}
fi
ignore="-f ${TMP1_FILE}"

${SCC_BIN}/scc-cmp "${prev_snapshot}" "${new_snapshot}" "${log_remark}" "${ignore}" >>${logfile}
rm -f ${TMP1_FILE}
lse	# if [ "${compare}" = "local" ]
# Just empty the logfile to indicate to the server a remote compare.
>${logfile};
i
f [ -f ${prev_snapshot} ]
hen
rm -f ${old_snapshot}
mv -f ${prev_snapshot} ${old_snapshot}
i
v -f ${new_snapshot} ${prev_snapshot}

f [ -s ${logfile} ]
hen
nice ${SCC_BIN}/scc-log2html ${host} <${logfile} >${logfile}.html
lse
> ${logfile}.html
i

enerate_class_counts <${prev_snapshot} >${TMP1_FILE}
at ${TMP1_FILE} >> ${prev_snapshot}
m -f ${TMP1_FILE}

The file with the previous snapshot has been replaced by the new (current) snapshot.

ice ${SCC_BIN}/scc-snap2html ${host} <${prev_snapshot} >${SCC_DATA}/scc.${host}.html


cd ${SCC_DATA}
if [ "${host}" = "${name}" -a "${SCC_HOST_ALIAS:-}" ]
then
	for suffix in cur html log log.html
	do
		rm -f scc.${SCC_HOST_ALIAS}.${suffix}
		ln -s scc.${host}.${suffix} scc.${SCC_HOST_ALIAS}.${suffix}
	done
	scc_pack ${SCC_HOST_ALIAS}
	rm -f scc.${SCC_HOST_ALIAS}.* ${SCC_DATA}/transfer/scc.${host}.*
elif [ "${host}" != "${name}" -o -${fqdn} -eq 0 ]
then
	scc_pack ${host}
else
	domain="$(sed -n -e 's/^fix:general::domain://p' scc.${host}.cur)"
	if [ -z "${domain}" ]
	then
		scc_pack ${host}
	else
		for suffix in cur html log log.html
		do
			rm -f scc.${host}.${domain}.${suffix}
			ln -s scc.${host}.${suffix} scc.${host}.${domain}.${suffix}
		done
		scc_pack ${host}.${domain}
		rm -f scc.${host}.${domain}.*
	fi
fi

# Upgrade after scc has been used with the -u option resets the ownership of transfer to root.
# Check the transfer (signal) file whether the -u option has been used and correct.
# Refer to --user option handling in scc.
owner="$(ls -ld transfer/scc*signal | awk '{ print $3; exit 0 }')"
if [ "${owner}" != "root" ]
then
	chmod o+x .. . transfer
	chmod o+r transfer
	chown "${owner}" transfer 2>/dev/null
fi


ne	# while read host compare

[ ${virtual_host} -eq 0 ]
en

To ease navigation on the server, the html-file of the snapshot contains a reference

to index.html (the main file for a realm). To avoid an 404 error when browsing on the client,
we create a simple index.html on the client.
cc_html_head "Index for ${name}" >${index}
at <<-_X_ >>${index}
<BODY>
	<A HREF=scc.${name}.html>snapshot of ${name}</A>
	<BR>
	<A HREF=scc.${name}.log.html>logbook of ${name}</A>
</BODY>
</HTML>
X_


[ "${SCC_INSTALL_PHASE}" = "preinstall" ]
en

Avoid that scc is started between the pre- and the post-install.


We create an empty lock-file for scc-log indicating to

the postinstall that the install is running.
${lock_cfg}


[ "${SCC_INSTALL_PHASE}" = "postinstall" ]
en
cho "NOTE:    The snapshot and logbook can be found in ${SCC_DATA}"


it 0