Oracle Operations Monitor – Admin task – High Availability configuration

Oracle Operations Monitor can be implemented in High Availability, this is achieved setting up two Mediation Engines, working as Master/Slave.

One important aspect using high availability is keep the configuration of the master and slave servers synchronized. With the new REST API, you can utilize a simple bash script to download the configuration from the master and upload and restore it on the slave.

Two important notes using this implementation are:

  • Slave server will restart the EOM process to upload and restore the information loaded from the Master.
  • All configurations must be performed in the Master EOM, as any change in the Slave will be discarded after the upload and restore process.

To start the process, generate the API Key (and take note of the values) in both EOM servers (the one selected as Master and the Slave).

Using the top left menu -> Settings -> General Settings -> Rest API -> Enable Rest API and take note of the API Key generated:

The next step is create a file in the Master server (in this example /tmp/ is used) with the following content:

#!/usr/bin/bash
#should be changed to /bin/bash in EOM, or create soft link

##################################
# Copyright March 2017
# Lonny Clark, Oracle Consulting
# Sync HA EOM installation
# This script should be installed
# on 'master' EOM, can be run
# manually or with cron.
# Should run as root, or with
# write permissions in /tmp
#################################

PROGNAME=$(basename $0)
logger "${PROGNAME}: Start"

pushd /tmp

#set extra script messaging for debug
[ -n "$DEBUG2" ] && set -x

#these URLs for local primary device
API_KEY="X-Api-Key: admin;API_Key_of_Master_EOM"
URL="https://Server_IP_of_Master_EOM/me/"
SAVE_URL="${URL}saveApplianceConfiguration"
DOWNLOAD_URL="${URL}downloadApplianceConfiguration"
DELETE_URL="${URL}deleteApplianceConfiguration"

#these URLs for remote secondary (slave EOM)
API_KEY2="X-Api-Key: admin;API_Key_of_Slave_EOM"
URL2="https://Server_IP_of_Slave_EOM/me/"
RESTORE_URL="${URL2}restoreApplianceConfiguration"
UPLOAD_URL="${URL2}uploadApplianceConfiguration"
DELETE_URL2="${URL2}deleteApplianceConfiguration"

#define an error handler, accepts one string arg. FYI logger requires syslog server
function error_exit
{
  logger "${PROGNAME}: ${1:-"Unknown Error"}"
  echo "${PROGNAME}: ${1:-"Unknown Error"}" 1>&2
  #attempt to clean up
  clean_up
  popd
  logger "${PROGNAME}: Error Stop"
  exit 1
}

#define a cleaning function
function clean_up
{
  [ -n "$DEBUG" ] && logger "${PROGNAME}: Cleaning up"

  if [ -n "$MFILE" ]; then
    if [ -f "$MFILE" ] && rm $MFILE; then
      curl -k -X POST -H "${API_KEY}" -d "name=${MFILE}" ${DELETE_URL}
    fi
    if [ $? -ne 0 ]; then
      error_exit "$LINENO: Error return from POST, attempting to DELETE_URL:$?"
    fi
  fi

  if [ -n "$SFILE" ]; then
    if [ -f "$SFILE" ] && rm $SFILE; then
      curl -k -X POST -H "${API_KEY2}" -d "name=${SFILE}" ${DELETE_URL2}
    fi
    if [ $? -ne 0 ]; then
      error_exit "$LINENO: Error return from POST, attempting to DELETE_URL2:$?"
    fi
  fi
}

#Start sync process
#Step 1 save the config on master EOM

[ -n "$DEBUG2" ] && echo saving: $SAVE_URL
[ -n "$DEBUG" ] && logger "${PROGNAME}: saving: ${SAVE_URL}"

CMD="curl -k -X POST -H \"${API_KEY}\" ${SAVE_URL}"
[ -n "$DEBUG2" ] && echo cmd: $CMD
[ -n "$DEBUG" ] && logger "${PROGNAME}: cmd: ${CMD}"

ret=$(curl -k -X POST -H"${API_KEY}" ${SAVE_URL})
success=$(echo "$ret" | grep "success...true")

if [ ! -n "$success" ]; then
  error_exit "$LINENO: Error return from POST, attempting to SAVE_URL:$ret"
fi

[ -n "$DEBUG2" ] && echo return is: $ret
[ -n "$DEBUG" ] && logger "${PROGNAME}: return is: $ret"
#get the filename from returned string
MFILE=$(echo "$ret" |grep -i name |cut -d '"' -f4)

[ -n "$DEBUG2" ] && echo master file is $MFILE
[ -n "$DEBUG" ] && logger "${PROGNAME}: master file is {$MFILE}"

if [ ! -n "$MFILE" ]; then
  error_exit "$LINENO: Error, can't find filename in return from POST after SAVE_URL:$ret"
fi

[ -n "$DEBUG2" ] && echo download is $DOWNLOAD_URL
[ -n "$DEBUG" ] && logger "${PROGNAME}: download is ${DOWNLOAD_URL}"

#step 2: download the new saved config to cwd
curl -k -H "${API_KEY}" -d "name=${MFILE}" -o ${MFILE} ${DOWNLOAD_URL}
if [ $? -ne 0 ]; then
  error_exit "$LINENO: Error return from POST, attempting to DOWNLOAD_URL:$?"
fi

#we will not need to rename the file in HA, this is so I can test with a single ME
NEWFILE=sync${MFILE}

[ -n "$DEBUG2" ] && echo new filename is $NEWFILE
mv ${MFILE} ${NEWFILE}

#step 3: upload locally saved config to secondary EOM
ret=$(curl -k -X POST -H "${API_KEY2}" -F "configuration=@${NEWFILE}" ${UPLOAD_URL})
success=$(echo "$ret" | grep "success...true")
[ -n "$DEBUG2" ] && echo ret: $success
if [ ! -n "$success" ]; then
  error_exit "$LINENO: Error return from POST, attempting to UPLOAD_URL:$ret"
fi
SFILE=$(echo "$ret" |grep -i name |cut -d '"' -f4)
[ -n "$DEBUG2" ] && echo secondary file is $SFILE
[ -n "$DEBUG" ] && logger "${PROGNAME}: secondary file is ${SFILE}"

#step 4: restore new config on secondary device
ret=$(curl -k -X POST -H "${API_KEY2}" -d "configuration=${SFILE}" ${RESTORE_URL})
success=$(echo "$ret" | grep "success...true")
[ -n "$DEBUG2" ] && echo cmd: $success
[ -n "$DEBUG" ] && logger "${PROGNAME}: restore cmd: ${success}"
if [ ! -n "$success" ]; then
  error_exit "$LINENO: Error return from POST, attempting to RESTORE_URL:$ret"
fi

#step 5 clean up
clean_up

#return to original dir
popd

logger "${PROGNAME}: Stop"
exit 0

#####################################################

File was saved with the name EOM_Sync.txt and don’t forget to modify the permissions to be able to be executed.

The final step in this process is modify the crontab to execute the this script, you can use the command crontab -e, for this case, the script is executed every Sunday at 8:55 am UTC.

This can be verified checking the Active calls graph and Settings -> General Settings -> Status