Computation dashboard

Users with long running computation or several simultaneous computation often have a need to keep track to progress and share it with their colleagues. In this scenario SeedMe Ticker messages may be used to create a dashboard that displays computation progress. For example by periodically appending a collection with Ticker messages about last file created by the computation.

Step 1: Create a file job.sh with following content.

This could be used to launch a computation job which generates files at a location that we poll to fetch the latest.

#!/bin/bash

#PBS/SLURM SETUP

# Make sure dashboard.sh background process with infinite loop exits 
trap 'kill $(jobs -pr)' SIGINT SIGTERM EXIT

# Run the dashboard.sh script in background
################################################
# Note: dashboard.sh has an infinite loop and must be terminated carefully which we do above using trap call
# Review the setup requirements in dashboard.sh 
#./dashbaord.sh $PBS_JOBNAME : $PBS_JOBID" # reuse PBS environment variables as collection title
#./dashbaord.sh $SLURM_JOB_NAME : $SLURM_JOB_ID" # reuse SLURM environment variables as collection title
./dashboard.sh "Sample title" & 

# Run your application
################################################
echo "Computation app goes here"

# Leave sometime for processes to cleanly exit
# Idling more than poll interval will ascertain a final poll in dashboard.sh
sleep 15

 

Step 2: Create a file dashboard.sh with following content.

This script demonstrates monitoring of computation progress by periodically sending a message about latest directory or file created at a given path to a Collection at SeedMe.org.
NOTE: This script uses polling with an infinite loop, so make sure to set appropriate poll interval and termination criteria.
Requirements

  1. SeedMe Python module/client. Download it from https://www.seedme.org/downloads.
  2. APIKey file for your SeedMe account. Sign-in > Download your apikey file > Move it your home dir

#!/bin/bash
# Usage: ./dashboard.sh "Sample title"

################################################
# Change the following as needed
################################################
# Location of seedme.py and allow execute privileges % chmod u+x seedme.py
seedme='~/seedme-python-client-v1.2.0/seedme.py'
# Location of seedme APIKey file if not in home dir
apikey_path='' # leave empty to use default at ~/.seedme or ~/seedme.txt
privacy='' # group or public, leave empty for private
sharing='' # comma seperated emails to share with others


# Poll interval in seconds, keep this as high as possible
interval=15 # in seconds
# Path to look for new files/directories
search_path='' # leave empty to search in this script's dir

# Append trailing slash and */ to given search path
if [ "$search_path" ]
    then search_path="${search_path}/*/"
    else search_path="*/"
fi 

# Create message that will be sent periodically
#message="ls -dlrt $search_path | tail -1 | cut -c 28-" # get latest dir only
message="ls -lrt $search_path | tail -1 | cut -c 28-" # get latest file or dir

# Send update only if different from previous message
constant_update=0 # change to 1 to send updates constantly 

################################################
# No changes should be needed below here

# Build command to create a new collection
create_collection=''

# Check if apikey path is set
if [ -n "$apikey_path" ]
    then # use apikey from provided location
        create_collection=" -auth_path $apikey_path"  
fi

# Check if title was provided
if [ -n "$1" ]
    then # use title from arg 1
        create_collection="${create_collection} -title '$1'" 
    else # create new title
        title="Dashboard: "$(date '+%d/%m/%Y %H:%M:%S')
        create_collection="${create_collection} -title '$title'" 
fi


if [ -n "$privacy" ]
   then create_collection="${create_collection} -privacy '$privacy'"
fi

if [ -n "$sharing" ]
   then create_collection="${create_collection} -emails '$sharing'"
fi
   
# Full create command 
create_collection="$seedme ${create_collection}"

# Create a new collection at SeedMe.org and capture response
response=$(eval $create_collection)
echo $response

# Extract collection_id from response 
id=`echo $response | sed -e 's/^.*"collection_id":[ \t]*"\([^"]*\)".*$/\1/'`; 

# Extract status response 
status=`echo $response | sed -e 's/^.*"status":[ \t]*"\([^"]*\)".*$/\1/'`; 

# Place holder for last ticker
previous_ticker=''

#echo "ID="$id, "\tStatus="$status, "\tLast ticker dir="$previous_ticker

# Update if response status indicates success or partial
if [ "$status"  = 'success' ] || [ "$status" = 'partial' ]
    then 
        # Infinite polling loop
        while :
        do
            # Generate ticker text
            ticker=$(eval $message)

            # Send update message if ticker is not empty
            # AND (different from previous OR constant update is set)
            if [ -n "$ticker" ] && ([ "$ticker" != "$previous_ticker" ] \
                                    || [ "$constant_update" -eq 1 ])
                then # Update collection if a new dir is found
                    echo "Updating collection $id with ticker '$ticker'"
                    eval $seedme -update $id -ticker "'"$ticker"'"
                    previous_ticker=$ticker # update to current ticker
            fi

            echo "dashboard.sh sleeping for $interval seconds"
            sleep $interval
        done
    else
        echo "ERROR: Failed to create new collection";
        echo "Aborted dashboard polling script";
fi