Hosted by CU logo University of Colorado
Boulder
Powered by ESGF-CoG logo
Welcome, Guest. | Login | Create Account
CoG logo
You are at the CoG-CU node
 

Setup cron jobs to execute periodic administration tasks

Following is a list of CoG tasks that should be executed periodically, and are therefore best setup as cron jobs on the server hosting CoG:

1) Periodically update the projects around the federation

Each CoG needs to periodically query its federated peers to obtain their latest projects. An example script to accomplis this task is included in the CoG source code itself: resources/scripts/sync_projects.sh - which may need to be customized to your environment. For example, to synchronize the projects every hour:

0 * * * * /esg/config/sync_projects.sh

The script sets up some needed environment variables. then executes the "sync_project" management command that is included with the CoG distribution:

#!/bin/sh

# script to update the state of CoG projects around the federation

# setup environment
source /usr/local/cog/venv/bin/activate
export LD_LIBRARY_PATH=/opt/esgf/python/lib:$LD_LIBRARY_PATH

# reference the COG installation
export COG_INSTALL_DIR=/usr/local/cog/cog_install

# reference the COG configuration
export COG_CONFIG_DIR=/usr/local/cog/cog_config

python $COG_INSTALL_DIR/manage.py sync_projects

2) Backup the postgres databases content every night

It is highly recommended that both the CoG and the ESGCET databases be backed up daily. This can be accomplished by setting up a cron job to run nightly as follows:

crontab -l
0 0 * * * /esg/config/db_backups.sh

where the script has the following content:

#!/bin.sh
export PATH=/usr/local/pgsql/bin:$PATH
pg_dump -p 5432 -U dbsuper cogdb > /tmp/cogdb_$(date +%Y-%m-%d).sql
pg_dump -p 5432 -U dbsuper esgcet > /tmp/esgcet_$(date +%Y-%m-%d).sql

NOTE: for the script to run as cron, the postgres database password must be located in file ~/.pgpass with the format:

host_name:port:database_name:database_user:database_password

Also the file permissions must be set as follows:

chmod 0600 ~/.pgpass

3) Remove stale user stubs from local database

When a user from another CoG logs onto the local CoG with their OpenID, a user object stub is created in the local database to store some needed information. Unfortunately, if the user account is deleted on the remote server, the local account is not. To remedy this situation, i.e. to delete local user stubs of remote users that have been deleted, the ESGF node administraor can run the following command (in the proper python virtual environment):

cd $COG_iNSTALL_DIR
python manage.py sync_users
The same command can also be run as a cron job. An example such script is provided as part of the CoG distribution: resources/scripts/sync_users.sh, and may need to be customized to your ESGF installation. For example, to run the script every night at midnight:
0 0 * * * /esg/config/sync_users.sh

where the script content is:

#!/bin/sh

# example script to delete stale user stubs from the local database

# setup environment
source /usr/local/cog/venv/bin/activate
export LD_LIBRARY_PATH=/opt/esgf/python/lib:$LD_LIBRARY_PATH

# reference the COG installation
export COG_INSTALL_DIR=/usr/local/cog/cog_install

# reference the COG configuration
export COG_CONFIG_DIR=/usr/local/cog/cog_config

python $COG_INSTALL_DIR/manage.py sync_users
Last Update: Nov. 10, 2015, 6:55 a.m. by Hydra Administrator
Read News
CoG 3.8 Released!
This release improves support for retracted datasets. Learn more at: https://earthsyst...
CoG 3.7 Released!
This release focuses on improving the security of the application and the node federation functionality. ...
CoG 3.6 Released!
This release adds the ability to add non-CoG links to the left navigation bar. Read ...
CoG 3.5 Released!
This release upgraded the Globus libraries. Learn more at: https://earthsyst...
CoG 3.4 Released!
This release changes how the ESGF search functions. Learn more at: https://earthsyst...
Browse Projects
 
 
 
 
Start typing, or use the 'Delete' key to show all available tags.
CoG Tags: Cyberinstrastructure