SQWareProduction:mongodb

Presentation

Usage

It's the module locally operating of mongodb.
It is entirely written in ksh and sql.
It allows to manage the operation for mongodb.
Example uses of the tool:

• Backup
• Recovery
• Compute statistics
• Indicators collection to store in referential database SQWareRepository
• Rotate logs
• ....

This module is deployed locally on the machine hosting the authorities to manage, in order to avoid any points of weakness related to an NFS mount or whatever. The deployment is done through a simple command SQWareCentral module using inventory located in SQWareRepository !
Installation guide for SQWareProduction for mongodb

Explanation of directories

Directories located in $gvsqw_MngBin:

• bin : Directory containing standard scripts
• bin_cust : Directory containing customs scripts for your environnement
• etc : Directory containing the standard files of configuration
• etc_cust : Directory containing the customs files of configuration for your environnement (overload standards)
• help : Directory containing the standard files of help
• help_cust : Directory containing the customs files of help for your environnement
• lib : Directory containing the generic shell function librairies
• lib_cust : Directory containing the generic shell function librairies (overload standards)
• spm : Contain sql query toolkit
• tools : Directory containing the standard scripts used on an ad hoc
• tools_cust : Directory containing the customs scripts used on an ad hoc for your environnement
• menu : Directory containing the standard menus
• menu_cust : Directory containing the custom menus for your environnement

Naming convention of files

The majority of files are named with suffixes:

• *.ksh : for scripts
• *.cfg : for configuration files
• *.lib : for function librairies
• *.hlp : for help files

Naming convention in scripts

In general, the following naming convention is used in all scripts of SQWareProduction:

• gvsqw_ {} : global variable initialized by the environment and / or generic library
• lvsqw_ {} : local variable initialized by the script and / or a specific library
• gfsqw_ {} : function defined by a generic library
• lfsqw_ {} : function defined by the script and / or a specific library

bin

sqwmng_CompactAllDatabases.ksh

Script $gvsqw_MngBin/bin/sqwmng_CompactAllDatabases.ksh:

compact tables of databases in parallel See the detailed help for sqwmng_CompactAllDatabases.ksh

sqwmng_CompactDatabase.ksh

Script $gvsqw_MngBin/bin/sqwmng_CompactDatabase.ksh:

compact tables of databas See the detailed help for sqwmng_CompactDatabase.ksh

sqwmng_CopyDumpAllDatabases.ksh

Script $gvsqw_MngBin/bin/sqwmng_CopyDumpAllDatabases.ksh:

copy of dump all databases in parallel See the detailed help for sqwmng_CopyDumpAllDatabases.ksh

sqwmng_CopyDumpDatabase.ksh

Script $gvsqw_MngBin/bin/sqwmng_CopyDumpDatabase.ksh:

copy of dump file $lvsqw_CopyType See the detailed help for sqwmng_CopyDumpDatabase.ksh

sqwmng_DumpAllDatabases.ksh

Script $gvsqw_MngBin/bin/sqwmng_DumpAllDatabases.ksh:

Dump databases in parallel See the detailed help for sqwmng_DumpAllDatabases.ksh

sqwmng_DumpDatabase.ksh

Script $gvsqw_MngBin/bin/sqwmng_DumpDatabase.ksh:

mongodump of database See the detailed help for sqwmng_DumpDatabase.ksh

sqwmng_ExecSQL.ksh

Script $gvsqw_MngBin/bin/sqwmng_ExecSQL.ksh:

execute SQL file on instance See the detailed help for sqwmng_ExecSQL.ksh

sqwmng_FailWithoutMail.ksh

Script $gvsqw_MngBin/bin/sqwmng_FailWithoutMail.ksh:

exit with error and display imput parameters See the detailed help for sqwmng_FailWithoutMail.ksh

sqwmng_GatherIndicators.ksh

Script $gvsqw_MngBin/bin/sqwmng_GatherIndicators.ksh:

gather DBA indicators See the detailed help for sqwmng_GatherIndicators.ksh

sqwmng_NetBackup.ksh

Script $gvsqw_MngBin/bin/sqwmng_NetBackup.ksh:

backup with NetBackup dump/backup files See the detailed help for sqwmng_NetBackup.ksh

sqwmng_ParallelRun.ksh

Script $gvsqw_MngBin/bin/sqwmng_ParallelRun.ksh:

Manages the execution of unix commands in parallel mode
Takes in parameter file with unix command to execute
You can choose:

• The parallel degree
• Aggressiveness of run (number of seconds between two runs)
• Aggressiveness of check end (number of seconds between two checks of end)

See the detailed help for sqwmng_ParallelRun.ksh

sqwmng_RestoreAllDatabases.ksh

Script $gvsqw_MngBin/bin/sqwmng_RestoreAllDatabases.ksh:

restore all databases in parallel See the detailed help for sqwmng_RestoreAllDatabases.ksh

sqwmng_RestoreDatabase.ksh

Script $gvsqw_MngBin/bin/sqwmng_RestoreDatabase.ksh:

restore database See the detailed help for sqwmng_RestoreDatabase.ksh

sqwmng_RunJob.ksh

Script $gvsqw_MngBin/bin/sqwmng_RunJob.ksh:

Run MongoDB scripts and generate log and error file
based on $gvsqw_RootCfg/sqwmng_Jobs.cfg configuration file
compress and purge oldest log files See the detailed help for sqwmng_RunJob.ksh

etc

sqwmng_CreInst_CrontabRef.cfg

Configuration file $gvsqw_MngBin/../etc/sqwmng_CreInst_CrontabRef.cfg:


See the detailed help for sqwmng_CreInst_CrontabRef.cfg

sqwmng_CreInst_Jobs.cfg

Configuration file $gvsqw_MngBin/../etc/sqwmng_CreInst_Jobs.cfg:


See the detailed help for sqwmng_CreInst_Jobs.cfg

sqwmng_GlobalVar.cfg

Configuration file $gvsqw_MngBin/../etc/sqwmng_GlobalVar.cfg:
Global var for MongoDB
It contains the operating variables of SQWareProduction.

See the detailed help for sqwmng_GlobalVar.cfg

.profile

Sample file for .profile $gvsqw_MngBin/../etc/.profile:
Sample of lines to add to .profile of mongodb users.
See the detailed help for .profile

.profile_confort

File to put aliases $gvsqw_MngBin/../etc/.profile_confort:
Contains confort aliases of SQWareProduction for mongodb.

See the detailed help for .profile_confort

lib

sqwmng_CompactAllDatabases.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_CompactAllDatabases.lib:
lib for sqwmng_CompactAllDatabases.ksh

See the detailed help for sqwmng_CompactAllDatabases.lib

sqwmng_CompactDatabase.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_CompactDatabase.lib:
lib for sqwmng_CompactDatabase.ksh

See the detailed help for sqwmng_CompactDatabase.lib

sqwmng_CopyDumpAllDatabases.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_CopyDumpAllDatabases.lib:
lib for sqwmng_CopyDumpAllDatabases.ksh

See the detailed help for sqwmng_CopyDumpAllDatabases.lib

sqwmng_CopyDumpDatabase.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_CopyDumpDatabase.lib:
lib for sqwmng_CopyDumpDatabase.ksh

See the detailed help for sqwmng_CopyDumpDatabase.lib

sqwmng_DumpAllDatabases.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_DumpAllDatabases.lib:
lib for sqwmng_DumpAllDatabases.ksh

See the detailed help for sqwmng_DumpAllDatabases.lib

sqwmng_DumpDatabase.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_DumpDatabase.lib:
lib for sqwmng_DumpDatabase.ksh

See the detailed help for sqwmng_DumpDatabase.lib

sqwmng_GatherIndicators.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_GatherIndicators.lib:
lib for sqwmng_GatherIndicators.ksh

See the detailed help for sqwmng_GatherIndicators.lib

sqwmng_GatherIndicators_Overload.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_GatherIndicators_Overload.lib:
lib for overLoad sqwmng_GatherIndicators.ksh

See the detailed help for sqwmng_GatherIndicators_Overload.lib

sqwmng_GenerateCreateInstance.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_GenerateCreateInstance.lib:
lib for generate create instance

See the detailed help for sqwmng_GenerateCreateInstance.lib

sqwmng_Global.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_Global.lib:
central lib of generic fonctions for mongodb

See the detailed help for sqwmng_Global.lib

sqwmng_NetBackup.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_NetBackup.lib:
lib for sqwgen_NetBackup.ksh

See the detailed help for sqwmng_NetBackup.lib

sqwmng_RestoreAllDatabases.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_RestoreAllDatabases.lib:
lib for sqwmng_RestoreAllDatabases.ksh

See the detailed help for sqwmng_RestoreAllDatabases.lib

sqwmng_RestoreDatabase.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_RestoreDatabase.lib:
lib for sqwmng_RestoreDatabase.ksh

See the detailed help for sqwmng_RestoreDatabase.lib

sqwmng_SetEnv.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_SetEnv.lib:
fonction to set env for $MNG_SID

See the detailed help for sqwmng_SetEnv.lib

tools

sqwmng_Crypt.ksh

Script $gvsqw_MngBin/../tools/sqwmng_Crypt.ksh:
Generic script to crypt file with openssl See the detailed help for sqwmng_Crypt.ksh

sqwmng_GenerateCreateInstance.ksh

Script $gvsqw_MngBin/../tools/sqwmng_GenerateCreateInstance.ksh:
Generate create instance files See the detailed help for sqwmng_GenerateCreateInstance.ksh

sqwmng_ListAsyncFilesGlobal.ksh

Script $gvsqw_MngBin/../tools/sqwmng_ListAsyncFilesGlobal.ksh:
List Async files (sql, log, mail) to upload in SQWareCentral (all in a directory) See the detailed help for sqwmng_ListAsyncFilesGlobal.ksh

sqwmng_ListAsyncFiles.ksh

Script $gvsqw_MngBin/../tools/sqwmng_ListAsyncFiles.ksh:
List Async files (sql, log, mail) to upload in SQWareCentral See the detailed help for sqwmng_ListAsyncFiles.ksh

sqwmng_SetEnv.ksh

Script $gvsqw_MngBin/../tools/sqwmng_SetEnv.ksh:
set env for $MNG_SID

See the detailed help for sqwmng_SetEnv.ksh

sqwmng_TestInstConnection.ksh

Script $gvsqw_MngBin/../tools/sqwmng_TestInstConnection.ksh:
Test instance connection See the detailed help for sqwmng_TestInstConnection.ksh

sqwmng_TestMySQL.ksh

Script $gvsqw_MngBin/../tools/sqwmng_TestMySQL.ksh:
Generic test MySQL connection script See the detailed help for sqwmng_TestMySQL.ksh

sqwmng_TestSendmail.ksh

Script $gvsqw_MngBin/../tools/sqwmng_TestSendmail.ksh:
Generic test sendmail script See the detailed help for sqwmng_TestSendmail.ksh

sqwmng_Uncrypt.ksh

Script $gvsqw_MngBin/../tools/sqwmng_Uncrypt.ksh:
Generic script to uncrypt file with openssl See the detailed help for sqwmng_Uncrypt.ksh

help

Contains help files for "-s" option of bin scripts.

spm

SQL query toolkit
See the detailed help for spm