SQWareProduction:mongodb

From Wiki_dbSQWare
Jump to: navigation, search

Presentation

Usage

It's the module locally operating of mongodb.
It is entirely written in ksh and sql.
It allows to manage the operation for mongodb.
Example uses of the tool:

  • Backup
  • Recovery
  • Compute statistics
  • Indicators collection to store in referential database SQWareRepository
  • Rotate logs
  • ....

This module is deployed locally on the machine hosting the authorities to manage, in order to avoid any points of weakness related to an NFS mount or whatever. The deployment is done through a simple command SQWareCentral module using inventory located in SQWareRepository !
Installation guide for SQWareProduction for mongodb

Explanation of directories

Directories located in $gvsqw_MngBin:

  • bin : Directory containing standard scripts
  • bin_cust : Directory containing customs scripts for your environnement
  • etc : Directory containing the standard files of configuration
  • etc_cust : Directory containing the customs files of configuration for your environnement (overload standards)
  • help : Directory containing the standard files of help
  • help_cust : Directory containing the customs files of help for your environnement
  • lib : Directory containing the generic shell function librairies
  • lib_cust : Directory containing the generic shell function librairies (overload standards)
  • spm : Contain sql query toolkit
  • tools : Directory containing the standard scripts used on an ad hoc
  • tools_cust : Directory containing the customs scripts used on an ad hoc for your environnement
  • menu : Directory containing the standard menus
  • menu_cust : Directory containing the custom menus for your environnement
Nuvola apps important.png
Warning:
Do not change the standard scripts and configuration files of dbSQWare.

Use directories *_cust for customizations. This will facilitate version upgrades.


Naming convention of files

The majority of files are named with suffixes:

  • *.ksh : for scripts
  • *.cfg : for configuration files
  • *.lib : for function librairies
  • *.hlp : for help files
Nuvola apps important.png
Warning:
Do not change the standard scripts and configuration files of dbSQWare.

Use directories *_cust for customizations. This will facilitate version upgrades.


Naming convention in scripts

In general, the following naming convention is used in all scripts of SQWareProduction:

  • gvsqw_ {} : global variable initialized by the environment and / or generic library
  • lvsqw_ {} : local variable initialized by the script and / or a specific library
  • gfsqw_ {} : function defined by a generic library
  • lfsqw_ {} : function defined by the script and / or a specific library

bin

sqwmng_CompactAllDatabases.ksh

Script $gvsqw_MngBin/bin/sqwmng_CompactAllDatabases.ksh:

compact tables of databases in parallel See the detailed help for sqwmng_CompactAllDatabases.ksh

sqwmng_CompactDatabase.ksh

Script $gvsqw_MngBin/bin/sqwmng_CompactDatabase.ksh:

compact tables of databas See the detailed help for sqwmng_CompactDatabase.ksh

sqwmng_CopyDumpAllDatabases.ksh

Script $gvsqw_MngBin/bin/sqwmng_CopyDumpAllDatabases.ksh:

copy of dump all databases in parallel See the detailed help for sqwmng_CopyDumpAllDatabases.ksh

sqwmng_CopyDumpDatabase.ksh

Script $gvsqw_MngBin/bin/sqwmng_CopyDumpDatabase.ksh:

copy of dump $lvsqw_CopyType See the detailed help for sqwmng_CopyDumpDatabase.ksh

sqwmng_DumpAllDatabases.ksh

Script $gvsqw_MngBin/bin/sqwmng_DumpAllDatabases.ksh:

Dump databases in parallel See the detailed help for sqwmng_DumpAllDatabases.ksh

sqwmng_DumpDatabase.ksh

Script $gvsqw_MngBin/bin/sqwmng_DumpDatabase.ksh:

mongodump of database See the detailed help for sqwmng_DumpDatabase.ksh

sqwmng_ExecSQL.ksh

Script $gvsqw_MngBin/bin/sqwmng_ExecSQL.ksh:

execute SQL file on instance See the detailed help for sqwmng_ExecSQL.ksh

sqwmng_FailWithoutMail.ksh

Script $gvsqw_MngBin/bin/sqwmng_FailWithoutMail.ksh:

See the detailed help for sqwmng_FailWithoutMail.ksh

sqwmng_GatherIndicators.ksh

Script $gvsqw_MngBin/bin/sqwmng_GatherIndicators.ksh:

gather DBA indicators See the detailed help for sqwmng_GatherIndicators.ksh

sqwmng_NetBackup.ksh

Script $gvsqw_MngBin/bin/sqwmng_NetBackup.ksh:

backup with NetBackup dump/backup files See the detailed help for sqwmng_NetBackup.ksh

sqwmng_ParallelRun.ksh

Script $gvsqw_MngBin/bin/sqwmng_ParallelRun.ksh:

Manages the execution of unix commands in parallel
Takes in parameter file with unix command to execute
You can choose:

  • The parallel degree
  • Aggressiveness of run (number of seconds between two runs)
  • Aggressiveness of check end (number of seconds between two checks of end)


See the detailed help for sqwmng_ParallelRun.ksh

sqwmng_RestoreAllDatabases.ksh

Script $gvsqw_MngBin/bin/sqwmng_RestoreAllDatabases.ksh:

restore all databases in parallel See the detailed help for sqwmng_RestoreAllDatabases.ksh

sqwmng_RestoreDatabase.ksh

Script $gvsqw_MngBin/bin/sqwmng_RestoreDatabase.ksh:

restore database See the detailed help for sqwmng_RestoreDatabase.ksh

sqwmng_RunJob.ksh

Script $gvsqw_MngBin/bin/sqwmng_RunJob.ksh:

Run MongoDB scripts and generate log and error file
based on $gvsqw_RootCfg/sqwmng_Jobs.cfg configuration file
compress and purge oldest log files See the detailed help for sqwmng_RunJob.ksh


etc

sqwmng_CreInst_CrontabRef.cfg

Configuration file $gvsqw_MngBin/../etc/sqwmng_CreInst_CrontabRef.cfg:


See the detailed help for sqwmng_CreInst_CrontabRef.cfg

sqwmng_CreInst_Jobs.cfg

Configuration file $gvsqw_MngBin/../etc/sqwmng_CreInst_Jobs.cfg:


See the detailed help for sqwmng_CreInst_Jobs.cfg

sqwmng_GlobalVar.cfg

Configuration file $gvsqw_MngBin/../etc/sqwmng_GlobalVar.cfg:
Global var for MongoDB
It contains the operating variables of SQWareProduction.

Nuvola apps information.png
Note:
You can overload variables contain in $gvsqw_MngBin/../etc/sqwmng_GlobalVar.cfg

by the same variables in $gvsqw_MngBin/../etc_cust/sqwmng_GlobalVar.cfg for all your environnement
or in $HOME/sqwConfig/sqwmng_GlobalVar.cfg specifically for this user.

See the detailed help for sqwmng_GlobalVar.cfg

.profile

Sample file for .profile $gvsqw_MngBin/../etc/.profile:
Sample of lines to add to .profile of mongodb users.
See the detailed help for .profile

.profile_confort

File to put aliases $gvsqw_MngBin/../etc/.profile_confort:
Contains confort aliases of SQWareProduction for mongodb.

Nuvola apps information.png
Note:
You can overload confort environnement $gvsqw_MngBin/../etc/.profile_confort

with $gvsqw_MngBin/../etc_cust/.profile_confort for all your environnement
or with $HOME/edtConfig/.profile_confort specifically for this user .


See the detailed help for .profile_confort


lib

sqwmng_CompactAllDatabases.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_CompactAllDatabases.lib:
lib for sqwmng_CompactAllDatabases.ksh

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_MngBin/../lib/sqwmng_CompactAllDatabases.lib

by function with same name in $gvsqw_MngBin/../lib_cust/sqwmng_CompactAllDatabases.lib for all your environnement
or by $HOME/sqwConfig/sqwmng_CompactAllDatabases.lib specifically for this user .


See the detailed help for sqwmng_CompactAllDatabases.lib

sqwmng_CompactDatabase.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_CompactDatabase.lib:
lib for sqwmng_CompactDatabase.ksh

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_MngBin/../lib/sqwmng_CompactDatabase.lib

by function with same name in $gvsqw_MngBin/../lib_cust/sqwmng_CompactDatabase.lib for all your environnement
or by $HOME/sqwConfig/sqwmng_CompactDatabase.lib specifically for this user .


See the detailed help for sqwmng_CompactDatabase.lib

sqwmng_CopyDumpAllDatabases.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_CopyDumpAllDatabases.lib:
lib for sqwmng_CopyDumpAllDatabases.ksh

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_MngBin/../lib/sqwmng_CopyDumpAllDatabases.lib

by function with same name in $gvsqw_MngBin/../lib_cust/sqwmng_CopyDumpAllDatabases.lib for all your environnement
or by $HOME/sqwConfig/sqwmng_CopyDumpAllDatabases.lib specifically for this user .


See the detailed help for sqwmng_CopyDumpAllDatabases.lib

sqwmng_CopyDumpDatabase.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_CopyDumpDatabase.lib:
lib for sqwmng_CopyDumpDatabase.ksh

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_MngBin/../lib/sqwmng_CopyDumpDatabase.lib

by function with same name in $gvsqw_MngBin/../lib_cust/sqwmng_CopyDumpDatabase.lib for all your environnement
or by $HOME/sqwConfig/sqwmng_CopyDumpDatabase.lib specifically for this user .


See the detailed help for sqwmng_CopyDumpDatabase.lib

sqwmng_DumpAllDatabases.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_DumpAllDatabases.lib:
lib for sqwmng_DumpAllDatabases.ksh

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_MngBin/../lib/sqwmng_DumpAllDatabases.lib

by function with same name in $gvsqw_MngBin/../lib_cust/sqwmng_DumpAllDatabases.lib for all your environnement
or by $HOME/sqwConfig/sqwmng_DumpAllDatabases.lib specifically for this user .


See the detailed help for sqwmng_DumpAllDatabases.lib

sqwmng_DumpDatabase.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_DumpDatabase.lib:
lib for sqwmng_DumpDatabase.ksh

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_MngBin/../lib/sqwmng_DumpDatabase.lib

by function with same name in $gvsqw_MngBin/../lib_cust/sqwmng_DumpDatabase.lib for all your environnement
or by $HOME/sqwConfig/sqwmng_DumpDatabase.lib specifically for this user .


See the detailed help for sqwmng_DumpDatabase.lib

sqwmng_GatherIndicators.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_GatherIndicators.lib:
lib for sqwmng_GatherIndicators.ksh

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_MngBin/../lib/sqwmng_GatherIndicators.lib

by function with same name in $gvsqw_MngBin/../lib_cust/sqwmng_GatherIndicators.lib for all your environnement
or by $HOME/sqwConfig/sqwmng_GatherIndicators.lib specifically for this user .


See the detailed help for sqwmng_GatherIndicators.lib

sqwmng_GatherIndicators_Overload.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_GatherIndicators_Overload.lib:
lib for overLoad sqwmng_GatherIndicators.ksh

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_MngBin/../lib/sqwmng_GatherIndicators_Overload.lib

by function with same name in $gvsqw_MngBin/../lib_cust/sqwmng_GatherIndicators_Overload.lib for all your environnement
or by $HOME/sqwConfig/sqwmng_GatherIndicators_Overload.lib specifically for this user .


See the detailed help for sqwmng_GatherIndicators_Overload.lib

sqwmng_GenerateCreateInstance.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_GenerateCreateInstance.lib:
lib for generate create instance

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_MngBin/../lib/sqwmng_GenerateCreateInstance.lib

by function with same name in $gvsqw_MngBin/../lib_cust/sqwmng_GenerateCreateInstance.lib for all your environnement
or by $HOME/sqwConfig/sqwmng_GenerateCreateInstance.lib specifically for this user .


See the detailed help for sqwmng_GenerateCreateInstance.lib

sqwmng_Global.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_Global.lib:
central lib of generic fonctions for mongodb

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_MngBin/../lib/sqwmng_Global.lib

by function with same name in $gvsqw_MngBin/../lib_cust/sqwmng_Global.lib for all your environnement
or by $HOME/sqwConfig/sqwmng_Global.lib specifically for this user .


See the detailed help for sqwmng_Global.lib

sqwmng_NetBackup.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_NetBackup.lib:
lib for sqwgen_NetBackup.ksh

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_MngBin/../lib/sqwmng_NetBackup.lib

by function with same name in $gvsqw_MngBin/../lib_cust/sqwmng_NetBackup.lib for all your environnement
or by $HOME/sqwConfig/sqwmng_NetBackup.lib specifically for this user .


See the detailed help for sqwmng_NetBackup.lib

sqwmng_RestoreAllDatabases.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_RestoreAllDatabases.lib:
lib for sqwmng_RestoreAllDatabases.ksh

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_MngBin/../lib/sqwmng_RestoreAllDatabases.lib

by function with same name in $gvsqw_MngBin/../lib_cust/sqwmng_RestoreAllDatabases.lib for all your environnement
or by $HOME/sqwConfig/sqwmng_RestoreAllDatabases.lib specifically for this user .


See the detailed help for sqwmng_RestoreAllDatabases.lib

sqwmng_RestoreDatabase.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_RestoreDatabase.lib:
lib for sqwmng_RestoreDatabase.ksh

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_MngBin/../lib/sqwmng_RestoreDatabase.lib

by function with same name in $gvsqw_MngBin/../lib_cust/sqwmng_RestoreDatabase.lib for all your environnement
or by $HOME/sqwConfig/sqwmng_RestoreDatabase.lib specifically for this user .


See the detailed help for sqwmng_RestoreDatabase.lib

sqwmng_SetEnv.lib

shell functions library $gvsqw_MngBin/../lib/sqwmng_SetEnv.lib:
fonction to set env for $MNG_SID

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_MngBin/../lib/sqwmng_SetEnv.lib

by function with same name in $gvsqw_MngBin/../lib_cust/sqwmng_SetEnv.lib for all your environnement
or by $HOME/sqwConfig/sqwmng_SetEnv.lib specifically for this user .


See the detailed help for sqwmng_SetEnv.lib


menu


tools

sqwmng_Crypt.ksh

Script $gvsqw_MngBin/../tools/sqwmng_Crypt.ksh:
Generic script to crypt file with openssl See the detailed help for sqwmng_Crypt.ksh

sqwmng_GenerateCreateInstance.ksh

Script $gvsqw_MngBin/../tools/sqwmng_GenerateCreateInstance.ksh:
Generate create instance files See the detailed help for sqwmng_GenerateCreateInstance.ksh

sqwmng_ListAsyncFilesGlobal.ksh

Script $gvsqw_MngBin/../tools/sqwmng_ListAsyncFilesGlobal.ksh:
List Async files (sql, log, mail) to upload in SQWareCentral (all in a directory) See the detailed help for sqwmng_ListAsyncFilesGlobal.ksh

sqwmng_ListAsyncFiles.ksh

Script $gvsqw_MngBin/../tools/sqwmng_ListAsyncFiles.ksh:
List Async files (sql, log, mail) to upload in SQWareCentral See the detailed help for sqwmng_ListAsyncFiles.ksh

sqwmng_SetEnv.ksh

Script $gvsqw_MngBin/../tools/sqwmng_SetEnv.ksh:
set env for $MNG_SID

Nuvola apps information.png
Note:
You can replace the file in $gvsqw_MngBin/../tools/sqwmng_SetEnv.ksh

by $gvsqw_MngBin/../tools_cust/sqwmng_SetEnv.ksh for all your environnement
or by $HOME/edtConfig/sqwmng_SetEnv.ksh specifically for this user .

See the detailed help for sqwmng_SetEnv.ksh

sqwmng_TestInstConnection.ksh

Script $gvsqw_MngBin/../tools/sqwmng_TestInstConnection.ksh:
Test instance connection See the detailed help for sqwmng_TestInstConnection.ksh

sqwmng_TestMySQL.ksh

Script $gvsqw_MngBin/../tools/sqwmng_TestMySQL.ksh:
Generic test MySQL connection script See the detailed help for sqwmng_TestMySQL.ksh

sqwmng_TestSendmail.ksh

Script $gvsqw_MngBin/../tools/sqwmng_TestSendmail.ksh:
Generic test sendmail script See the detailed help for sqwmng_TestSendmail.ksh

sqwmng_Uncrypt.ksh

Script $gvsqw_MngBin/../tools/sqwmng_Uncrypt.ksh:
Generic script to uncrypt file with openssl See the detailed help for sqwmng_Uncrypt.ksh


help

Contains help files for "-s" option of bin scripts.


spm

SQL query toolkit
See the detailed help for spm