SQWareProduction:postgres

Presentation

Usage

It's the module locally operating of postgres.
It is entirely written in ksh and sql.
It allows to manage the operation for postgres.
Example uses of the tool:

• Backup
• Recovery
• Compute statistics
• Indicators collection to store in referential database SQWareRepository
• Rotate logs
• ....

This module is deployed locally on the machine hosting the authorities to manage, in order to avoid any points of weakness related to an NFS mount or whatever. The deployment is done through a simple command SQWareCentral module using inventory located in SQWareRepository !
Installation guide for SQWareProduction for postgres

Explanation of directories

Directories located in $gvsqw_PgBin:

• bin : Directory containing standard scripts
• bin_cust : Directory containing customs scripts for your environnement
• etc : Directory containing the standard files of configuration
• etc_cust : Directory containing the customs files of configuration for your environnement (overload standards)
• help : Directory containing the standard files of help
• help_cust : Directory containing the customs files of help for your environnement
• lib : Directory containing the generic shell function librairies
• lib_cust : Directory containing the generic shell function librairies (overload standards)
• spm : Contain sql query toolkit
• tools : Directory containing the standard scripts used on an ad hoc
• tools_cust : Directory containing the customs scripts used on an ad hoc for your environnement
• menu : Directory containing the standard menus
• menu_cust : Directory containing the custom menus for your environnement

Naming convention of files

The majority of files are named with suffixes:

• *.ksh : for scripts
• *.cfg : for configuration files
• *.lib : for function librairies
• *.hlp : for help files

Naming convention in scripts

In general, the following naming convention is used in all scripts of SQWareProduction:

• gvsqw_ {} : global variable initialized by the environment and / or generic library
• lvsqw_ {} : local variable initialized by the script and / or a specific library
• gfsqw_ {} : function defined by a generic library
• lfsqw_ {} : function defined by the script and / or a specific library

bin

sqwArchmon

Script $gvsqw_PgBin/bin/sqwArchmon:

See the detailed help for sqwArchmon

sqwpg_AnalyzeAllDatabases.ksh

Script $gvsqw_PgBin/bin/sqwpg_AnalyzeAllDatabases.ksh:

Run of sqwpg_AnalyzeDatabase.ksh in parallel mode on all databases
You can choose:

• The parallel degree
• Aggressiveness of run (number of seconds between two runs)
• Aggressiveness of check end (number of seconds between two checks of end)

Filters to generate database list (you can mix all off them):

• Include list of database
• Exclude list of database
• Include like clause
• Exclude like clause
• Where clause

See the detailed help for sqwpg_AnalyzeAllDatabases.ksh

sqwpg_AnalyzeDatabase.ksh

Script $gvsqw_PgBin/bin/sqwpg_AnalyzeDatabase.ksh:

Update statistics of all tables for the database
Use command: analyse local table <TableName> See the detailed help for sqwpg_AnalyzeDatabase.ksh

sqwpg_ArchiveXlog.ksh

Script $gvsqw_PgBin/bin/sqwpg_ArchiveXlog.ksh:

archive xlog/wal See the detailed help for sqwpg_ArchiveXlog.ksh

sqwpg_BackupArchiveXlog.ksh

Script $gvsqw_PgBin/bin/sqwpg_BackupArchiveXlog.ksh:

Backup all archivelogs (often runned by the sqwarchmon daemon) See the detailed help for sqwpg_BackupArchiveXlog.ksh

sqwpg_BackupPitr.ksh

Script $gvsqw_PgBin/bin/sqwpg_BackupPitr.ksh:

backup PITR from PG cluster See the detailed help for sqwpg_BackupPitr.ksh

sqwpg_DumpAllDatabases.ksh

Script $gvsqw_PgBin/bin/sqwpg_DumpAllDatabases.ksh:

Run of sqwpg_DumpDatabase.ksh in parallel mode
You can choose:

• The parallel degree
• Aggressiveness of run (number of seconds between two runs)
• Aggressiveness of check end (number of seconds between two checks of end)

Filters to generate database list (you can mix all off them):

• Include list of database
• Exclude list of database
• Include like clause
• Exclude like clause
• Where clause

See the detailed help for sqwpg_DumpAllDatabases.ksh

sqwpg_DumpDatabase.ksh

Script $gvsqw_PgBin/bin/sqwpg_DumpDatabase.ksh:

Generate pg_dump of database
Script makes purge of old generation automatically
You can also choose:

• different retention policy
• add option to pg_dump command
• choose compressor

See the detailed help for sqwpg_DumpDatabase.ksh

sqwpg_DumpGlobalsOnly.ksh

Script $gvsqw_PgBin/bin/sqwpg_DumpGlobalsOnly.ksh:

Generate pg_dumpall -g (--globals-only) compressed by pipe
Script makes purge of old generation automatically
You can also choose:

• different retention policy
• add option to pg_dumpall command
• choose compressor

See the detailed help for sqwpg_DumpGlobalsOnly.ksh

sqwpg_ExecSQL.ksh

Script $gvsqw_PgBin/bin/sqwpg_ExecSQL.ksh:

execute SQL file on instance See the detailed help for sqwpg_ExecSQL.ksh

sqwpg_GatherIndicators.ksh

Script $gvsqw_PgBin/bin/sqwpg_GatherIndicators.ksh:

Gather PostgreSQL indicators and put them in SQWareRepository
Summary of collection:

• Volumetry of instance
• Volumetry of databases
• Volumetry of tablespaces
• Volumetry of tables
• Volumetry (use, free) of FS
• Version, options, configuration, ... of instance

See the detailed help for sqwpg_GatherIndicators.ksh

sqwpg_GatherStructure.ksh

Script $gvsqw_PgBin/bin/sqwpg_GatherStructure.ksh:

Gather PostgreSQL structure and put them in SQWareRepository
Summary of collection:

• Tables list and options
• Columns list, types and options
• Indexes list, types and options

See the detailed help for sqwpg_GatherStructure.ksh

sqwpg_NetBackup.ksh

Script $gvsqw_PgBin/bin/sqwpg_NetBackup.ksh:

backup with NetBackup dump/backup files See the detailed help for sqwpg_NetBackup.ksh

sqwpg_ParallelRun.ksh

Script $gvsqw_PgBin/bin/sqwpg_ParallelRun.ksh:

Manages the execution of unix commands in parallel mode
Takes in parameter file with unix command to execute
You can choose:

• The parallel degree
• Aggressiveness of run (number of seconds between two runs)
• Aggressiveness of check end (number of seconds between two checks of end)

See the detailed help for sqwpg_ParallelRun.ksh

sqwpg_ReindexAllDatabases.ksh

Script $gvsqw_PgBin/bin/sqwpg_ReindexAllDatabases.ksh:

Run of sqwpg_ReindexDatabase.ksh in parallel mode
You can choose:

• The parallel degree
• Aggressiveness of run (number of seconds between two runs)
• Aggressiveness of check end (number of seconds between two checks of end)

Filters to generate database list (you can mix all off them):

• Include list of database
• Exclude list of database
• Include like clause
• Exclude like clause
• Where clause

See the detailed help for sqwpg_ReindexAllDatabases.ksh

sqwpg_ReindexDatabase.ksh

Script $gvsqw_PgBin/bin/sqwpg_ReindexDatabase.ksh:

Reorganize of all tables for the database
Use command: optimize local table <TableName> See the detailed help for sqwpg_ReindexDatabase.ksh

sqwpg_RsyncBackup.ksh

Script $gvsqw_PgBin/bin/sqwpg_RsyncBackup.ksh:

rsync dir (without delete) to an other host See the detailed help for sqwpg_RsyncBackup.ksh

sqwpg_RunJob.ksh

Script $gvsqw_PgBin/bin/sqwpg_RunJob.ksh:

Run PostgreSQL scripts and generate log and error file
based on $gvsqw_RootCfg/sqwpg_Jobs.cfg configuration file
compress and purge oldest log files See the detailed help for sqwpg_RunJob.ksh

sqwpg_SqwArchmon.ksh

Script $gvsqw_PgBin/bin/sqwpg_SqwArchmon.ksh:

See the detailed help for sqwpg_SqwArchmon.ksh

sqwpg_VacuumAllDatabases.ksh

Script $gvsqw_PgBin/bin/sqwpg_VacuumAllDatabases.ksh:

Run of sqwpg_VacuumDatabase.ksh in parallel mode
You can choose:

• The parallel degree
• Aggressiveness of run (number of seconds between two runs)
• Aggressiveness of check end (number of seconds between two checks of end)

Filters to generate database list (you can mix all off them):

• Include list of database
• Exclude list of database
• Include like clause
• Exclude like clause
• Where clause

See the detailed help for sqwpg_VacuumAllDatabases.ksh

sqwpg_VacuumDatabase.ksh

Script $gvsqw_PgBin/bin/sqwpg_VacuumDatabase.ksh:

VACUUM of all tables for the database
Use command: VACUUM See the detailed help for sqwpg_VacuumDatabase.ksh

etc

sqwpg_CreInst_CrontabRef.cfg

Configuration file $gvsqw_PgBin/../etc/sqwpg_CreInst_CrontabRef.cfg:


See the detailed help for sqwpg_CreInst_CrontabRef.cfg

sqwpg_CreInst_Jobs.cfg

Configuration file $gvsqw_PgBin/../etc/sqwpg_CreInst_Jobs.cfg:


See the detailed help for sqwpg_CreInst_Jobs.cfg

sqwpg_GlobalVar.cfg

Configuration file $gvsqw_PgBin/../etc/sqwpg_GlobalVar.cfg:
Global var for Postgres
It contains the operating variables of SQWareProduction.

See the detailed help for sqwpg_GlobalVar.cfg

.profile

Sample file for .profile $gvsqw_PgBin/../etc/.profile:
Sample of lines to add to .profile of postgres users.
See the detailed help for .profile

.profile_confort

File to put aliases $gvsqw_PgBin/../etc/.profile_confort:
Contains confort aliases of SQWareProduction for RDBMS postgres.

See the detailed help for .profile_confort

lib

sqwpg_ArchiveXlog.lib

shell functions library $gvsqw_PgBin/../lib/sqwpg_ArchiveXlog.lib:
lib for sqwpg_ArchiveXlog.ksh

See the detailed help for sqwpg_ArchiveXlog.lib

sqwpg_BackupArchiveXlog.lib

shell functions library $gvsqw_PgBin/../lib/sqwpg_BackupArchiveXlog.lib:
lib for sqwpg_BackupArchiveXlog.ksh

See the detailed help for sqwpg_BackupArchiveXlog.lib

sqwpg_BackupPitr.lib

shell functions library $gvsqw_PgBin/../lib/sqwpg_BackupPitr.lib:
lib for sqwpg_BackupPitr.ksh

See the detailed help for sqwpg_BackupPitr.lib

sqwpg_DumpAllDatabases.lib

shell functions library $gvsqw_PgBin/../lib/sqwpg_DumpAllDatabases.lib:
lib for sqwpg_DumpDatabase.ksh

See the detailed help for sqwpg_DumpAllDatabases.lib

sqwpg_DumpDatabase.lib

shell functions library $gvsqw_PgBin/../lib/sqwpg_DumpDatabase.lib:
lib for sqwpg_DumpDatabase.ksh

See the detailed help for sqwpg_DumpDatabase.lib

sqwpg_DumpGlobalsOnly.lib

shell functions library $gvsqw_PgBin/../lib/sqwpg_DumpGlobalsOnly.lib:
lib for sqwpg_DumpGlobalsOnly.ksh

See the detailed help for sqwpg_DumpGlobalsOnly.lib

sqwpg_ExecSQL.lib

shell functions library $gvsqw_PgBin/../lib/sqwpg_ExecSQL.lib:
lib for sqwpg_ExecSQL.ksh

See the detailed help for sqwpg_ExecSQL.lib

sqwpg_GatherIndicators.lib

shell functions library $gvsqw_PgBin/../lib/sqwpg_GatherIndicators.lib:
lib for sqwpg_GatherIndicators.ksh

See the detailed help for sqwpg_GatherIndicators.lib

sqwpg_GatherIndicators_Overload.lib

shell functions library $gvsqw_PgBin/../lib/sqwpg_GatherIndicators_Overload.lib:
lib for overLoad sqwpg_GatherIndicators.ksh

See the detailed help for sqwpg_GatherIndicators_Overload.lib

sqwpg_GatherStructure.lib

shell functions library $gvsqw_PgBin/../lib/sqwpg_GatherStructure.lib:
lib for sqwpg_GatherStructure.ksh

See the detailed help for sqwpg_GatherStructure.lib

sqwpg_GenerateCreateInstance.lib

shell functions library $gvsqw_PgBin/../lib/sqwpg_GenerateCreateInstance.lib:
lib for generate create instance

See the detailed help for sqwpg_GenerateCreateInstance.lib

sqwpg_Global.lib

shell functions library $gvsqw_PgBin/../lib/sqwpg_Global.lib:
central lib of generic fonctions for postgres

See the detailed help for sqwpg_Global.lib

sqwpg_NetBackup.lib

shell functions library $gvsqw_PgBin/../lib/sqwpg_NetBackup.lib:
lib for sqwgen_NetBackup.ksh

See the detailed help for sqwpg_NetBackup.lib

sqwpg_RsyncBackup.lib

shell functions library $gvsqw_PgBin/../lib/sqwpg_RsyncBackup.lib:
lib for sqwgen_RsyncBackup.ksh.ksh

See the detailed help for sqwpg_RsyncBackup.lib

sqwpg_SetEnv.lib

shell functions library $gvsqw_PgBin/../lib/sqwpg_SetEnv.lib:
fonction to set env for $PG_SID

See the detailed help for sqwpg_SetEnv.lib

tools

sqwpg_GenerateCreateInstance.ksh

Script $gvsqw_PgBin/../tools/sqwpg_GenerateCreateInstance.ksh:
Generate create instance files See the detailed help for sqwpg_GenerateCreateInstance.ksh

sqwpg_SetEnv.ksh

Script $gvsqw_PgBin/../tools/sqwpg_SetEnv.ksh:
set env for $PG_SID

See the detailed help for sqwpg_SetEnv.ksh

sqwpg_TestMySQL.ksh

Script $gvsqw_PgBin/../tools/sqwpg_TestMySQL.ksh:
Generic test MySQL connection script See the detailed help for sqwpg_TestMySQL.ksh

sqwpg_TestSendmail.ksh

Script $gvsqw_PgBin/../tools/sqwpg_TestSendmail.ksh:
Generic test sendmail script See the detailed help for sqwpg_TestSendmail.ksh

help

Contains help files for -s option of bin scripts.

spm

SQL query toolkit
See the detailed help for spm