SQWareProduction:cassandra

From Wiki_dbSQWare
Revision as of 15:16, 22 July 2018 by BatchUser (talk | contribs) (Updated by BatchUser)
Jump to: navigation, search

Presentation

Usage

It's the module locally operating of cassandra.
It is entirely written in ksh and sql.
It allows to manage the operation for cassandra.
Example uses of the tool:

  • Backup
  • Recovery
  • Compute statistics
  • Indicators collection to store in referential database SQWareRepository
  • Rotate logs
  • ....

This module is deployed locally on the machine hosting the authorities to manage, in order to avoid any points of weakness related to an NFS mount or whatever. The deployment is done through a simple command SQWareCentral module using inventory located in SQWareRepository !
Installation guide for SQWareProduction for cassandra

Explanation of directories

Directories located in $gvsqw_CasBin:

  • bin : Directory containing standard scripts
  • bin_cust : Directory containing customs scripts for your environnement
  • etc : Directory containing the standard files of configuration
  • etc_cust : Directory containing the customs files of configuration for your environnement (overload standards)
  • help : Directory containing the standard files of help
  • help_cust : Directory containing the customs files of help for your environnement
  • lib : Directory containing the generic shell function librairies
  • lib_cust : Directory containing the generic shell function librairies (overload standards)
  • spm : Contain sql query toolkit
  • tools : Directory containing the standard scripts used on an ad hoc
  • tools_cust : Directory containing the customs scripts used on an ad hoc for your environnement
  • menu : Directory containing the standard menus
  • menu_cust : Directory containing the custom menus for your environnement
Nuvola apps important.png
Warning:
Do not change the standard scripts and configuration files of dbSQWare.

Use directories *_cust for customizations. This will facilitate version upgrades.


Naming convention of files

The majority of files are named with suffixes:

  • *.ksh : for scripts
  • *.cfg : for configuration files
  • *.lib : for function librairies
  • *.hlp : for help files
Nuvola apps important.png
Warning:
Do not change the standard scripts and configuration files of dbSQWare.

Use directories *_cust for customizations. This will facilitate version upgrades.


Naming convention in scripts

In general, the following naming convention is used in all scripts of SQWareProduction:

  • gvsqw_ {} : global variable initialized by the environment and / or generic library
  • lvsqw_ {} : local variable initialized by the script and / or a specific library
  • gfsqw_ {} : function defined by a generic library
  • lfsqw_ {} : function defined by the script and / or a specific library

bin

sqwcas_Backup.ksh

Script $gvsqw_CasBin/bin/sqwcas_Backup.ksh:

Run a backup on all nodes

  • Scope d'execution (defaut: Local)


** Local => seulement le node local
** LocalNode => seulement le node local
** Node_$NodeName => seulement sur le node '$NodeName'
** LocalDC => tous les nodes sur local datacenter
** DC_$DCName => tous les nodes sur datacenter '$DCName'
** All => tous les nodes sur cluster(tous les datacenters)
** AllNodes => tous les nodes sur cluster(tous les datacenters)
** AllDC => tous les nodes sur cluster(tous les datacenters) See the detailed help for sqwcas_Backup.ksh

sqwcas_CentralisedNodetool.ksh

Script $gvsqw_CasBin/bin/sqwcas_CentralisedNodetool.ksh:

Execute nodetool command on all selected nodes in parallel
You have to select le options of nodetool to execute
You can also choose:

  • Aggressiveness of run (number of seconds between two runs)
  • Aggressiveness of check end (number of seconds between two checks of end)
  • Scope of the execution (default: Local)


** Local => only on the local node
** LocalNode => only on the local node
** Node_$NodeName => only on the node '$NodeName'
** LocalDC => all nodes on the local datacenter
** DC_$DCName => all nodes on the datacenter '$DCName'
** All => all nodes on the cluster(all datacenters)
** AllNodes => all nodes on the cluster(all datacenters)
** AllDC => all nodes on the cluster(all datacenters)

  • Force dynamic discovery of nodes (by default, search in $gvsqw_RootCfg/CAS_CHR_PRD.clucfg generated by sqwcas_GenNodesConf.ksh)

See the detailed help for sqwcas_CentralisedNodetool.ksh

sqwcas_ExecSQL.ksh

Script $gvsqw_CasBin/bin/sqwcas_ExecSQL.ksh:

execute SQL file on instance See the detailed help for sqwcas_ExecSQL.ksh

sqwcas_GatherIndicators.ksh

Script $gvsqw_CasBin/bin/sqwcas_GatherIndicators.ksh:

Gather Cassandra indicators and put them in SQWareRepository
Summary of collection:

  • Volumetry of instance
  • Volumetry of nodes
  • Volumetry of keyspaces (by node and global)
  • Volumetry of column familly (by node and global)
  • Volumetry (use, free) of FS
  • Version, options, configuration, ... of instance

See the detailed help for sqwcas_GatherIndicators.ksh

sqwcas_NetBackup.ksh

Script $gvsqw_CasBin/bin/sqwcas_NetBackup.ksh:

backup with NetBackup dump/backup files See the detailed help for sqwcas_NetBackup.ksh

sqwcas_ParallelRun.ksh

Script $gvsqw_CasBin/bin/sqwcas_ParallelRun.ksh:

Manages the execution of unix commands in parallel
Takes in parameter file with unix command to execute
You can choose:

  • The parallel degree
  • Aggressiveness of run (number of seconds between two runs)
  • Aggressiveness of check end (number of seconds between two checks of end)


See the detailed help for sqwcas_ParallelRun.ksh

sqwcas_RollingRestart.ksh

Script $gvsqw_CasBin/bin/sqwcas_RollingRestart.ksh:

Run 'stop/start/restart' cassandra on selected nodes See the detailed help for sqwcas_RollingRestart.ksh

sqwcas_RunJob.ksh

Script $gvsqw_CasBin/bin/sqwcas_RunJob.ksh:

Run Cassandra scripts and generate log and error file
based on $gvsqw_RootCfg/sqwcas_Jobs.cfg configuration file
compress and purge oldest log files See the detailed help for sqwcas_RunJob.ksh


etc

sqwcas_CreInst_CrontabRef.cfg

Configuration file $gvsqw_CasBin/../etc/sqwcas_CreInst_CrontabRef.cfg:


See the detailed help for sqwcas_CreInst_CrontabRef.cfg

sqwcas_CreInst_Jobs.cfg

Configuration file $gvsqw_CasBin/../etc/sqwcas_CreInst_Jobs.cfg:


See the detailed help for sqwcas_CreInst_Jobs.cfg

sqwcas_GlobalVar.cfg

Configuration file $gvsqw_CasBin/../etc/sqwcas_GlobalVar.cfg:
Global var for Cassandra
It contains the operating variables of SQWareProduction.

Nuvola apps information.png
Note:
You can overload variables contain in $gvsqw_CasBin/../etc/sqwcas_GlobalVar.cfg

by the same variables in $gvsqw_CasBin/../etc_cust/sqwcas_GlobalVar.cfg for all your environnement
or in $HOME/sqwConfig/sqwcas_GlobalVar.cfg specifically for this user.

See the detailed help for sqwcas_GlobalVar.cfg

.profile

Sample file for .profile $gvsqw_CasBin/../etc/.profile:
Sample of lines to add to .profile of cassandra users.
See the detailed help for .profile

.profile_confort

File to put aliases $gvsqw_CasBin/../etc/.profile_confort:
Contains confort aliases of SQWareProduction for RDBMS cassandra.

Nuvola apps information.png
Note:
You can overload confort environnement $gvsqw_CasBin/../etc/.profile_confort

with $gvsqw_CasBin/../etc_cust/.profile_confort for all your environnement
or with $HOME/edtConfig/.profile_confort specifically for this user .


See the detailed help for .profile_confort


lib

sqwcas_Backup.lib

shell functions library $gvsqw_CasBin/../lib/sqwcas_Backup.lib:
lib for sqwcas_Backup.ksh

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_CasBin/../lib/sqwcas_Backup.lib

by function with same name in $gvsqw_CasBin/../lib_cust/sqwcas_Backup.lib for all your environnement
or by $HOME/sqwConfig/sqwcas_Backup.lib specifically for this user .


See the detailed help for sqwcas_Backup.lib

sqwcas_CentralisedNodetool.lib

shell functions library $gvsqw_CasBin/../lib/sqwcas_CentralisedNodetool.lib:
lib for sqwcas_CentralisedNodetool.ksh

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_CasBin/../lib/sqwcas_CentralisedNodetool.lib

by function with same name in $gvsqw_CasBin/../lib_cust/sqwcas_CentralisedNodetool.lib for all your environnement
or by $HOME/sqwConfig/sqwcas_CentralisedNodetool.lib specifically for this user .


See the detailed help for sqwcas_CentralisedNodetool.lib

sqwcas_GatherIndicators.lib

shell functions library $gvsqw_CasBin/../lib/sqwcas_GatherIndicators.lib:
lib for sqwcas_GatherIndicators.ksh

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_CasBin/../lib/sqwcas_GatherIndicators.lib

by function with same name in $gvsqw_CasBin/../lib_cust/sqwcas_GatherIndicators.lib for all your environnement
or by $HOME/sqwConfig/sqwcas_GatherIndicators.lib specifically for this user .


See the detailed help for sqwcas_GatherIndicators.lib

sqwcas_GatherIndicators_Overload.lib

shell functions library $gvsqw_CasBin/../lib/sqwcas_GatherIndicators_Overload.lib:
lib for overLoad sqwcas_GatherIndicators.ksh

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_CasBin/../lib/sqwcas_GatherIndicators_Overload.lib

by function with same name in $gvsqw_CasBin/../lib_cust/sqwcas_GatherIndicators_Overload.lib for all your environnement
or by $HOME/sqwConfig/sqwcas_GatherIndicators_Overload.lib specifically for this user .


See the detailed help for sqwcas_GatherIndicators_Overload.lib

sqwcas_GenerateCreateInstance.lib

shell functions library $gvsqw_CasBin/../lib/sqwcas_GenerateCreateInstance.lib:
lib for generate create instance

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_CasBin/../lib/sqwcas_GenerateCreateInstance.lib

by function with same name in $gvsqw_CasBin/../lib_cust/sqwcas_GenerateCreateInstance.lib for all your environnement
or by $HOME/sqwConfig/sqwcas_GenerateCreateInstance.lib specifically for this user .


See the detailed help for sqwcas_GenerateCreateInstance.lib

sqwcas_GenNodesConf.lib

shell functions library $gvsqw_CasBin/../lib/sqwcas_GenNodesConf.lib:
lib for sqwcas_GenNodesConf.ksh

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_CasBin/../lib/sqwcas_GenNodesConf.lib

by function with same name in $gvsqw_CasBin/../lib_cust/sqwcas_GenNodesConf.lib for all your environnement
or by $HOME/sqwConfig/sqwcas_GenNodesConf.lib specifically for this user .


See the detailed help for sqwcas_GenNodesConf.lib

sqwcas_Global.lib

shell functions library $gvsqw_CasBin/../lib/sqwcas_Global.lib:
central lib of generic fonctions for cassandra

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_CasBin/../lib/sqwcas_Global.lib

by function with same name in $gvsqw_CasBin/../lib_cust/sqwcas_Global.lib for all your environnement
or by $HOME/sqwConfig/sqwcas_Global.lib specifically for this user .


See the detailed help for sqwcas_Global.lib

sqwcas_NetBackup.lib

shell functions library $gvsqw_CasBin/../lib/sqwcas_NetBackup.lib:
lib for sqwgen_NetBackup.ksh

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_CasBin/../lib/sqwcas_NetBackup.lib

by function with same name in $gvsqw_CasBin/../lib_cust/sqwcas_NetBackup.lib for all your environnement
or by $HOME/sqwConfig/sqwcas_NetBackup.lib specifically for this user .


See the detailed help for sqwcas_NetBackup.lib

sqwcas_RollingRestart.lib

shell functions library $gvsqw_CasBin/../lib/sqwcas_RollingRestart.lib:
lib for sqwcas_RollingRestart.ksh

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_CasBin/../lib/sqwcas_RollingRestart.lib

by function with same name in $gvsqw_CasBin/../lib_cust/sqwcas_RollingRestart.lib for all your environnement
or by $HOME/sqwConfig/sqwcas_RollingRestart.lib specifically for this user .


See the detailed help for sqwcas_RollingRestart.lib

sqwcas_SetEnv.lib

shell functions library $gvsqw_CasBin/../lib/sqwcas_SetEnv.lib:
fonction to set env for $CAS_SID

Nuvola apps information.png
Note:
You can overload functions contained in $gvsqw_CasBin/../lib/sqwcas_SetEnv.lib

by function with same name in $gvsqw_CasBin/../lib_cust/sqwcas_SetEnv.lib for all your environnement
or by $HOME/sqwConfig/sqwcas_SetEnv.lib specifically for this user .


See the detailed help for sqwcas_SetEnv.lib


menu

sqwcas_GenNodesConf.ksh

Menu $gvsqw_CasBin/../menu/sqwcas_GenNodesConf.ksh:
Generate conf files for all node in the cluster (nodes list by DC) See the detailed help for sqwcas_GenNodesConf.ksh


tools

sqwcas_GenerateCreateInstance.ksh

Script $gvsqw_CasBin/../tools/sqwcas_GenerateCreateInstance.ksh:
Generate create instance files See the detailed help for sqwcas_GenerateCreateInstance.ksh

sqwcas_SetEnv.ksh

Script $gvsqw_CasBin/../tools/sqwcas_SetEnv.ksh:
set env for $CAS_SID

Nuvola apps information.png
Note:
You can replace the file in $gvsqw_CasBin/../tools/sqwcas_SetEnv.ksh

by $gvsqw_CasBin/../tools_cust/sqwcas_SetEnv.ksh for all your environnement
or by $HOME/edtConfig/sqwcas_SetEnv.ksh specifically for this user .

See the detailed help for sqwcas_SetEnv.ksh

sqwcas_TestMySQL.ksh

Script $gvsqw_CasBin/../tools/sqwcas_TestMySQL.ksh:
Generic test MySQL connection script See the detailed help for sqwcas_TestMySQL.ksh

sqwcas_TestSendmail.ksh

Script $gvsqw_CasBin/../tools/sqwcas_TestSendmail.ksh:
Generic test sendmail script See the detailed help for sqwcas_TestSendmail.ksh


help

Contains help files for -s option of bin scripts.


spm

SQL query toolkit
See the detailed help for spm