Difference between revisions of "FAQ:NewInstanceMySQL"
m |
m (→Instance declaration in SQWareRepository with SQWareWeb) |
||
| Line 93: | Line 93: | ||
|- | |- | ||
| Db Alias | | Db Alias | ||
| − | | Unique key that identifies the instance in dbSQWare ( | + | | Unique key that identifies the instance in dbSQWare (no value for MySQL/MariaDB). |
|- | |- | ||
| Dbms Name | | Dbms Name | ||
Revision as of 09:59, 28 April 2025
Work In Progress
Contents
- 1 Generalties
- 2 Integration of an Oracle instance
- 2.1 Setting up the environment
- 2.2 Specific procedure for the "local" part (status ON)
- 2.2.1 Verification of system prerequisites for the "local" part (bash + rsync)
- 2.2.2 Deployment of SSH key(s) from SQWareCentral to the target machine(s)
- 2.2.3 SSH connection test from SQWareCentral
- 2.2.4 Test of prerequisites on the target host (bash+rsync) via SSH from SQWareCentral
- 2.2.5 Deployment of SQWareProduction
- 2.2.6 Adding the dbSQWare environment to ~/.profile or ~/.bash_profile
- 2.2.7 Test sendmail (Non mandatory)
- 2.2.8 Manual connection tests to the Oracle instance
- 2.3 Procédure spécifique pour la partie "distante" (statut DIST)
- 3 Mise en exploitation de l'instance
Generalties
In this chapter, we will consider that SQWareCentral has been installed on dbsqware@sqwarebox.
Limits of this section
This chapter only deals with adding a new instance to a base that is already configured and functional.
Use the portions that apply to your environment, and choose either the automatic or manual method as desired
General operating procedure for integrating a new Oracle instance
There will potentially be 1 or 2 connection cases to process
- "local" for Unix machines (or those with Cygwin) where a "/ as sysdba" will be performed on the machine hosting the instance (via SSH connection from dbsqware@sqwarebox)
- "Distant", for generating AWRs (if needed), or in cases where SSH access to the machine is not available (Windows, RDS, etc.)
Classic steps:
- Setting up the environment
- Checking system prerequisites for the "local" part (bash + rsync)
- Manual tests
- Deployment of the instance
You will see that these steps can be handled one by one or in batches!
Integration of an Oracle instance
Setting up the environment
General
Explanation of "DbAlias" (the unique dbSQWare key)
The dbSQWare key must be unique and consistent with what has been configured for the gvsqw_DbAlias variable in sqwora_GlobalVar.cfg!
There are more or less 3 cases...
- Your environment is "standalone" and your ORACLE_SID are unique across the entire environment => gvsqw_DbAlias='$ORACLE_SID'
- Your environment is "standalone" but you have "duplicates" of ORACLE_SID => gvsqw_DbAlias='$ORACLE_SID:$(hostname|cut -d '.' -f1)'
- You have RAC and/or DG => gvsqw_DbAlias='$gvsqw_DB_UNIQUE_NAME:$gvsqw_HOST_UNIQUE_NAME' (gvsqw_DB_UNIQUE_NAME, remove the node number!)
+ any other case that might occur or that "suits you"!
Explanation of statuses
- An "open" instance will be handled in "local" mode, meaning we will connect via "/ as sysdba" locally on the machine hosting the instance (ssh access from dbsqware@sqwarebox), status "ON".
- A "closed" instance, like dataguard, will be handled in "local" mode, meaning we will connect via "/ as sysdba" locally on the machine hosting the instance (ssh access from dbsqware@sqwarebox), status "DG".
- An instance where we don’t have local unix access will be managed in "remote" mode (like Windows, RDS, ...), status "DIST".
- The "OFF" status makes the instance "disappear" from the web view.
- The "NEW" status is used as an intermediate status between the beginning and the end of setting up a new instance.
- All other statuses allow the instance to be displayed in the repository (especially "KEEP") but will not be taken into account by the automatic processes.
Resume of statuses :
| Status | Description |
| ON | locale under Unix (the most common) |
| DIST | remote-only management |
| KEEP | temporary status, during maintenance for example or before complete deletion |
| NEW | temporary status, during configuration |
| OFF | visual removal of the instance (without deleting its configuration) |
| XXX | not supported... |
Instance declaration in SQWareRepository with SQWareWeb
We will declare the new instance from the SQWareWeb administration interface.
Declare the new instance with the status "NEW" (Add or Duplicate).
Explanation of the fields :
| Fields | Description |
| Db Alias | Unique key that identifies the instance in dbSQWare (no value for MySQL/MariaDB). |
| Dbms Name | Type of SGBD |
| Virtual Host | Virtual host (same as Host if not a cluster, will be used for generating the tnsname.ora). |
| Host Name | Hostname of the instance |
| User Name | User associated with the instance |
| Port | Listening port of the instance (will be used for generating the tnsname.ora). |
| Comments | A brief description of what the instance hosts |
| Status | Instance status (see above for explanations). |
| Contact | A contact if needed |
| Environnement | Instance environment (PRD, PPR, REC, DEV, TST, ...) |
| Client | Used only for filtering (enter a client name and/or department and/or service...) |
| GlobalHost | Free field in which the hypervisor host is often entered, for example. |
| Custom1 | Free field 1 |
| Custom2 | Free field 2 |
Regenerate the reference files of SQWareCentral.
Type the following command which will generate the reference files:
1 # From : dbsqware@sqwarebox
2 gen_all
Specific procedure for the "local" part (status ON)
/!\ Only instances that will be in the "ON" status !
Verification of system prerequisites for the "local" part (bash + rsync)
Since SQWareProduction is mainly written in bash shell and synchronized from SQWareCentral using rsync, we therefore need "bash" and "rsync" to be installed !
Verification
Adapt to your username and machine name.
bash:
1 # From : oracle@my_oracle_host
2 type bash
3 #ou
4 which bash
rsync:
1 # From : oracle@my_oracle_host
2 type rsync
3 #ou
4 which rsync
Installation
Adapt to your machine type (use sudo if you are not root).
RedHat / CentOS / ... :
1 # From : root@my_oracle_host
2 yum install -y bash rsync
3 #ou
4 dnf install -y bash rsync
Ubuntu / Debian / ... :
1 # From : root@my_oracle_host
2 apt install -y bash rsync
Deployment of SSH key(s) from SQWareCentral to the target machine(s)
There are two ways to proceed:
- An automatic one (from dbsqware@sqwarebox, but it requires you to know the password of the oracle Unix account)
- A manual one that you apply on each oracle Unix account
/!\ The password of your oracle Unix account must have been initialized, otherwise key-based authentication will not work!
Deployment of SSH key(s): batch method
1 # From : dbsqware@sqwarebox
2 menu_ora GenDeplSshKeys_SQWareCentral GenLstInstanceNew
Verify that this is indeed the list you want to deploy, then choose option 1 ...
=> Enter the Unix password when prompted !
Deployment of SSH key(s): manual method
1 # From : oracle@my_oracle_host
2 if [ ! -r $HOME/.ssh/id_rsa ]
3 then
4 ssh-keygen -t rsa -N ''''''' -f $HOME/.ssh/id_rsa
5 fi
6 chmod go-w $HOME
7 echo "ssh-rsa AAAAB......XSPpdV11 dbsqware@sqwarebox" >>$HOME/.ssh/authorized_keys
8 chmod 700 $HOME/.ssh
9 chmod 600 $HOME/.ssh/authorized_keys
=> Put the correct key in the "echo" (the one from dbsqware@sqwarebox)
SSH connection test from SQWareCentral
1 # From : dbsqware@sqwarebox
2 menu_ora TestSshConnection GenLstInstanceNew
Test of prerequisites on the target host (bash+rsync) via SSH from SQWareCentral
1 # From : dbsqware@sqwarebox
2 menu_ora TestSysPrerequisites GenLstInstanceNew
Deployment of SQWareProduction
1 # From : dbsqware@sqwarebox
2
3 #If deployment on "unique" user
4 menu_ora DeplScripts GenLstUniqueNew
5
6 #If deployment on "instance" user, the one specified in SQWareRepository
7 menu_ora DeplScripts GenLstInstanceNew
For "unique" user, it is the "standard" user. By default, we set it to "oracle", and it can be modified in SQWareCentral.
Adding the dbSQWare environment to ~/.profile or ~/.bash_profile
Once again, there are two ways to proceed:
- An automatic one (from dbsqware@sqwarebox)
- A manual one that you apply on each oracle Unix account
Adding the dbSQWare environment: batch method
1 # From : dbsqware@sqwarebox
2 menu_ora AdddbSQWareProfile GenLstInstanceNew
Check that this is indeed the list you want to deploy, then choose 1 ...
=> In ~/.profile or ~/.bash_profile, change the following variable with the appropriate value: gvsqw_Env='XXX'
Adding the dbSQWare environment: manual method
Add the following lines to ~/.profile or ~/.bash_profile
1 # From : oracle@my_oracle_host
2
3 #dbSQWare
4 export gvsqw_OraBin=$HOME/SQWareProduction/oracle/bin
5 export gvsqw_Env='PRD'
6 lvsqw_IsTerminal=$(tty 2>&1 >/dev/null;echo $?)
7 if [ "$lvsqw_IsTerminal" = "0" ] && [ -r $gvsqw_OraBin/../etc/.profile_confort ]
8 then
9 . $gvsqw_OraBin/../etc/.profile_confort
10 fi
Test sendmail (Non mandatory)
1 # From : dbsqware@sqwarebox
2 menu_ora TestSendmail GenLstInstanceNew
Manual connection tests to the Oracle instance
The goal is to test the automatic connection methods to the Oracle instance.
- "local", for instances that will be in "ON" status
- "remote", to generate AWR (possibly) or for cases where there is no SSH access to the machine (Windows, RDS, etc...)
Test connection "local"
/!\ Only instances that will be in "ON" status!
1 # From : dbsqware@sqwarebox
2 menu_ora TestInstConnectionOnNoMail GenLstInstanceNew
Procédure spécifique pour la partie "distante" (statut DIST)
Création d'un compte MySQL/MariaDB spécifique dbSQWare (Pour connexion distante)
Exemple pour une connexion "distante"
1 grant all privileges on *.* to 'DBSDBA'@'%' IDENTIFIED BY 'NePasMettreCePassword!' with grant option;
2
3 /* ou */
4 create user 'DBSDBA'@'%' IDENTIFIED BY 'NePasMettreCePassword!';
5 grant all privileges on *.* to 'DBSDBA'@'%' with grant option;
6 flush privileges;
Vous pouvez mettre le nom et le mot de passe que vous souhaitez !
Ce compte devra être déclaré sur le compte unix mysql@sqwarebox ...
1 vi /home/mysql/.passwd_mysql
2 DbAlias;UserName;Passwd;HostName;Port
3 MYS_APPLI_PRD;DBSDBA;NePasMettreCePassword!;srvdbmys01;3306
Test connexion "distante"
1 # From : dbsqware@sqwarebox
2 menu_mys TestInstConnectionDistNoMail GenLstInstanceNew
Mise en exploitation de l'instance
Génération de la configuration par défaut SQWareProduction
/!\ Uniquement que les instances qui seront en statut "ON" !
1 # From : dbsqware@sqwarebox
2 menu_mys GenDefConf_SQWareProduction GenLstInstanceNew
Mise à jour de SQWareRepository
Dans le repository dbSQWare via la console d’administration, mettre à jour le statut de la nouvelle instance MySQL/MariaDB (à NEW actuellement) avec la valeur souhaitée (ON, DIST)
Test de "reprise" des indicateurs
A ce moment là, vous pouvez tester la reprise des indicateurs manquants (en principe, uniquement les instances que vous venez d'intégrer !)
1 # From : dbsqware@sqwarebox
2 ## Pour les statuts "ON"
3 repind_mys
4 ## Pour les statuts "DIST"
5 repind_mys_dist
Exploitation (stats, backups, …) pour les instance "ON"
Génération des conf SQWareProduction
Si vous avez suivi la procédure d’insertion de l’instance, cette partie est déjà fait par l’étape "menu_mys GenDefConf_SQWareProduction GenLstInstanceNew".
Vous pouvez vérifier avec le chapitre suivant.
Se connecter à la machine qui porte l'instance « MySQL/MariaDB » et taper les commandes suivantes :
1 # From dbsqware@vmyswarebox
2 c MYS_INST_PRD
1 #Source de l'env si multi-instance (from mysql@my_mysql_host)
2 e MYS_INST_PRD
1 too
2 ./sqwmys_GenerateCreateInstance.ksh -dbsOnly
Vérifier que ce qui est proposé est bon et tapez « y » si c’est le cas.
A ce moment, vous avez généré les fichiers de configuration par défaut de SQWareProduction.
Explications
Les étapes précédentes ont créé certains fichiers par défaut. ($HOME/MYS_INST_PRD/sqwConfig)
1 #Entrez le nom de l’instance
2 e MYS_INST_PRD
3 cfg
4 cat sqwmys_Jobs.cfg
1 #############################################
2 #IndicDba: Gather DBAs indicators
3 IndicDba:$gvsqw_MysBin/sqwmys_GatherIndicators.ksh -I MYS_INST_PRD
4
5 #Dump: backup all databases with mysqlDump
6 Dump:$gvsqw_MysBin/sqwmys_DumpAllDatabases.ksh -I MYS_INST_PRD -Exec
7
8 #Optimize : optimize tables for all databases
9 Optimize:$gvsqw_MysBin/sqwmys_OptimizeAllDatabases.ksh -I MYS_INST_PRD -Exec
10
11 #Analyse : analyze tables for all databases
12 Analyze:$gvsqw_MysBin/sqwmys_AnalyzeAllDatabases.ksh -I MYS_INST_PRD -Exec
13
14 #XtraFull: Backup full with XtraBackup
15 XtraFull:$gvsqw_MysBin/sqwmys_XtraBackup.ksh -I MYS_INST_PRD -Type full -Exec
16
17 #XtraInc: Backup incremental with XtraBackup
18 XtraInc:$gvsqw_MysBin/sqwmys_XtraBackup.ksh -I MYS_INST_PRD -Type inc -Exec
19
20 #MariaFull: Backup full with MariaBackup
21 MariaFull:$gvsqw_MysBin/sqwmys_MariaBackup.ksh -I MYS_INST_PRD -Type full -Exec
22
23 #MariaInc: Backup incremental with MariaBackup
24 MariaInc:$gvsqw_MysBin/sqwmys_MariaBackup.ksh -I MYS_INST_PRD -Type inc -Exec
25
26 #PurgeAlert: Rotate error log
27 PurgeAlert:$gvsqw_MysBin/sqwmys_PurgeAlert.ksh -I MYS_INST_PRD
C’est un fichier de paramétrage qui est utilisé par le script sqwmys_RunJob.ksh pour exécuter des actions.
Fichier pour une cron par défaut $HOME/CrontabRef_MYS_INST_PRD :
1 ## mm(0-59) hh(0-23) dd(1-31) MM(0-12) DAY(0-sunday, 1-monday, ...) command
2 ## MYS_INST_PRD
3 #Purge alert
4 00 08 * * 0 bash -c '. $HOME/.profile MYS_INST_PRD;$gvsqw_MysBin/sqwmys_RunJob.ksh -I MYS_INST_PRD -A PurgeAlert > $HOME/tmp/PurgeAlert_MYS_INST_PRD.log 2>&1'
5 # Dump
6 00 20 * * * bash -c '. $HOME/.profile MYS_INST_PRD;$gvsqw_MysBin/sqwmys_RunJob.ksh -I MYS_INST_PRD -A Dump > $HOME/tmp/Dump_MYS_INST_PRD.log 2>&1'
7 # Optimize
8 00 04 * * 6 bash -c '. $HOME/.profile MYS_INST_PRD;$gvsqw_MysBin/sqwmys_RunJob.ksh -I MYS_INST_PRD -A Optimize > $HOME/tmp/Optimize_MYS_INST_PRD.log 2>&1'
9 # Analyze
10 00 05 * * 6 bash -c '. $HOME/.profile MYS_INST_PRD;$gvsqw_MysBin/sqwmys_RunJob.ksh -I MYS_INST_PRD -A Analyze > $HOME/tmp/Analyze_MYS_INST_PRD.log 2>&1'
11 # XtraBackup
12 #00 21 * * 6 bash -c '. $HOME/.profile MYS_INST_PRD;$gvsqw_MysBin/sqwmys_RunJob.ksh -I MYS_INST_PRD -A XtraFull > $HOME/tmp/XtraFull_MYS_INST_PRD.log 2>&1'
13 #00 21 * * 1-5 bash -c '. $HOME/.profile MYS_INST_PRD;$gvsqw_MysBin/sqwmys_RunJob.ksh -I MYS_INST_PRD -A XtraInc > $HOME/tmp/XtraInc_MYS_INST_PRD.log 2>&1'
14 # MariaBackup
15 #00 21 * * 6 bash -c '. $HOME/.profile MYS_INST_PRD;$gvsqw_MysBin/sqwmys_RunJob.ksh -I MYS_INST_PRD -A MariaFull > $HOME/tmp/MariaFull_MYS_INST_PRD.log 2>&1'
16 #00 21 * * 1-5 bash -c '. $HOME/.profile MYS_INST_PRD;$gvsqw_MysBin/sqwmys_RunJob.ksh -I MYS_INST_PRD -A MariaInc > $HOME/tmp/MariaInc_MYS_INST_PRD.log 2>&1'
Mise en exploitation
Mise en place de la cron :
1 mv $HOME/CrontabRef_MYS_INST_PRD $HOME/CrontabRef
2 vi $HOME/CrontabRef
3 crontab $HOME/CrontabRef
4 crontab -l
Mise à jour du fichier CrontabRef :
1 crontab -l > $HOME/CrontabRef
2 cat $HOME/CrontabRef
Commandes utiles
alias intéressants :
e DBALIAS => source l'env pour l'instance bin, etc, too ... => pour aller dans arbo SQWareProduction standard cbin, cetc, ctoo ... => pour aller dans arbo SQWareProduction custom log => logs SQWareProduction dmp => on va dans les chemins de backup
Quelques options des scripts :
-h => aide en ligne -s => exemples en ligne -Exec => lorsque elle est présente, si non settée ne sort que les commandes sans les exécuter
Exemples :
mysql@my_mysql_host:$HOME/admin/MYS_INST_PRD/logs/DumpAllDatabases (MYS_DBA_PRD) $ bin total 100 drwxr-x--- 2 dbsqware dba 4096 Sep 6 14:09 . drwxr-x--- 15 dbsqware dba 187 Aug 16 15:50 .. -rwxr-x--- 1 dbsqware dba 2957 Aug 16 15:50 sqwmys_AnalyzeAllDatabases.ksh -rwxr-x--- 1 dbsqware dba 2514 Aug 16 15:50 sqwmys_AnalyzeDatabase.ksh -rwxr-x--- 1 dbsqware dba 2608 Aug 16 15:50 sqwmys_CheckAllDatabases.ksh -rwxr-x--- 1 dbsqware dba 2257 Aug 16 15:50 sqwmys_CheckDatabase.ksh -rwxr-x--- 1 dbsqware dba 5418 Aug 16 15:50 sqwmys_CopyDumpAllDatabases.ksh -rwxr-x--- 1 dbsqware dba 5324 Aug 16 15:50 sqwmys_CopyDumpDatabase.ksh -rwxr-x--- 1 dbsqware dba 5610 Sep 6 10:45 sqwmys_CopyMariaBackup.ksh -rwxr-x--- 1 dbsqware dba 5594 Sep 6 14:08 sqwmys_CopyXtraBackup.ksh -rwxr-x--- 1 dbsqware dba 3248 Aug 16 15:50 sqwmys_DumpAllDatabases.ksh -rwxr-x--- 1 dbsqware dba 3541 Aug 16 15:50 sqwmys_DumpDatabase.ksh lrwxrwxrwx 1 dbsqware dba 36 Jan 2 2022 sqwmys_ExecSQL.ksh -> ../../generic/bin/sqwgen_ExecSQL.ksh -rwxr-x--- 1 dbsqware dba 2455 Aug 16 15:50 sqwmys_GatherIndicators.ksh -rwxr-x--- 1 dbsqware dba 2454 Aug 16 15:50 sqwmys_GatherStructure.ksh -rwxr-x--- 1 dbsqware dba 3043 Aug 16 15:50 sqwmys_MariaBackup.ksh lrwxrwxrwx 1 dbsqware dba 38 Jan 2 2022 sqwmys_NetBackup.ksh -> ../../generic/bin/sqwgen_NetBackup.ksh -rwxr-x--- 1 dbsqware dba 2959 Aug 16 15:50 sqwmys_OptimizeAllDatabases.ksh -rwxr-x--- 1 dbsqware dba 2522 Aug 16 15:50 sqwmys_OptimizeDatabase.ksh lrwxrwxrwx 1 dbsqware dba 40 Jan 2 2022 sqwmys_ParallelRun.ksh -> ../../generic/bin/sqwgen_ParallelRun.ksh -rwxr-x--- 1 dbsqware dba 2252 Aug 16 15:50 sqwmys_PurgeAlert.ksh -rwxr-x--- 1 dbsqware dba 3855 Aug 16 15:50 sqwmys_RestoreAllDatabases.ksh -rwxr-x--- 1 dbsqware dba 3463 Aug 16 15:50 sqwmys_RestoreDatabase.ksh -rwxr-x--- 1 dbsqware dba 3825 Aug 16 15:50 sqwmys_RotateLogSlowQuery.ksh lrwxrwxrwx 1 dbsqware dba 40 Jan 2 2022 sqwmys_RsyncBackup.ksh -> ../../generic/bin/sqwgen_RsyncBackup.ksh lrwxrwxrwx 1 dbsqware dba 35 Jan 2 2022 sqwmys_RunJob.ksh -> ../../generic/bin/sqwgen_RunJob.ksh -rwxr-x--- 1 dbsqware dba 3025 Aug 16 15:50 sqwmys_XtraBackup.ksh
mysql@my_mysql_host:$HOME/SQWareProduction/mysql/bin (MYS_INST_PRD) $
./sqwmys_DumpAllDatabases.ksh -h
Sourcing sqwmys_Global.lib v2023.06 SQWareProduction for MySQL (dbSQWare) ...
Usage: sqwmys_DumpAllDatabases.ksh [-h] -I instance [+ options]
DESCRIPTION
sqwmys_DumpAllDatabases.ksh dump databases in parallel
SUPPORT
MySql supported versions: 5.0 <= v <= 8
MariaDB supported versions: 5.5 <= v <= 11.6
PARAMETERS
-I instance : Target instance to mysqldump.
OPTIONS
-h help : Display the full usage.
-s : Display samples of usage.
-P Nb threads : Number of threads in parallel (by default 2).
-RD directory : Directory to write dump file (default /backups/mysql/$MYSQL_SID).
-ID listdb : List of databases to be dumped (by default all).
-ED listdb : List of databases to exclude from dump.
-IL likeclause : Like clause to generate databases list (example: 'db%').
-EL likeclause : Like clause to exclude databases (example: 'nodb%').
-HI Nb_Generations : Number of generations to keep (by default 2).
-WCD whereclause : Where clause to generate databases list.
-AGR Nb sec : Nb of seconds between two runs (by default 1s).
-AGE Nb sec : Nb of seconds between two checks of end (by default 10s).
-FRT return_code : Force return code value on error.
-Opt option : Additional options for mysqldump.
-Dist : For distant connection to database (change $gvsqw_ConnectString to $gvsqw_DistConnectString ).
-AddMail email : Email address to add at 'xxx'.
-SendReport : Send execution log report.
-NoMail : Deactivate sendmail on error xxx (by default, send on error).
-Locale locale : Force Locale for help display (fr,en).
-Consistent : Lock for consistent mode (by default, no lock).
-Exec : Execute commands (default, display generated commands)
Enjoy !
