Difference between revisions of "FAQ:NewInstanceMongoDB"
Dtavernier (talk | contribs) (→Tests de connexion "manuels" à l'instance MongoDB) |
Dtavernier (talk | contribs) (→Manual connection test to the Oracle instance) |
||
| Line 255: | Line 255: | ||
</syntaxhighlight> | </syntaxhighlight> | ||
| − | ==== | + | ====Test de connexion "manuel" à l'instance MongoDB (connexion "locale")==== |
The goal is to test the automatic connection methods to the MongoDB instance. | The goal is to test the automatic connection methods to the MongoDB instance. | ||
=====Test connection "local"===== | =====Test connection "local"===== | ||
Revision as of 14:57, 30 April 2025
Work In Progress
Contents
- 1 General Information
- 2 Integration of a mongodb Instance
- 2.1 Setting Up the Environment
- 2.2 Specific procedure for the "local" part (ON status)
- 2.2.1 Checking the system requirements for the "local" part (bash + rsync)
- 2.2.2 Deploying SSH Key(s) from SQWareCentral to the Target Machine(s)
- 2.2.3 Testing the ssh connection from SQWareCentral
- 2.2.4 Testing prerequisites on the target host (bash+rsync) ssh from SQWareCentral
- 2.2.5 Deploying SQWareProduction
- 2.2.6 Adding the dbSQWare environment to ~/.profile or ~/.bash_profile
- 2.2.7 Test sendmail (Non mandatory)
- 2.2.8 Test de connexion "manuel" à l'instance MongoDB (connexion "locale")
- 2.3 Specific procedure for the "remote" part (DIST status)
- 3 Starting the Instance
General Information
In this chapter, we'll assume that SQWareCentral has been installed on dbsqware@sqwarebox...
Limitations of this section
This chapter only deals with adding a new instance to an already configured and functional base.
Use the sections that apply to your environment and use the match or manual method, as preferred!
General MODOP for integrating a new mongodb instance
There will potentially be one or two connection cases to handle:
- "local", for Unix machines (or with Cygwin), where you connect to the machine hosting the instance (ssh connection from dbsqware@sqwarebox)
- "remote", for the case where you has access to the machine via ssh (Windows, RDS, etc.)
Typical steps:
- Setting up the environment
- Checking system requirements for the "local" part (bash + rsync)
- Manual testing
- Starting the instance
You'll see that these steps can be completed one by one or in batches!
Integration of a mongodb Instance
Setting Up the Environment
General
Explanation of "DbAlias" (the unique dbSQWare key)
The dbSQWare key must be unique and consistent with what has been configured for the gvsqw_DbAlias variable in sqwsyb_GlobalVar.cfg!
There are roughly two cases...
- Your DSQUERYs are unique across the entire database => gvsqw_DbAlias='$DSQUERY'
- You have "duplicate" DSQUERYs => gvsqw_DbAlias='$DSQUERY:$(hostname|cut -d '.' -f1)'
+ any other cases that may arise or that "suit you" !
Status Explanation
- A "classic" instance will be processed in "local" mode, meaning that the user will connect locally to the machine hosting the instance (SSH access from dbsqware@sqwarebox), status "ON".
- An instance without local UNIX access will be processed in "remote" mode (Windows style, RDS, etc.), status "DIST".
- The "OFF" status makes the instance "disappear" from the web view.
- The "NEW" status is used as an intermediate status between the start and end of the configuration of a new instance.
- All other statuses allow the instance to be displayed in the repository (notably "KEEP") but will not be taken into account by automatic processing.
Status Summary:
| Status | Description |
| ON | Local management under UNIX (most common) |
| DIST | Remote management only |
| KEEP | Temporary status, for example, during maintenance or before complete deletion |
| NEW | Temporary status, during configuration |
| OFF | Visual deletion of the instance (without deleting its configuration) |
| XXX | Not supported... |
Declaring the instance in SQWareRepository with SQWareWeb
We will declare the new instance from the SQWareWeb administration interface.
Declare the new instance in "NEW" status (Add or Duplicate)
Explanation of fields:
| Field | Description |
| Db Alias | Unique key that identifies the instance in dbSQWare |
| Dbms Name | DBMS Type |
| Virtual Host | Virtual Host (same as Host if not clustered, will be used to generate tnsname.ora) |
| Host Name | Hostname of the instance |
| User Name | User that hosts the instance |
| Port | Listening port of the instance (will be used to generate tnsname.ora) |
| Comments | A short description of what the instance hosts |
| Status | Instance status (see above for explanations) |
| Contact | A contact if needed |
| Environment | Instance environment (PRD, PPR, REC, DEV, TST, etc.) |
| Client | Used only for filtering (enter a client name and/or department and/or service name, etc.) |
| GlobalHost | Free field, often used to enter the hypervisor host, for example. |
| Custom1 | Free field 1 |
| Custom2 | Free field 2 |
Regenerate SQWareCentral Reference Files
Type the following command to generate the reference files:
1 # From : dbsqware@sqwarebox
2 gen_all
Specific procedure for the "local" part (ON status)
/!\ Only instances that will be in "ON" status!
Checking the system requirements for the "local" part (bash + rsync)
Since SQWareProduction is written primarily in the bash shell and is synchronized from SQWareCentral with rsync, we need "bash" and "rsync" installed!
Verification
Adapt to your username and machine name.
bash:
1 # From : mongodb@mongodbhost
2 type bash
3 #or
4 which bash
rsync:
1 # mongodb@mmongodbhost
2 type rsync
3 #or
4 which rsync
Installation
Adapt to your machine type (sudo if you're not root).
RedHat / CentOS / ... :
1 # From : root@my_syb_host
2 yum install -y bash rsync
3 #or
4 dnf install -y bash rsync
Ubuntu / Debian / ... :
1 # From : root@my_syb_host
2 apt install -y bash rsync
Deploying SSH Key(s) from SQWareCentral to the Target Machine(s)
There are two ways to do this:
- Automatic (from dbsqware@sqwarebox, but this requires you to know the password for your MongoDB Unix account)
- Manual, which you apply to each mongodb Unix account
/!\ Your MongoDB Unix account password must have been initialized; otherwise, the key exchange connection will not work!
Deploying SSH Key(s): Batch Method
1 # From : dbsqware@sqwarebox
2 menu_syb GenDeplSshKeys_SQWareCentral GenLstInstanceNew
Check that this is the list you want to deploy, then choose 1...
=> Enter your UNIX password when prompted!
Deploying SSH Key(s): Manual Method
1 # From : MongoDB@my_sybcle_host
2 if [ ! -r $HOME/.ssh/id_rsa ]
3 then
4 ssh-keygen -t rsa -N ''''''' -f $HOME/.ssh/id_rsa
5 fi
6 chmod go-w $HOME
7 echo "ssh-rsa AAAAB......XSPpdV11 dbsqware@sqwarebox" >>$HOME/.ssh/authorized_keys
8 chmod 700 $HOME/.ssh
9 chmod 600 $HOME/.ssh/authorized_keys
=> Enter the correct key in the "echo" field (the one for dbsqware@sqwarebox)
Testing the ssh connection from SQWareCentral
1 # From : dbsqware@sqwarebox
2 menu_syb TestSshConnection GenLstInstanceNew
Testing prerequisites on the target host (bash+rsync) ssh from SQWareCentral
1 # From : dbsqware@sqwarebox
2 menu_syb TestSysPrerequisites GenLstInstanceNew
Deploying SQWareProduction
1 # From : dbsqware@sqwarebox
2
3 #If deployment on "unique" user
4 menu_syb DeplScripts GenLstUniqueNew
5
6 #If deployment on "instance" user, the one specified in SQWareRepository
7 menu_syb DeplScripts GenLstInstanceNew
Adding the dbSQWare environment to ~/.profile or ~/.bash_profile
Once again, there are two ways to do this:
- Automatic (from dbsqware@sqwarebox)
- Manual, which you apply to each MongoDB Unix account
Adding the dbSQWare environment: Batch method
1 # From : dbsqware@sqwarebox
2 menu_syb AdddbSQWareProfile GenLstInstanceNew
Verify that this is the list you want to deploy, then select 1...
=> In ~/.profile or ~/.bash_profile, change the following variable to the appropriate value: gvsqw_Env='XXX'
Adding the dbSQWare environment: manual method
Add the following lines to ~/.profile or ~/.bash_profile
1 # From : MongoDB@my_syb_host
2
3 #dbSQWare
4 export gvsqw_SybBin=$HOME/SQWareProduction/MongoDB/bin
5 export gvsqw_Env='PRD'
6 lvsqw_IsTerminal=$(tty 2>&1 >/dev/null;echo $?)
7 if [ "$lvsqw_IsTerminal" = "0" ] && [ -r $gvsqw_SybBin/../etc/.profile_confort ]
8 then
9 . $gvsqw_SybBin/../etc/.profile_confort
10 fi
Test sendmail (Non mandatory)
1 # From : dbsqware@sqwarebox
2 menu_syb TestSendmail GenLstInstanceNew
Test de connexion "manuel" à l'instance MongoDB (connexion "locale")
The goal is to test the automatic connection methods to the MongoDB instance.
Test connection "local"
/!\ Only instances that will be in "ON" status!
1 # From : dbsqware@sqwarebox
2 menu_ora TestInstConnectionOnNoMail GenLstInstanceNew
Specific procedure for the "remote" part (DIST status)
Creating a specific MongoDB dbSQWare account
/!\ Only instances that will be in "DIST" status!
Example of creation order:
1 mongo
You can enter any username and password you like!
This account must be registered under the Unix account MongoDB@sqwarebox...
1 MongoDB@sqwarebox:/home/Mongodb () $ cat .passwd_MongoDB
2 DbAlias;UserName;Passwd;HostName;Port
3 MNG_MONAPPLI_PRD;dbsdba;myPassword;srvdbmng01;9999
Test "remote" connection
1 # From : dbsqware@sqwarebox
2 menu_syb TestInstConnectionDistNoMail GenLstInstanceNew
Starting the Instance
Generating the Default SQWareProduction Configuration
/!\ Only instances that will be in "ON" status!
1 # From : dbsqware@sqwarebox
2 menu_syb GenDefConf_SQWareProduction GenLstInstanceNew
Update SQWare Repository
In the dbSQWare repository via the administration console, update the status of the new MongoDB instance (currently NEW) to the desired value (ON, DG, DIST).
Testing the "Recovery" of Indicators
At this point, you can test the recovery of the missing indicators (in principle, only the instances you have just integrated!)
1 # From : dbsqware@sqwarebox
2 ## For "ON" status
3 repind_mng
4 ## For "DIST" status
5 repind_mng_dist
Operation (stats, backups, etc.) for "ON" instances
Generating SQWareProduction conf
If you followed the instance insertion procedure, this part is already done by the step "menu_syb GenDefConf_SQWareProduction GenLstInstanceNew".
You can check with the next chapter.
Log in to the machine hosting the "MongoDB" instance and type the following commands:
1 # From dbsqware@vsybwarebox
2 c MNG_INST_PRD
1 #Sourcing environment if multi-instance (from MongoDB@my_syb_host)
2 e MNG_INST_PRD
1 too
2 ./sqwsyb_GenerateCreateInstance.ksh -dbsOnly
Check that the proposed settings are correct and type "y" if so.
At this point, you have generated the default configuration files for SQWareProduction.
Explanations
The previous steps created some default files. ($HOME/SYBDBS/sqwConfig)
1 #Enter the instance name
2 e SYBDBS
3 cfg
4 cat sqwsyb_Jobs.cfg
1 #IndicDba: Gather indicators DBAs
2 IndicDba:$gvsqw_SybBin/sqwsyb_GatherIndicators.ksh -S SYBDBS
3
4 #DumpFull: Dump full all databases
5 DumpFull:$gvsqw_SybBin/sqwsyb_DumpAllDatabases.ksh -S SYBDBS -Type full -Exec
6
7 #DumpInc: Dump tran
8 DumpInc:$gvsqw_SybBin/sqwsyb_DumpAllDatabases.ksh -S SYBDBS -ID mydblist -Type inc -Exec
9
10 #Checkstorage: Checkstorage all databases
11 Checkstorage:$gvsqw_SybBin/sqwsyb_CheckstorageAllDatabases.ksh -S SYBDBS -Exec
12
13 #Reorg: Reorg all databases
14 Reorg:$gvsqw_SybBin/sqwsyb_RebuildAllDatabases.ksh -S SYBDBS -Exec
15
16 #UpdateStats: Update statistics all databases
17 UpdateStats:$gvsqw_SybBin/sqwsyb_StatisticsAllDatabases.ksh -S SYBDBS -Exec
18
19 #Dbcc: Dbcc all databases
20 Dbcc:$gvsqw_SybBin/sqwsyb_DbccAllDatabases.ksh -S SYBDBS -Exec
This is a configuration file used by the sqwsyb_RunJob.ksh script to execute actions.
File for a default cron job $HOME/CrontabRef_SYBDBS:
1 ## mm(0-59) hh(0-23) dd(1-31) MM(0-12) DAY(0-sunday, 1-monday, ...) command
2 ## dump full
3 00 17 * * 1-6 bash -c '. $HOME/.profile SYBDBS;$gvsqw_SybBin/sqwsyb_RunJob.ksh -S SYBDBS -A DumpFull 2>&1 >~/tmp/DumpFull_SYBDBS.log'
4 ## Checkstorage
5 00 12 * * 6 bash -c '. $HOME/.profile SYBDBS;$gvsqw_SybBin/sqwsyb_RunJob.ksh -S SYBDBS -A Checkstorage 2>&1 >~/tmp/Checkstorage_SYBDBS.log'
6 ## Reorg
7 00 04 * * 6 bash -c '. $HOME/.profile SYBDBS;$gvsqw_SybBin/sqwsyb_RunJob.ksh -S SYBDBS -A Reorg 2>&1 >~/tmp/Reorg_SYBDBS.log'
8 ## Update statistics
9 00 16 * * 2 bash -c '. $HOME/.profile SYBDBS;$gvsqw_SybBin/sqwsyb_RunJob.ksh -S SYBDBS -A UpdateStats 2>&1 >~/tmp/UpdateStats_SYBDBS.log'
Startup
Setting up cron:
1 mv $HOME/CrontabRef_SYBDBS $HOME/CrontabRef
2 vi $HOME/CrontabRef
3 crontab $HOME/CrontabRef
4 crontab -l
Update CrontabRef file:
1 crontab -l > $HOME/CrontabRef
2 cat $HOME/CrontabRef
Useful commands
interesting aliases:
e DBALIAS => source env for the instance bin, etc, too ... => to go to the standard SQWareProduction tree cbin, cetc, ctoo ... => to go to the custom SQWareProduction tree log => SQWareProduction logs dmp => go to the backup paths
Some script options:
-h => online help -s => online examples -Exec => when present, if not set, only outputs the commands without executing them
Examples:
MongoDB@my_syb_host:$HOME/admin/SYBDBS/logs/DumpAllDatabases (SYBDBS) $ bin total 112 drwxr-x--- 2 sybase dba 4096 Aug 16 15:50 . drwxr-x--- 15 sybase dba 4096 Aug 16 15:50 .. -rwxr-x--- 1 sybase dba 3043 Aug 16 15:50 sqwsyb_CheckstorageAllDatabases.ksh -rwxr-x--- 1 sybase dba 2616 Aug 16 15:50 sqwsyb_CheckstorageDatabase.ksh -rwxr-x--- 1 sybase dba 4507 Aug 16 15:50 sqwsyb_CopyDumpAllDatabases.ksh -rwxr-x--- 1 sybase dba 3369 Sep 5 11:33 sqwsyb_CopyDumpDatabase.ksh -rwxr-x--- 1 sybase dba 2988 Aug 16 15:50 sqwsyb_DbccAllDatabases.ksh -rwxr-x--- 1 sybase dba 2616 Aug 16 15:50 sqwsyb_DbccDatabase.ksh -rwxr-x--- 1 sybase dba 4213 Aug 16 15:50 sqwsyb_DumpAllDatabases.ksh -rwxr-x--- 1 sybase dba 3575 Aug 16 15:50 sqwsyb_DumpDatabase.ksh lrwxrwxrwx 1 sybase dba 36 Jul 22 2018 sqwsyb_ExecSQL.ksh -> ../../generic/bin/sqwgen_ExecSQL.ksh -rwxr-x--- 1 sybase dba 2742 Aug 16 15:50 sqwsyb_GatherDistantFsSize.ksh -rwxr-x--- 1 sybase dba 2488 Aug 16 15:50 sqwsyb_GatherIndicators.ksh -rwxr-x--- 1 sybase dba 2353 Aug 16 15:50 sqwsyb_GatherStructure.ksh -rwxr-x--- 1 sybase dba 2464 Aug 16 15:50 sqwsyb_KillSessions.ksh -rwxr-x--- 1 sybase dba 4101 Aug 16 15:50 sqwsyb_LoadAllDatabases.ksh -rwxr-x--- 1 sybase dba 4685 Aug 16 15:50 sqwsyb_LoadDatabase.ksh lrwxrwxrwx 1 sybase dba 38 Dec 6 2015 sqwsyb_NetBackup.ksh -> ../../generic/bin/sqwgen_NetBackup.ksh lrwxrwxrwx 1 sybase dba 40 Dec 6 2015 sqwsyb_ParallelRun.ksh -> ../../generic/bin/sqwgen_ParallelRun.ksh -rwxr-x--- 1 sybase dba 3351 Aug 16 15:50 sqwsyb_RebuildAllDatabases.ksh -rwxr-x--- 1 sybase dba 2955 Aug 16 15:50 sqwsyb_RebuildDatabase.ksh lrwxrwxrwx 1 sybase dba 40 Dec 20 2017 sqwsyb_RsyncBackup.ksh -> ../../generic/bin/sqwgen_RsyncBackup.ksh lrwxrwxrwx 1 sybase dba 35 Dec 6 2015 sqwsyb_RunJob.ksh -> ../../generic/bin/sqwgen_RunJob.ksh -rwxr-x--- 1 sybase dba 3112 Aug 16 15:50 sqwsyb_StatisticsAllDatabases.ksh -rwxr-x--- 1 sybase dba 2707 Aug 16 15:50 sqwsyb_StatisticsDatabase.ksh -rwxr-x--- 1 sybase dba 3798 Aug 16 15:50 sqwsyb_UpdtStatsAllDatabases.ksh -rwxr-x--- 1 sybase dba 3691 Aug 16 15:50 sqwsyb_UpdtStatsDatabase.ksh -rwxr-x--- 1 sybase dba 3966 Aug 16 15:50 sqwsyb_UpdtStatsParallelDatabase.ksh -rwxr-x--- 1 sybase dba 3366 Aug 16 15:50 sqwsyb_UpdtStatsRebuildWithFlag.ksh
sybase@my_syb_host:$HOME/SQWareProduction/sybase/bin (SYBDBS) $
./sqwsyb_DumpAllDatabases.ksh -h
Sourcing sqwsyb_Global.lib v2023.06 SQWareProduction for Sybase (dbSQWare) ...
Usage: sqwsyb_DumpAllDatabases.ksh [-h] -I <dataserver> [+ options]
DESCRIPTION
sqwsyb_DumpAllDatabases.ksh Dump all databases in parallel
SUPPORT
Sybase supported versions: 12.5 <= v <= 16.0
PARAMETERS
-I dataserver : Target dataserver to Dump.
OPTIONS
-P Nb threads : Number of threads in parallel (by default 2).
-ID listdb : List of databases holding tables to analyze (by default all).
-ED listdb : List of databases to exclude.
-IL likeclause : Like clause to generate databases list (example: 'db%').
-EL likeclause : Like clause to exclude databases (example: 'nodb%').
-WCD whereclause : Where clause to generate databases list.
-FRT return_code : Force return code value on error.
-RD directory : Directory to write dump file (default /syb_dump/$DSQUERY).
-HI Nb : Nb of generations to keep (by default 2).
-CL Level : Compression level (by default 1).
-CR Rate : Estimated rate of compression (by default 22).
-AGR Nb sec : Nb of seconds between two runs (by default 10s).
-AGE Nb sec : Nb of seconds between two checks of end (by default 30s).
-SMS SizeMax : Size Max Stripe (default 4096 Mo).
-Type type : Dump type full,inc,stb,fstb (default full).
-NoMail : Deactivate sendmail on error xxx (by default, send on error).
-OldSyntax : Force dump old syntax (by default, 'NewSyntax').
-NewSyntax : Force dump new syntax (by default, 'NewSyntax').
-NoSyslogins : Do not backup syslogins tables.
-AddMail email : Email address to add at 'xxx'.
-SendReport : Send execution log report.
-AddFlagName mflag : Put '.mflag' at end of dump name.
-CrossPlatform : Put db in 'single user' mode and execute sp_flushstats before Dump.
-TryCum : Try 'dump cumulative' if 'dump tran' error (need to customize lfsqw_TryCum function).
-h help : Display the full usage.
-s : Display samples of usage.
-Locale locale : Force Locale for help display (fr,en).
-Exec : Execute commands (default, display generated commands)
Enjoy !
