Oracle Database 10g TM Automatic Storage Management NEC istorage S An Oracle and NEC White Paper August 2005 2006/04-1 -
Copyright 2005 NEC and Oracle Corporation. All rights reserved. Oracle, Oracle Database 10g Linux Linus Torvalds / Oracle Corporation /NEC Using Oracle Database 10gAutomatic Storage Management with NEC Storage S series products Rev. 1.00 August 2005 Rev. 2.00 April 2006-2 -
1.... 4 2. istorage... 4 2.1 (RAID-6)...4 2.2...5 2.3 SystemGlobe StoragePathSavior (SPS)...6 2.4 DynamicDataReplication...7 3. Oracle... 7 3.1...7 3.1.2 ASM...7 3.1.3...8 3.1.4 ASM...8 4. ASM... 9 4.1 SPS...9 4.2 RAW...9 4.3 RAID ASM... 11 5. istorage ASM... 11 5.1 RAID6... 11 5.2 SystemGlobe StoragePathSavior...12 5.3 LUN...13 5.4...13 5.4.1...13 6.... 15 7.... 15 7.1...15 7.2 DynamicDataReplication...18-3 -
1. Oracle Database 10g Automatic Storage Management( ASM NEC istorage S ASM Oracle Database 10g ASM TCO ASM ASM ASM ASM Raw Device ASM RAID ASM Oracle Database 10g NEC Storage S -6 ASM ASM istorage 2. istorage 2.1 (RAID-6) NEC Storage S -6 NEC HDD - 4 -
Two HDD Fault Tolerant Technology RAID-6 : (P, Q) istorage S4900/S2800/S2400/S1800AT/S1400/S400 RAID RAID 2.2 RANK RAID-6 RAID RANK HDD - 5 -
2.3 SystemGlobe StoragePathSavior (SPS) istorage restoration I/O - 6 -
2.4 DynamicDataReplication DynamicDataReplication 3. Oracle 3.1 Automatic Storage Management ( ASM ) Oracle ASM RMAN Oracle I/O ASM DBA ASM Real Application Clusters (RAC) RAC 3.1.1 ASM I/O LUN ASM DB file destination ASM 1MB ASM ASM ( ) ASM (128k) I/O 3.1.2 ASM ASM Oracle ASM ASM ASM I/O ( ) ASM - 7 -
ASM ASM ASM Real Application Clusters ASM ASM ( ) ASM ASM 3.1.3 ASM ASM I/O I/O ASM I/O ASM I/O DBA I/O ASM RAID ASM I/O I/O ASM 3.1.4 ASM 2 3 ASM ASM ASM - 8 -
ASM ASM ASM ASM I/O (Dirty Region Logging) 4. ASM Linux Oracle Database Standard Edition RAC istorage RAID ASM Linux OS Oracle Database 10g 4.1 SPS SystemGlobe StoragePathSavior SPS istorage ls /dev /dev/dda1 /dev/dda2 /dev/dda3 /dev/ddb1 /dev/ddc1 /dev/ddd1 /dev/ddd1 /dev/dde1 4.2 RAW /etc/sysconfig/rawdevices RAC RAW SPS /dev/raw/raw /dev/raw/raw5 RAW Setting sample for /etc/sysconfig/rawdevices #vi /etc/sysconfig/rawdevices #for OCR Voting /dev/raw/raw1 /dev/dda1 /dev/raw/raw2 /dev/dda2-9 -
#for Oracle DATA/SPFILE /dev/raw/raw3 /dev/ddb1 #for Flash Recovery Area/control1/Archive /dev/raw/raw4 /dev/ddc1 #for LOG1/control2 /dev/raw/raw5 /dev/ddd1 rawdevices Starting rawdevices service # / etc/rc.d/init.d/rawdevices start raw qa" RAW Oracle chown - 10 -
# chown oracle:oinstall /dev/raw/raw[1-5] RAC 4.3 RAID ASM ASM RAID ASM S -6 SQL> CREATE DISKGROUP DATA EXTERNAL REDUNDANCY DISK /dev/raw/raw4, RAW Raw device ASM disk group Name name Redundancy Data /dev/raw/raw3 +DATA EXTERNAL Table space/spfile /dev/raw/raw4 +Flash Recovery Area EXTERNAL Online Redo log(primary) Control file(primary)/archive log file /dev/raw/raw5 +LOG1 EXTERNAL Online Redo log(secondary) Control file(secondary) 5. istorage ASM Oracle ISV Oracle RMAN (Recovery Manager) ASM 5.1 RAID6-11 -
istorage RAID6 ASM RAID6 istorage S S1400 OS LD Pool Path(RAW) Path(RAW) Path(RAW) File ASM disk group Physical RAID Name Resource No System Disk Level /dev/sdb /dev/dda /dev/raw/raw1 RAW Non-ASM LX_OCR OCR area 0005h 00h-01h RAID1 /dev/raw/raw2 Voting disk /dev/sdc /dev/ddb /dev/raw/raw3 RAW +DATA LX_DATA DATA/SPFILE 0006h 02h-07h RAID6 /dev/sdd /dev/ddc /dev/raw/raw4 RAW +Flash RecoveryLX_Flash Redo log1 Area Recovery Area Control file1/archive log /dev/sde /dev/ddd /dev/raw/raw5 RAW +LOG1 LX_LOG1 Redo log2 Control file2 0007h 08h-0dh RAID6 0007h 08h-0dh RAID6 Note: OCR Voting Release2 OracleClusterware OCR Voting ASM Disk Group +DATA ASM Disk Group +Flash Recovery Area ASM Disk Group +LOG1 Hot spare disk Redo Redo Flash Recovery Area Redo ASM Log1 Redo istorage S Oracle 5.2 SystemGlobe StoragePathSavior - 12 -
istorage S StoragePathSavior LUN LUN LUN I/O SystemGlobe StoragePathSavoir 5.3 LUN istorage RAID6 I/O I/O ASM I/O I/O ASM LUN DATA LUN I/O 5.4 Linux Oracle Database 10 RAC ASM istorage OS 5.4.1 LUN ASM Linux OS OS # echo scsi add-single-device a b c d >/proc/scsi/scsii OS Linux srvctl RAC Oracle [oracle@lx410 oracle]$ srvctl stop instance d SERAC i SERAC1 o immidiate [root@lx410 root]# shutdown r now After the reboot completion [root@lx410 root]# dmesg less SystemGlobe StoragePathSavior - 13 -
(/etc/dualpathrc) [root@lx410 root]# mv /etc/dualpathrc /etc/dualpathrc.bak [root@lx410 root]# mkdd Couldn nt open /etc/dualpathrc I try auto setting [root@lx410 root]# /etc/rc.d/init.d/dd_daemon restart [root@lx410 root]# mkdd rename and store for backup execute mkdd command temporary activate daemon for path patrol again fdisk [root@lx410 root]# fdisk /dev/dde use Path Manager device file name etc/sysconfig/rawdevices RAW rawdevices RAW Oracle Oracle [root@lx410 root]# vi /etc/sysconfig/rawdevices ex. /dev/raw/raw6 /dev/dde1 assign the additional disk as /dev/raw/raw6 [root@lx410 root]# /etc/rc.d/init.d/rawdevices restart [root@lx410 root]# raw qa confirm normal setting [root@lx410 root]# chown oracle:oinstall /dev/raw/raw6 ASM ASM [oracle@lx410 oracle]$ ORACLE_SID=+ASM1 sqplus SQL*Plus: Release 10.2.0.1.0 - Production on Fri Apr 7 16:52:03 2006 Copyright (c) 1982, 20045 Oracle. All rights reserved. Enter user-name: / as sysdba Connected to: Oracle Database 10g Enterprise Edition Release 10.2.1.0 Production With the Partitioning, Real Application Cluster and Data Mining options SQL> SELECT TOTAL_MB SQL> SELECT * FROM v$asm_diskgroup WHERE NAME='DATA'; GROUP_NUMBER NAME SECTOR_SIZE BLOICK_SIZE ALLOCATION_UNIT STATE TYPE TOTAL_MB FREE_MB ------------------------------------------------------------------------------------------------------------------------------------------------------------ 2 DATA 512 4096 1048576 MOUNTED EXTERN 269308 265244 SQL> ASM +DATA ALTER RAW - 14 -
SQL> alter diskgroup DATA add disk /dev/raw/raw6 ; Diskgroup altered SQL> SELECT TOTAL_MB 33.6GB SQL> SELECT * FROM v$asm_diskgroup WHERE NAME='DATA'; GROUP_NUMBER NAME SECTOR_SIZE BLOICK_SIZE ALLOCATION_UNIT STATE TYPE TOTAL_MB FREE_MB ------------------------------------------------------------------------------------------------------------------------------------------------------------ 2 DATA 512 4096 1048576 MOUNTED EXTERN 303351 265244 SQL> 6. Oracle NEC Oracle NEC ASM NEC Oracle NEC 7. 7.1 ASM LUN SELECT [oracle@lx410 oracle]$ ORACLE_SID=+ASM1 sqplus SQL*Plus: Release 10.2.0.1.0 - Production on Fri Apr 7 16:52:03 2006 Copyright (c) 1982, 2005, Oracle. All rights reserved. Enter user-name: / as sysdba Connected to: Oracle Database 10g Enterprise Edition Release 10.2.1.0 Production With the Partitioning, Real Application Cluster and Data Mining options SQL> SELECT * FROM v$asm_diskgroup WHERE NAME='DATA'; GROUP_NUMBER NAME SECTOR_SIZE BLOICK_SIZE ALLOCATION_UNIT STATE TYPE TOTAL_MB FREE_MB ----------------------------------------------------------------------------------------------------------------------------------------------------------- 2 DATA 512 4096 1048576 MOUNTED EXTERN 269308 265244 SQL> - 15 -
RAW Oracle [oracle@lx410 oracle]$ srvctl stop instance d SERAC i SERAC1 o immediate; ASM [oracle@lx410 oracle]$ srvctl stop asm -n lx410 o immediate; HDD istorage Manager (ism) LUN RAW [root@lx410 root]# raw /dev/raw/raw4 0 0 fdisk LUN [root@lx410 root]# fdisk /dev/ddb SPS The number of cylinders for this disk is set to 4422. There is nothing wrong with that, but this is larger than 1024, and could in certain setups cause problems with: 1) software that runs at boot time (e.g., old versions of LILO) 2) booting and partitioning software from other OSs (e.g., DOS FDISK, OS/2 FDISK) Command (m for help):w w fdisk RAW RAW RAW [root@lx410 root]# raw /dev/raw/raw4 /dev/ddb1 ASM $ORACLE_BASE/admin/+ASM/bdump/alert+ASM1.log +DATA [oracle@lx410 oracle]$ srvctl start asm -n lx410 o open; ASM Oracle $ORACLE_BASE/admin/SERAC/bdump/alertSERAC1.log Oracle - 16 -
[oracle@lx410 oracle]$ srvctl start instance d SERAC I SERAC1 o open; ASM [oracle@lx410 oracle]$ ORACLE_SID=+ASM1 sqplus SQL*Plus: Release 10.2.0.1.0 - Production on Fri Apr 7 16:52:303 2006 Copyright (c) 1982, 2004, Oracle. All rights reserved. Enter user-name: / as sysdba Connected to: Oracle Database 10g Enterprise Edition Release 10.2.1.0 Production With the Partitioning, Real Application Cluster and Data Mining options SQL> +DATA ALTER SQL> ALTER DISKGROUP DATA RESIZE DISK DATA_0000 SIZE 300G; Diskgroup altered SQL> SELECT TOTAL_MB SQL> SELECT * FROM v$asm_diskgroup WHERE NAME='DATA'; GROUP_NUMBER NAME SECTOR_SIZE BLOICK_SIZE ALLOCATION_UNIT STATE TYPE TOTAL_MB FREE_MB 2 DATA 512 4096 1048576 MOUNTED EXTERN 312320 265244 SQL> - 17 -
7.2 DynamicDataReplication - 18 -
- 19 -
- 20 -
Oracle MV # ismrc_replicate file -wait 0:LX_RAC_DATA_MV 0:LX_RAC_DATA_RV - 21 -
SQL> alter tablespace begin backup; ASM SQL> select * from v$asm_operation SQL> alter diskgroup rebalance power 0 MV RV # ismrc_separate file wait MV SQL> alter tablespace end backup; - 22 -
SQL> alter database backup controlfile to /u01/app/oracle/ctlsid.bak reuse; SQL> create pfile='/u01/app/oracle/initsid.bak'from spfile= [root@ ] # umount - 23 -
[root@ ] # ismrc_replicate file -nowait 0:LX_RAC_ARCTMP_MV 0:LX_RAC_ARCTMP_RV RMAN> RUN { BACKUP DEVICE TYPE DISK FORMAT tempmv /%U ARCHIVELOG ALL DELETE INPUT }; [root@ ] # umount - 24 -
[root@ ] # ismrc_separate file rvuse immediate -nowait [root@ ] # mount t [root@ ] # ismrc_wait cond sep file - 25 -
$ srvctl stop database d o immediate - 26 -
$ srvctl stop asm n i ASM o immediate [root@ ] # ismrc_restore file mode protect -nowait $ srvctl start asm -n i ASM o open $ srvctl start instance d i o mount [oracle@ ] $ sqlplus / as sysdba SQL> SET AUTORECOVERY ON; SQL> SET LOGSOURCE tempmv ; SQL> RECOVER DATABASE; SQL> ALTER DATABASE OPEN; - 27 -
$ srvctl stop instance d i o immediate $ srvctl start database d o open - 28 -
- 29 -