Veritas Volume Replicator VVR quick start

=== setup vvr
/etc/init.d/vras-vradmind.sh start                                              [master/secondary]
/etc/init.d/vxnm-vxnetd start                                                   [master/secondary]
vxassist -g repdg make mysrl 200M alloc=disk02                                  [master/secondary]
vradmin -g repdg createpri myrvg vol01 mysrl                                    [master]
vxdg list

put dgid into /etc/vx/vras/.rdg on the secondary site. eg
echo “1169092729.31.slave” >> /etc/vx/vras/.rdg  (on secondary only)            [secondary]
vradmin -g repdg addsec myrvg slave test01 prlink=link_slave srlink=link_test01 [master]
vradmin -g repdg -a startrep myrvg test01                                       [master]
===checking

vradmin -g repdg -l printrvg myrvg
vradmin -g repdg repstatus myrvg
vxrvg -g repdg getdatavols myrvg
vxrvg -g repdg getrlinks myrvg
vxrlink -g repdg status link_slave

vradmin -g repdg pauserep myrvg test01                                          [master]
vradmin -g repdg resumerep myrvg test01                                         [master]

===remove vvr

vradmin -g repdg -f stoprep myrvg test01                                        [master]
vradmin -g repdg delsec myrvg test01                                            [master]
vradmin -g repdg -f delpri myrvg                                                [master]

vxedit -g repdg -f -r rm mysrl                                                  [master/secondary]

vxassist -g repdg remove log vol01  [delete one log each time],  need to run twice if there are 2 logs   [master/secondary]

Veritas Cluster Server tip to manually import type/agent

It’s convenient to import a new type from hagui, but under some case you might be unable to do so, like no Xserver, or slow network access etc.

Here I will show you how to import  the new agent type from command line.

/opt/VRTSvcs/bin/hacf -typetocmd /etc/VRTSvcs/conf/VVRTypes.cf -display | ksh

easy and straightforward.  but maybe  you don’t know it.

hacf -typetocmd is not showing from “hacf -help”.

test01:/opt/VRTSvcs/bin #hacf -help
VCS INFO V-16-1-10601 Usage:
         hacf -cftocmd   <conf_dir> [-dest <dest_dir>] [-display]
         hacf -cmdtocf   <conf_dir> [-dest <dest_dir>]
         hacf -verify    <conf_dir> [-display]
         hacf -help

test01:/opt/VRTSvcs/bin #

vxvm snapshot quick start

source dg: testdg
source vol:  vol01
data volmune mounted at:  /slavefs
snapshot mounted at:  /snapfs

 

space-optimized instant snapshot
====================================
1. check dco
vxprint -g testdg -F%instant vol01
vxprint -g testdg -F%fastresync vol01
vxprint

2.if dco not available, create it by prepare
vxsnap -g testdg prepare vol01
vxprint -g testdg -F%instant vol01
vxprint -g testdg -F%fastresync vol01

3. create cache volume in preparation of space-optimized snap
vxassist -g testdg make cachevol 100M layout=nostripe init=active disk01
normally it should be 10% of original volume size

4. create cache objects in preparation of space-optimized snap
vxmake -g testdg cache mycacheobj cachevolname=cachevol autogrow=on
vxcache -g testdg start mycacheobj
vxprint -g testdg

5. create the instant snap with space-optimized
vxsnap -g testdg make source=vol01/newvol=snapvol01/cache=mycacheobj alloc=disk02
vxprint -g testdg -F%incomplete snapvol01
6. check file system
fsck -F vxfs /dev/vx/rdsk/testdg/snapvol01

7. mount file system
mount -F vxfs /dev/vx/dsk/testdg/snapvol01 /snapfs/

8. refresh
umount /snapfs
vxsnap -g testdg refresh snapvol01 source=vol01
fsck -F vxfs /dev/vx/rdsk/testdg/snapvol01
mount -F vxfs /dev/vx/dsk/testdg/snapvol01 /snapfs/

9. restore data volume
umount /slavefs  (data volume)
vxsnap -g testdg restore vol01 source=snapvol01
fsck -F vxfs /dev/vx/rdsk/testdg/vol01
mount -F vxfs /dev/vx/dsk/testdg/vol01 /slavefs

10. remove instant space-optimized snapshot
umount /snapfs
vxedit -g testdg -r -f rm snapvol01     [-f is used as it is still enabled, otherwise stop it first]
vxedit -g testdg -f -r rm mycacheobj
vxsnap -g testdg -f unprepare vol01

 

 
full-sized instant snapshot
====================================
source dg: testdg
source vol:  vol01
1. check dco
vxprint -g testdg -F%instant vol01
vxprint -g testdg -F%fastresync vol01
vxprint

2.if dco not available, create it by prepare command
vxsnap -g testdg prepare vol01
vxprint -g testdg -F%instant vol01
vxprint -g testdg -F%fastresync vol01

3. create a full-sized volume in preparation of space-optimized snap
LEN=`vxprint -g testdg -F%len vol01`
DCONAME=`vxprint -g testdg -F%dco_name vol01`
RSZ=`vxprint -g testdg -F%regionsz $DCONAME`
vxassist -g testdg make snapvol01 $LEN layout=nostripe init=active alloc=disk02
vxsnap -g testdg prepare snapvol01 regionsize=$RSZ
4. create the snapshot
vxsnap -g testdg make source=vol01/snapvol=snapvol01 [syncing=off]
vxprint -g testdg -F%incomplete snapvol01

5. check file system
fsck -F vxfs /dev/vx/rdsk/testdg/snapvol01

6. mount file system
mount -F vxfs /dev/vx/dsk/testdg/snapvol01 /snapfs

7. refresh
umount /snapfs
vxsnap -g testdg refresh snapvol01 source=vol01
fsck -F vxfs /dev/vx/rdsk/testdg/snapvol01
mount -F vxfs /dev/vx/dsk/testdg/snapvol01 /snapfs/
8. restore data from snap
umount /slavefs 
vxsnap -g testdg restore vol01 source=snapvol01
fsck -F vxfs /dev/vx/rdsk/testdg/vol01
mount -F vxfs /dev/vx/dsk/testdg/vol01 /slavefs
9. remove snap
vxprint -g testdg
umount /snapfs
vxedit -g testdg -f -r rm snapvol01
vxsnap -g testdg -f unprepare vol01

 

third-mirror break-off snapshots (must be instant ready)
=================================
1. dco/drl
vxsnap -g testdg prepare vol01
2. create mirror and run it in background
vxsnap -b -g testdg addmir vol01 nmirror=1 alloc=disk01,disk02  
You must spread into two disk for mirrored volume, as dcl is mirrored.

or

1. prepare volume
vxsnap -g testdg prepare vol01 ndcomirs=1 alloc=disk01

2. create mirror
vxsnap -b -g testdg addmir vol01 nmirror=1 alloc=disk02

 

continue
3. vxprint -g testdg  [to find out the plex with snapdone]/ vxtask list

4. vxsnap -g testdg make source=vol01/newvol=snapvol01/plex=vol01-02

5. fsck -F vxfs /dev/vx/rdsk/testdg/snapvol01

6. mount -F vxfs /dev/vx/dsk/testdg/snapvol01 /snapfs

7. refresh
umount /snapfs
vxsnap -g testdg refresh snapvol01 source=vol01
fsck -F vxfs /dev/vx/rdsk/testdg/snapvol01
mount -F vxfs /dev/vx/dsk/testdg/snapvol01 /snapfs/

8. restore
umount /slavefs 
vxsnap -g testdg restore vol01 source=snapvol01
fsck -F vxfs /dev/vx/rdsk/testdg/vol01
mount -F vxfs /dev/vx/dsk/testdg/vol01 /slavefs
9. remove snap
vxprint -g testdg
umount /snapfs
vxedit -g testdg -f -r rm snapvol01
vxsnap -g testdg -f unprepare vol01

 
traditional third-mirror break-off snaphost
=============================================

1
vxassist -b -g testdg snapstart nmirror=1 vol01

2.
vxassist -g testdg snapshot vol01 snapvol01

3.
fsck -F vxfs /dev/vx/rdsk/testdg/snapvol01

4.
mount -F vxfs /dev/vx/dsk/testdg/snapvol01 /snapfs

5. refresh
umount /snapfs
vxassist -b -g testdg snapback snapvol01
vxassist -g testdg snapshot vol01 snapvol01
fsck -F vxfs /dev/vx/rdsk/testdg/snapvol01
mount -F vxfs /dev/vx/dsk/testdg/snapvol01 /snapfs

6. restore
umount /slavefs
umount /snapfs
vxassist -b -g testdg -o resyncfromreplica snapback snapvol01
fsck -F vxfs /dev/vx/rdsk/testdg/vol01
mount -F vxfs /dev/vx/dsk/testdg/vol01 /slavefs

after snapback,  snapvol-1 is gone. The plex will go back to vol01 for next split
vxassist -g testdg snapshot vol01 snapvol01
fsck -F vxfs /dev/vx/rdsk/testdg/snapvol01
mount -F vxfs /dev/vx/dsk/testdg/snapvol01 /snapfs/

7. remove snap
umount /snapfs
vxedit -g testdg -r -f rm snapvol01

chroot for ftp/telnet/rsh/ssh on solaris 10

By properly setting up a new root environment, you can fool the user and make them feel they are logging into a real root environment, rather than a jailed root. the funny part is to make a perfect root environment, with everything setting up, not exposure of the real /etc/passwd and /etc/shadow etc… The way to make a root environment I can think of includes 1. copy all required files (binary files, libraries, configurations …) down to the new root. 2. loopback mount the relevant file systems down to the new root. (the drawback is it will expose the real /etc to the user) 3. set the root path to the root path of a non-global zone. (make sure zone is running if you are running a sparse local zone)

 

1. ftp

It is fairly simple to get chroot for ftp session, change home directory from /etc/passwd as below,
usertest:55502:0:comments:/export/home/usertest/newroot/./test:/bin/ksh

/export/home/usertest/newroot/./test  is the new home directory, of which, /export/home/usertest/newroot is the new root, and /test is relateive path to the new root.

ftp server will change root to “/export/home/usertest/newroot/” after ftp session established.

Pls make sure you copy some essential required file down to the new root, such as /bin /sbin /etc,  etc.
the easy way is to loopback mount those file system as below

cd /export/home/usertest/newroot
mkdir bin sbin etc usr lib
mount -F lofs /bin  /export/home/usertest/newroot/bin
mount -F lofs /sbin  /export/home/usertest/newroot/sbin
mount -F lofs /usr  /export/home/usertest/newroot/usr
mount -F lofs /etc  /export/home/usertest/newroot/etc
mount -F lofs /lib  /export/home/usertest/newroot/lib

after user logs in via ftp, the session will be limited under the new root (/export/home/usertest/newroot)

pls note:  for ftp only, even the new root is total empty, the ftp session is still able to establish. But list functionality from ftp gui client doesn’t work, command line works well.
           authentication is based /etc/passwd and /etc/shadow under real root.

2.  telnet  (this example is based on solaris 10, change the procedure accordingly if you are running sol9 or sol
    it doesn’t matter if you do not change home directory in /etc/passwd

2.1 prepare the new root environment (/export/home/usertest/newroot)
    copy all required files/directories down to the new root and test it.
    eg:  /usr/sbin/chroot  /export/home/usertest/newroot  /bin/ksh

2.2
    inetadm -l telnet  > telnet.bak
    inetadm -m telnet exec=”/usr/sbin/a.ksh”
    svcadm restart telnet
   
    a.ksh
    #!/bin/ksh
    /usr/sbin/chroot /export/home/usertest/newroot/ /usr/sbin/in.telnetd
   
pls note, as chroot is explicitly run by a.ksh, the authentication will be based on /etc/passwd and /etc/shadow under new root, namely
/export/home/usertest/newroot/etc/passwd and /export/home/usertest/newroot/etc/shadow

3.  rsh
    it doesn’t matter if you do not change home directory in /etc/passwd
    rsh is using SMF svc:/network/login:rlogin and associated daemon is /usr/sbin/in.rlogind
   
3.1 prepare the new root
3.2 inetadm -m shell/rlogin exec=”/usr/sbin/b.ksh”

    b.ksh
    #!/bin/ksh
    /usr/sbin/chroot /export/home/usertest/newroot/ /usr/sbin/in.rlogind
   
pls note, authentication is based on /export/home/usertest/newroot/etc/passwd and /export/home/usertest/newroot/etc/shadow.

4.  openssh   

    authentication is based /etc/passwd and /etc/shadow under real root.
4.1  it is a must to change the home directory as above for ftp.
4.2  prepare the new root and test it.
4.3  download the below packages from sunfreeware.com and install them
     
     gcc
     libiconv
     openssl

4.4  download openssh source code and save into a temporary directory
     modify session.c as stated below

+ #define CHROOT                     /* to be added (of course without + mark at the beginning) */

  /* func */

  Session *session_new(void);
***************
*** 1159,1164 ****
— 1161,1171 —-
  void
  do_setusercontext(struct passwd *pw)
  {
+ #ifdef CHROOT                      /* to be added (of course without + mark at the beginning ) */
+       char *user_dir;              /* to be added (of course without + mark at the beginning) */
+       char *new_root;              /* to be added (of course without + mark at the beginning) */
+ #endif /* CHROOT */                /* to be added (of course without + mark at the beginning) */
+
        char tty=”;

  #ifdef HAVE_CYGWIN

                if (setlogin(pw->pw_name) < 0)
                        error(“setlogin failed: %s”, strerror(errno));

+               #ifdef CHROOT                  /* to be added (of course without + mark at the beginning) */
+                       user_dir = xstrdup(pw->pw_dir);  /* to be added (of course without + mark at the beginning) */
+                       new_root = user_dir + 1;  /* to be added (of course without + mark at the beginning) */
+
+                       while((new_root = strchr(new_root, ‘.’)) != NULL) {   /* to be added (of course without + mark at the beginning) */
+                               new_root–;                                   /* to be added (of course without + mark at the beginning) */
+                               if(strncmp(new_root, “/./”, 3) == 0) {        /* to be added (of course without + mark at the beginning) */
+                                       *new_root = ”;                     /* to be added (of course without + mark at the beginning) */
+                                       new_root += 2;                        /* to be added (of course without + mark at the beginning) */
+
+                                       if(chroot(user_dir) != 0)             /* to be added (of course without + mark at the beginning) */
+                                               fatal(“Couldn’t chroot to user directory %s”, user_dir);   /* to be added (of course without + mark at the beginning) */
+
+                                       pw->pw_dir = new_root;                 /* to be added (of course without + mark at the beginning) */
+                                       break;                                 /* to be added (of course without + mark at the beginning) */
+                               }                                              /* to be added (of course without + mark at the beginning) */
+                               new_root += 2;                                 /* to be added (of course without + mark at the beginning) */
+                       }                                                      /* to be added (of course without + mark at the beginning) */
+               #endif /* CHROOT */                                            /* to be added (of course without + mark at the beginning) */
+
                if (setgid(pw->pw_gid) < 0) {
                        perror(“setgid”);
                        exit(1);
     

4.5
./configure –with-random=/dev/random \
  –prefix=/usr/local –libexecdir=/usr/libexec/openssh \
  –sysconfdir=/usr/local/etc –mandir=/usr/share/man

4.6
make

4.7
mkdir /var/empty
chown root:sys /var/empty
chmod 755 /var/empty
groupadd sshd
useradd -g sshd -c ’sshd privsep’ -d /var/empty -s /bin/false sshd

4.8
make install

4.9
/usr/local/bin/ssh-keygen -t rsa1 -f /usr/local/etc/ssh_host_key -N
/usr/local/bin/ssh-keygen -t dsa -f /usr/local/etc/ssh_host_dsa_key -N
/usr/local/bin/ssh-keygen -t rsa -f /usr/local/etc/ssh_host_rsa_key -N

4.10
create file /etc/init.d/sshd

#!/bin/sh
pid=`/usr/bin/ps -e | /usr/bin/grep sshd |   /usr/bin/sed -e ’s/^  *//’ -e ’s/ .*//’`
case $1 in
’start’)
  /usr/local/sbin/sshd;;
’stop’)
  if [ "${pid}" != "" ]; then
  /usr/bin/kill ${pid}; fi;;
*)
  echo “usage: /etc/init.d/sshd (start|stop)”;;
esac

4.11
ln -s /etc/init.d/sshd /etc/rc2.d/S100sshd_chroot
ln -s /etc/init.d/sshd /etc/rc0.d/K100sshd_chroot

4.12
/etc/rc2.d/S100sshd_chroot start

call mum!

today, being the last day of 2008,  when I finished my day and stepped out the office,  a plane was writing above words on the sky.

I took a picture and attached it here as a reminder that when you are less busy and have a moment, give your parents a ring, let them know you are going well everything.

two years already

By today, I had been here, Sydney for two years.

on 31st, Dec 2006, I arrived in Sydney in the morning,  everying is still so fresh in my mind. My brother-in-law picked me up and drove me back his home, we live in his house for 20 days, then we rent another townhouse, and later on we bought a townhouse.

Then we got a baby, and now my baby is already nearly 4 months  old.

how time flies!!!

problem with disk capacity

Yesterday, I was tasked to configure some new LUNs that allocated to a M500o server.  All luns worked very well except one.

syminq command showed the disk is with 18G space, luxadmin showed the same result as well.  But format command only recognize it as 9G.  

 

bash-3.00# syminq | grep -i 195E

/dev/rdsk/c3t60060480000290100478533031393545d0s2 M(2) EMC SYMMETRIX 5771 780195E008 17677440 (here 18G)

bash-3.00# luxadm display /dev/rdsk/c3t60060480000290100478533031393543d0s2

DEVICE PROPERTIES for disk: /dev/rdsk/c3t60060480000290100478533031393543d0s2

Vendor: EMC

Product ID: SYMMETRIX

Revision: 5771

Serial Num: 10047895C008

Unformatted capacity: 17263.125 Mbytes <—————–18G

Read Cache: Enabled

Minimum prefetch: 0×0

Maximum prefetch: 0xffff

Device Type: Disk device

Path(s):

/dev/rdsk/c3t60060480000290100478533031393543d0s2

/devices/scsi_vhci/ssd@g60060480000290100478533031393543:c,raw

Controller /devices/pci@12,600000/SUNW,qlc@0/fp@0,0

Device Address 5006048452a53fb9,6a

Host controller port WWN 2100001b320f2023

Class primary

State ONLINE

Controller /devices/pci@12,600000/SUNW,qlc@0,1/fp@0,0

Device Address 5006048452a53fb6,6a

Host controller port WWN 2101001b322f2023

Class primary

State ONLINE

 

But, with format

55. c3t60060480000290100478533031393545d0 <EMC-SYMMETRIX-5771 cyl 18412 alt 2 hd 15 sec 128>

/scsi_vhci/ssd@g60060480000290100478533031393545

56. c3t60060480000290100478533031393543d0 <EMC-SYMMETRIX-5771 cyl 9205 alt 2 hd 15 sec 128> default4 <— only 9205 cylinders

/scsi_vhci/ssd@g60060480000290100478533031393543

57. c3t60060480000290100478533031343033d0 <EMC-SYMMETRIX-5771 cyl 36826 alt 2 hd 30 sec 128>

/scsi_vhci/ssd@g60060480000290100478533031343033

partition> p

Volume: default4

Current partition table (original):

Total disk cylinders available: 9205 + 2 (reserved cylinders)

Part Tag Flag Cylinders Size Blocks

0 unassigned wu 0 0 (0/0/0) 0

1 unassigned wu 0 0 (0/0/0) 0

2 backup wu 0 – 9204 8.43GB (9205/0/0) 17673600 <—– 9G

3 unassigned wu 0 0 (0/0/0) 0

4 unassigned wu 0 0 (0/0/0) 0

5 unassigned wu 0 0 (0/0/0) 0

6 unassigned wu 0 0 (0/0/0) 0

7 unassigned wu 0 0 (0/0/0) 0

partition>

 

This is becuase the disk got wrongly labelled before, we just need to manually destroy the label and re-label it.

dd if=/dev/zero of=/dev/rdsk/c3t60060480000290100478533031393543d0s2 bs=512k count=1

then go back to format and re-label the disk,  afterwards, format reported it as 18G emc disk.

圣诞节还是来了

因为对圣诞节没有太多的感受，因此没有选择在圣诞节期间休假。还是选择这段时间工作好阿，每天support  电话就响有限大几次，比较轻松。今天经理过来，说今天下午大家要是没事，可以提前走，让我想到了国内的时候，每次大年30的时候，领导都是说，没事的时候，大家可以早点走，而且，公司的班车都是1:30  就发车。

昨天还碰到一件有意思大事情，下班回家，开到 delhi  路和  epping  路交界处，等红灯，绿灯一亮，大家都准备发动，但是有个吉普车，超级强悍，它发出了我所见过大最强烈的大浓烟，导致它后面大5量车都不敢动地方，估计这车是烧柴禾的，而且还是那种湿柴禾。

还是关于修改主机名，也要照顾VCS的感受！

上周末，成功将一个cluster中的两个节点替换了。原有系统是一个四节点的cluster，节点app1和app2  是两台 v480, db1和db2 是两台F6800。用两台M4000替换app1和app2。两台M4000分别命名为app11, app21，建立一个2节点的cluster，测试成功后，rename app11, app21 to app1，app2。 修改/etc/llttab /etc/llthosts,  相应的修改了main.cf 后，新节点加入后，一直显示节点app11加入，真是气人啊。

后来仔细看看文档， 发现还有个文件需要修改 /etc/VRTSvcs/conf/sysname, 如果该文件存在，cluster中的节点名字就来自它，如果不存在，就来在”uname -n”, 修改了这个文件后，成功将节点app1,app2重新加入到cluster中。

修改主机名，要考虑vxvm的感受

一些时候，需要修改主机名，对于vcs，相应的配置比较明了，不再累叙。对于vxvm，可以

1.  collect a copy of vxexplorer

2. vxdg -h <new-hostname> deport dgname 

    这步是修改vxvm disk上的信息

3. vxdctl hostid <newhostname>

    这步是修改/etc/vx/volboot 文件，以便将来有新盘加入的时候，写入新的hostname信息。

4. Change the hostname and IP address of the system

5. reboot server

但是在vxvm4.1 以及 5.0上，未做如上修改，主机重启后，disk groups 也都全部被自动import进来了。所以，估计除了hostname，vxvm肯定还是在disk上写了其他的信息，主机重启后，能自动识别disk 就是应该属于本机，就自动import进来了。