Diogo Cortiz da Silva

Title in portuguese (Solaris Zonas – Como configurar).

Zones is the technology that provides a system isolation for Solaris OS that is extremly secure and scalable. It is a virtualized OS Service that allows users to use different instances of Operating System in the same server. It is called lightweight VM because Solaris Zones works a lillte bit different than others Virtualization Products such as VMware. It is important to highlight that Solaris Zones have smallest process overhead because it duplicates only some processes for each zone and not the whole system. If you are familiar with BSD jails, you notice similar concept in Solaris zones.

The idea of Solaris Zone is to create an isolated environment and processes tree. The users will use each zone as an separated server, each one with its processes. Processes inside one Zone can not affect processes in other zones. Users connected to one zone can not affect processes in other zones.

Let’s imagine one scenario that you need to run 3 instances of the same application for different customers. You do not want that the application for customer A affects the application for customer B but you have a server with capacity to run those 3 instances. You do not need to use 3 different servers, you can put those 3 application in the same server using Solaris zones. This technology will delivery you an isolated environment where you can use each zone for each customer like separated servers.

When you install a Solaris 10 you will have a Global Zone and from this main zone you can install additional Non-Global Zones. You can imagine a Non-Global Zone as a separated Operating System with its own hostaname, IP and so on. Users can boot, shutdown or reboot only one zone and it will not affect others zones. Global zone is the responsible to manage Non-Global zones.

Note: Zones + Resource Management = Solaris Container

I will show how to create, configure and boot Non-Global Zones. I believe it will clarify some possible doubts:

Before creating a zone the best idea is to prepare the environment that local zone will be mounted (including filesystem, Network Interface, Processors and so on).

In this tutorial I am assuming a server with 1GB ram and 2 CPU’s and 4 disks.

For example:

• Mount a filesystem on Global Zone to store the non-global zone. (It is very useful when using Veritas Storage Foundation and HA).

• Plumb a specific network interface dedicated to a non-global zone when applicable.

• Create a Resource Pool to assign this to a non-global zone (CPU and so on).

Remind that this is only and example and it should be customized according to the platform/application.

bash-3.00# format

Searching for disks…done

AVAILABLE DISK SELECTIONS:

0. c1t0d0 <DEFAULT cyl 4092 alt 2 hd 128 sec 32>

/pci@0,0/pci15ad,1976@10/sd@0,0

1. c1t1d0 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>

/pci@0,0/pci15ad,1976@10/sd@1,0

2. c1t2d0 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>

/pci@0,0/pci15ad,1976@10/sd@2,0

3. c1t3d0 <DEFAULT cyl 6 alt 2 hd 64 sec 32>

/pci@0,0/pci15ad,1976@10/sd@3,0

I will divide disk utilization according to the following table:

• 0. c1t0d0 – Will be used for the original instalation (Global Zone)

• 1. c1t1d0 – Will be used for the Zone1 and will be mounted on /zone1

• 2. c1t2d0 – Will be used for the Zone2 and will be mounted on /zone2

• 3. c1t3d0 – Will be used for a filesystem that will be shared by Zone1 and Zone2

Note: Keep in mind that we do not need to specify a dedicate filesystem for a zone, so we will create a non-global zone (Zone3) in the /zone3 directory on / filesystem on Global Zone.

GlobalZone

Disk to mount / filesystem = c1t0d0

Network Interface: e1000g0 (shared by all zones)

IP: 192.168.50.1

CPU: 0 and 1

Note: Global Zone is responsible to all resources in the servers.

Zone1

Path to be installed on Global-Zone = /zone1

Disk used to mount /zone1 = c1t1d0

Network Interface: e1000g0 (shared by all zones)

IP:192.168.50.2

CPU = 0 ( I will assign CPU ID 0 to Zone1)

Zone2

Path to be installed on Global-Zone = /zone2

Disk used to mount /zone1 = c1t2d0

Network Interface: e1000g0 (shared by all zones)

IP:192.168.50.3

CPU = 1 (I will assign CPU ID to Zone2)

Zone3

Path to be installed on Global-Zone = /zone3

Disk used to mount /zone1 = c1t0d0 (In this case I am not using a dedicate disk to install the zone. I will be used a direcoty “/zone3” that is mounted on “/” filesystem on global-zone).

Network Interface: e1000g0 (shared by all zones)

IP:192.168.50.4

CPU = 1 and 0 (I will not specify a zone for this Zone, therefore it will use all CPU’s in the server).

In this point I am assuming that I have on the global zone:

• Network Inteface Plumbed and Up

• /zone1 mounted

• /zone2 mounted

• /zone3 directory created

Okay, our environment on Global-Zone is ready and we can start. (I showed a simple way to prepare an environment for 3 non-global zones. In this tutorial I will show how to create Zone1).

Before start creating the zones, let’s configure the resource pool (CPU and Memory) to be used by the Zone1. (remember that Zone1 has already its directory “/zone1” that will be used to install the zone ready on global-zone, also the network interface is up).

Okay, the first step to create the resource pool is to have in mind which resources will be assign to this pool. In our case, we will create a “zone1-pool” which will have only one processor from those 2 processors installed in the Global-Zone.

1. Configuring resource pool (If you are not interesting to use resource pool and will use only the zones in the traditional way, please go the nest section.)

Let’s start. You should configure the resource pool in the global-zone. Once it is created you will be able to set this for a specific non-global zone (Solaris 10 you just installed).

Verify if pool daemon is running:

bash-3.00# ps -ef | grep poold
bash-3.00# pooladm
pooladm: couldn’t open pools state file: Facility is not active

Start Pooladm. Use -e option to enable pool facility.

bash-3.00# pooladm -e

Create a file with the following information:

vi zone1pool
create pset zone1-pset ( uint pset.min = 1; uint pset.max = 1 )
create pool zone1-pool
associate pool zone1-pool ( pset zone1-pset )

In this example I created a zone1-pset with only 1 processors.

I created a zone1-pool and associate zone1-pset into this pool.

Update the pooladm configuration with the file zone1pool:

bash-3.00# poolcfg -f zone1pool
poolcfg: cannot load configuration from /etc/pooladm.conf: No such file or director

If this error occurs, use -s opation to save current configuration in memory into the file /etc/pooladm.conf

Go ahead and load again the configuration from the file into pooladm configuration.

bash-3.00# poolcfg -f zone1pool

The file is loaded but we need to turn this active:

bash-3.00# pooladm -c

Check the current configuration on Pooladm:

bash-3.00# pooladm

system default

string system.comment

int system.version 1

boolean system.bind-default true

string system.poold.objectives wt-load

pool zone1-pool

int pool.sys_id 1

boolean pool.active true

boolean pool.default false

int pool.importance 1

string pool.comment

pset zone1-pset

pool pool_default

int pool.sys_id 0

boolean pool.active true

boolean pool.default true

int pool.importance 1

string pool.comment

pset pset_default

pset zone1-pset

int pset.sys_id 1

boolean pset.default false

uint pset.min 1

uint pset.max 1

string pset.units population

uint pset.load 0

uint pset.size 1

string pset.comment

cpu

int cpu.sys_id 0

string cpu.comment

string cpu.status on-line

pset pset_default

int pset.sys_id -1

boolean pset.default true

uint pset.min 1

uint pset.max 65536

string pset.units population

uint pset.load 33

uint pset.size 1

string pset.comment

cpu

int cpu.sys_id 1

string cpu.comment

string cpu.status on-line

Okay, at this point you have the pool resource configured but it is not assigned to any zone yet.

In the next chapter we will create a zone and in chapter 3 I will associate with the resource pool.

2. Configuring a non-global zone

Okay, let’s start listing all the zones configured in the platform

bash-3.00# zoneadm list -cv

ID NAME STATUS PATH BRAND IP

0 global running / native shared

Create the zone using the zonecfg command:

bash-3.00# zonecfg -z zone1

zone1: No such zone configured

Use ‘create’ to begin configuring a new zone.

zonecfg:zone1> create

zonecfg:zone1> set zonepath=/zone1

zonecfg:zone1> set autoboot=false

zonecfg:zone1> add net

zonecfg:zone1:net> set physical=e1000g0

zonecfg:zone1:net> set address=192.168.50.2

zonecfg:zone1:net> end

zonecfg:zone1> verify

zonecfg:zone1> exit

zonepath = Path that the zone will be installed.

Autoboot = if the zone will be booted during the globalzone boot.

Physical = Network Interface that will be used to add an IP for non-global zone. ( you should specify a real interface on Global Zone. It will be added an IP on this interface for a non-global zone.

Address = IP Address that will be assigned for the non-global zone.

List again the zones in the global-zones:

bash-3.00# zoneadm list -cv

ID NAME STATUS PATH BRAND IP

0 global running / native shared

zone1 configured /zone1 native shared

Yeah!!! The zone is configured. It was the first step. You need to perform some more steps in order to get the zone running!!

Here is the list of status that a Zone can be assigned:

• configured – It has the initial configuration. Zone was not installed and it can not be booted.

• incomplete – This is the status when zoneadm is installing or unistalling a zone.

• installed – Zone is configured and packages are installed. You only need to boot it

• ready – The zone is ready. There is a zsched process and all interfaces are plumbed. It will be moved to running status.

• Running – It is running!  Everything is ready you can use the zone.

• Shutting down and down – When the zone is being halted.

Great!! Next step is the installation.

Change the permission for the Zonepath:

bash-3.00# chmod 700 /zone1

Start the installation:

bash-3.00# zoneadm -z zone1 install

Check the status:

bash-3.00# zoneadm list -cv

– zone1 installed /zone1 native shared

ID NAME STATUS PATH BRAND IP

0 global running / native shared

If the status is marked as “installed”, you are ready to boot it!!!

bash-3.00# zoneadm -z zone1 boot

Check the status:

ID NAME STATUS PATH BRAND IP

0 global running / native shared

1 zone1 running /zone1 native shared

It is installed and it is running. The next step is to set the configuration of the OS installed.

To proceed you will need access via CONSOLE.

bash-3.00# zlogin -C zone1

Reply the questions to finish the system configuration step. If you would like to exit the CONSOLE, use “~.”.

Fantastic!!! It is running and you are ready to use your new non-global zone, you can add your application and so on.

Note: Keep in mind that the IP used by non-global zone is added to the interface in the global zone. My sugesstion is to add the IP in the same sub-net as the IP of non-global to the global zone, so you will be able to access it.

That is the ifconfig ouput from the global zone:

lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1

inet 127.0.0.1 netmask ff000000

e1000g0: flags=1004843<UP,BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 1500 index 2

inet 146.250.176.35 netmask ffffff00 broadcast 146.250.176.255

ether 0:c:29:88:22:71

e1000g0:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2

zone zone1

inet 192.168.50.2 netmask ffffff00 broadcast 192.168.50.255

e1000g0:2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2

inet 192.168.50.1 netmask ffffff00 broadcast 192.168.50.255

lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1

inet6 ::1/128

e1000g0: flags=2004841<UP,RUNNING,MULTICAST,DHCP,IPv6> mtu 1500 index 2

inet6 fe80::20c:29ff:fe88:2271/10

ether 0:c:29:88:22:71

Note:

e100g0 is the interface for the traffic network.

e100g0:1 is the interface added by the non-global zone. (zone1)

e100g0:2 is the interface of global zone to access the zones.

Now we can acess this non-global zone via network. (Remind to configure ssh when applicable).

bash-3.00# ssh 192.168.50.2

Password:

Last login: Fri Jul 18 09:05:09 2008 from 192.168.50.1

Sun Microsystems Inc. SunOS 5.10 Generic January 2005

# uname -a

SunOS zone1 5.10 Generic_127128-11 i86pc i386 i86pc

Nice, you are connected into the non-global zone, so, I will show the filesystem configuration.

Use the following command to show the zone configuration:

bash-3.00# zonecfg -z zone1 export

create -b

set zonepath=/zone1

set autoboot=false

set ip-type=shared

add inherit-pkg-dir

set dir=/lib

end

add inherit-pkg-dir

set dir=/platform

end

add inherit-pkg-dir

set dir=/sbin

end

add inherit-pkg-dir

set dir=/usr

end

add net

set address=192.168.50.2

set physical=e1000g0

end

On Solaris Configuration Zone there are two types to configure the filesystem that is the Sparse Root and Whole Root.

Whole Root: Copy all the files required for the system from global-zone.(/usr/ , /lib and so on). The problem on whole root configuration is the space. It requires more space to run the Zones.

Sparse Root: No copy is executed. The main directories are mounted as loop back file system and read-only from the global zone. It helps to save space. That is the most common configuration that I would recommend.

Note in the zone configuration example above that the /usr, /lib, /platform and /sbin are configured on sparse mode (shared with Global Zone). That is the reason it was configured by “inherit-pkg-dir”.

Let’s take a look on the filesystems configured in our non-global zone:

bash-3.00# more /etc/mnttab

/ / ufs rw,intr,largefiles,logging,xattr,onerror=panic,dev=740081 1216324231

/dev /dev lofs zonedevfs,dev=4880003 1216382586

/lib /lib lofs ro,nodevices,nosub,dev=740000 1216382586

/platform /platform lofs ro,nodevices,nosub,dev=740000 1216382586

/sbin /sbin lofs ro,nodevices,nosub,dev=740000 1216382586

/usr /usr lofs ro,nodevices,nosub,dev=740000 1216382586

proc /proc proc nodevices,zone=zone1,dev=4640003 1216382587

ctfs /system/contract ctfs nodevices,zone=zone1,dev=4600004 1216382587

mnttab /etc/mnttab mntfs nodevices,zone=zone1,dev=4680004 1216382587

objfs /system/object objfs nodevices,zone=zone1,dev=4700004 1216382587

swap /etc/svc/volatile tmpfs nodevices,xattr,zone=zone1,dev=46c000a 1216382587

/usr/lib/libc/libc_hwcap1.so.1 /lib/libc.so.1 lofs zone=zone1,dev=740000 1216382589

fd /dev/fd fd rw,nodevices,zone=zone1,dev=48c0004 1216382593

swap /tmp tmpfs nodevices,xattr,zone=zone1,dev=46c000b 1216382593

swap /var/run tmpfs nodevices,xattr,zone=zone1,dev=46c000c 1216382593

-hosts /net autofs nosuid,indirect,ignore,nobrowse,zone=zone1,dev=4980005 1216382602

auto_home /home autofs nodevices,indirect,ignore,nobrowse,zone=zone1,dev=4980006 1216382602

Okay, you can see that /dev, /lib, /platform/, /sbin, /usr are filesystem configured using loop back file system, so it is shared by Global Zone.

The “/” filesystem on non-global zone (zone1) is mounted on the zonepath set during the zone installation. On this case, we defined that the zone1 use the “/zone1” filesystem on global zone as the non-global zone path. In other words, “/” filesystem in non-global zone is the “/zone1” on global zone.

Okay, the next step is to show how to add a new filesystem on global zone to be used on non-global zone as well.

Let’s suppose that /shared is the filesystem on global zone that we will share with non-global zones.

That is the df from global zone:

bash-3.00# uname -a

SunOS solaris-global 5.10 Generic_127128-11 i86pc i386 i86pc

bash-3.00# df -h

Filesystem size used avail capacity Mounted on

/dev/dsk/c1t0d0s0 4.6G 3.1G 1.5G 69% /

/devices 0K 0K 0K 0% /devices

ctfs 0K 0K 0K 0% /system/contract

proc 0K 0K 0K 0% /proc

mnttab 0K 0K 0K 0% /etc/mnttab

swap 647M 692K 647M 1% /etc/svc/volatile

objfs 0K 0K 0K 0% /system/object

/usr/lib/libc/libc_hwcap1.so.1

4.6G 3.1G 1.5G 69% /lib/libc.so.1

fd 0K 0K 0K 0% /dev/fd

swap 647M 548K 647M 1% /tmp

swap 647M 40K 647M 1% /var/run

/dev/dsk/c1t1d0s1 940M 453M 430M 52% /zone1

/dev/dsk/c1t2d0s1 940M 453M 430M 52% /zone2

/dev/dsk/c1t0d0s7 2.7G 2.8M 2.7G 1% /export/home

/dev/dsk/c1t3d0s1 4.0M 1.0M 2.6M 29% /shared

You can see that /shared filesystem is mounted on global zone, now we need to add this to non-global zone:

bash-3.00# zonecfg -z zone1

zonecfg:zone1> add fs

zonecfg:zone1:fs> set dir=/data

zonecfg:zone1:fs> set special=/shared

zonecfg:zone1:fs> set type=lofs

zonecfg:zone1:fs> end

zonecfg:zone1> commit

zonecfg:zone1> exit

bash-3.00# zoneadm -z zone1 reboot

bash-3.00# ssh 192.168.50.2

Password:

Last login: Fri Jul 18 11:40:51 2008 from 192.168.50.1

Sun Microsystems Inc. SunOS 5.10 Generic January 2005

# df -h

Filesystem size used avail capacity Mounted on

/ 940M 453M 430M 52% /

/data 4.0M 1.0M 2.6M 29% /data

/dev 940M 453M 430M 52% /dev

/lib 4.6G 3.1G 1.5G 69% /lib

/platform 4.6G 3.1G 1.5G 69% /platform

/sbin 4.6G 3.1G 1.5G 69% /sbin

/usr 4.6G 3.1G 1.5G 69% /usr

proc 0K 0K 0K 0% /proc

ctfs 0K 0K 0K 0% /system/contract

mnttab 0K 0K 0K 0% /etc/mnttab

objfs 0K 0K 0K 0% /system/object

swap 641M 300K 641M 1% /etc/svc/volatile

/usr/lib/libc/libc_hwcap1.so.1

4.6G 3.1G 1.5G 69% /lib/libc.so.1

fd 0K 0K 0K 0% /dev/fd

swap 641M 68K 641M 1% /tmp

swap 641M 12K 641M 1% /var/run

The first step was adding a new fs in the zone with some properties:

dir – Specify the directory in non-global zone.

Special – Specify the filesystem on global zone that will be used.

Type – Filesystem type. In this case, loop back filesystem (lofs).

After this we need to reboot the zone and connect to check if the filesystem was added to the non-global zone.

3. Setting a resource pool to a zone

In the first section on this article I create a pool zone called “zone1-pool”. The next step is to associate this pool to the zone1.

bash-3.00# zonecfg -z zone1

zonecfg:zone1> set pool=zone1-pool

zonecfg:zone1> verify

zonecfg:zone1> exit

Okay, now the resource pool is associated with the zone. We only need to reboot this.

bash-3.00# zoneadm -z zone1 reboot

Connect to the zone and check if it has only the processor assigned:

bash-3.00# hostname

zone1

bash-3.00# psrset

user processor set 1: processor 0

bash-3.00# psrinfo

0 on-line since 07/17/2008 16:50:08

4. Enabling Fair Share Scheduling on zones.

With FSS, the relative importance of applications is expressed by allocating CPU resources based on shares—a portion of the system’s CPU resources assigned to an application.

In our example, we have already configured the resource pool named zone1-pool and assigned this for zone1.

We also have already installed a second zone (zone2) to proceed with FSS enable example.

The first step we need to set the scheduler for the zone1-pool:

bash-3.00# poolcfg -c ‘modify pool zone1-pool (string pool.scheduler=”FSS”)’

Create a instance of the configuration:

bash-3.00# pooladm -c

Move all the processes in the pool and its associated zones under the FSS.

bash-3.00# priocntl -s -c FSS -i class TS

bash-3.00# priocntl -s -c FSS -i pid 1

Now we need to set zone1-pool into zone2 (We have already set zone1-pool to zone1 before).

bash-3.00# zonecfg -z zone2

zonecfg:zone2> set pool=zone1-pool

zonecfg:zone2> verify

zonecfg:zone2> exit

Now, both zone1 and zone2 are set to use zone1-pool and zone1-pool is assigned to use FSS.

With FSS, the relative importance of applications is expressed by allocating CPU resources based on shares—a portion of the system’s CPU resources assigned to an application. The larger the number of shares assigned to an application, the more CPU resources it receives from the FSS software relative to other applications. The number of shares an application receives is not absolute—what is important is how many shares it has relative to other applications, and whether they will compete with it for CPU resources.

Lets’ assign 3 shares to zone1 and 1 share to zone2.

bash-3.00# zonecfg -z zone1

zonecfg:zone1> add rctl

zonecfg:zone1:rctl> set name=zone.cpu-shares

zonecfg:zone1:rctl> add value (priv=privileged,limit=3,action=none)

zonecfg:zone1:rctl> end

zonecfg:zone1> exit

bash-3.00# zonecfg -z zone2

zonecfg:zone2> add rctl

zonecfg:zone2:rctl> set name=zone.cpu-shares

zonecfg:zone2:rctl> add value (priv=privileged,limit=1,action=none)

zonecfg:zone2:rctl> end

zonecfg:zone2> exit

In this example, we assigned 3 shares to zone1 and 1 share to zone2,then the total number of shares is 1+3 =4. In this example, CPU demand is allocated 75 percent for zone1 and 25 percent for zone2.

Note: Keep in mind that you should have more than one CPU assigned to have a good work of FSS.

According to Sun Manual (System Administrator Guide: Solaris Container-Resource Management and Solaris Zones), if you are using only one CPU the shares you specied are not relevant and each zone will use 50% of CPU. In this example we only provide a quick way to set FSS and Shares.

Tips:

How to monitor CPU usage of projects that run on the system:

bash-3.00# prstat -J

That’s it!Zone1 is fully configured and you can start to use this.

I hope this tutorial helps you to configure your zone.

I took the main steps to create a full zone, you can find great tips @ Sun web site.

If you have any question or comments (would like to add/change something on this article to improve its quality) please contact me.