Last week I wrote about
creating our own Vagrant base box, based on the new and fresh Oracle Linux 7 Update 5. As a reaction on my article I got noted that Oracle also keeps base boxes for their latest linuxes at
http://yum.oracle.com/boxes. They're also a great start.
To begin, I created a project structure for all my vagrant projects:
As you can see, I have a vagrant main projects folder on my D: drive,
D:\Projects\vagrant. Besides my actual vagrant projects, like
oel74, oel74_wls12c etc., I have a
boxes folder and a
Stage folder. The
boxes folder, as you can guess, contains my base boxes:
The
Stage folder contains all my installation-binaries, for database, Weblogic, Java, FusionMiddleware and so on.
Today I want to create a basic VM that will be a base for further VM's, like database, weblogic, etc.
I want the VM to have:
- Linux prepared with correct kernel settings for database, FusionMiddleware, etc.
- Filesystem created on a second disk. I did not add a second disk to the base box, only a root disk. Thus we need to extend the VM with one.
- Create an oracle user that can sudo.
Initialize a vagrant project
Begin with creating a folder, like
ol75 in the structure, for this project. Open a command window and navigate to it:
Microsoft Windows [Version 10.0.16299.371]
(c) 2017 Microsoft Corporation. All rights reserved.
C:\Windows\system32>d:
D:\>cd d:\Projects\vagrant\ol75\
d:\Projects\vagrant\ol75>vagrant help init
==> vagrant: A new version of Vagrant is available: 2.0.4!
==> vagrant: To upgrade visit: https://www.vagrantup.com/downloads.html
Usage: vagrant init [options] [name [url]]
Options:
--box-version VERSION Version of the box to add
-f, --force Overwrite existing Vagrantfile
-m, --minimal Use minimal Vagrantfile template (no help comments). Ignored with --template
--output FILE Output path for the box. '-' for stdout
--template FILE Path to custom Vagrantfile template
-h, --help Print this help
d:\Projects\vagrant\ol75>
The vagrant command
init creates a new vagrant project, with a so called
Vagrantfile in it. By default, you'll get a
Vagrantfile with the most common settings and comments explaining the most common additional settings. But using the
-m or
--minimal setting a
just enough vagrant file is created, without comments. I do like a file with the most common settings in the comments, as it allows me to quickly extend it without having to lookup every thing. So I create a basic file:
d:\Projects\vagrant\ol75>vagrant init
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.
d:\Projects\vagrant\ol75>dir /w
Volume in drive D is DATA
Volume Serial Number is 62D7-9456
Directory of d:\Projects\vagrant\ol75
[.] [..] Vagrantfile
1 File(s) 3,081 bytes
2 Dir(s) 651,847,397,376 bytes free
Now, let's expand the file bit by bit. So, open it in your favorite ASCII editor, like Notepad++, for instance.
# -*- mode: ruby -*-
# vi: set ft=ruby :
# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.
Vagrant.configure("2") do |config|
# The most common configuration options are documented and commented below.
# For a complete reference, please see the online documentation at
# https://docs.vagrantup.com.
# Every Vagrant development environment requires a box. You can search for
# boxes at https://vagrantcloud.com/search.
config.vm.box = "base"
A vagrant file has about the following format:
Vagrant.configure("2") do |config|
# ...
end
Apparently you can have multiple occurences with multiple configuration versions ("1" refers to Vagrant 1.0 configuration, "2" to 1.1 and onwards).
As with many other scripted languages, I find it convenient to have all the configurable values declared as global variables at the top of the file. So let's declare some:
# -*- mode: ruby -*-
# vi: set ft=ruby :
#
BOX_NAME="ol75"
BOX_URL="file://../boxes/OL75v1.0.box"
VM_MEMORY = 12288 # 12*1024 MB
VM_CPUS=4
VMS_HOME="d:\\VirtualMachines\\VirtualBox"
VM_NAME="OL7U5"
VM_DISK2=VMS_HOME+"\\"+VM_NAME+"\\"+VM_NAME+".disk2.vdi"
VM_DISK2_SIZE=1024 * 512
# Stage folders
STAGE_HOST_FOLDER="d:/Projects/vagrant/Stage"
STAGE_GUEST_FOLDER="/media/sf_Stage"
Then find the line ‘Vagrant.configure("2") do |config|’. This
starts the block that configures the box. All the following configuration is done between that line and the corresponnding closing end line.
Start with renaming the box based on the global variable:
config.vm.box = BOX_NAME
And add the following lines:
config.vm.box_url=BOX_URL
config.vm.define "darwin"
The
config.vm.box_url directive refers to the base box used. Often this is just the name of the box that is automatically downloaded from the Hashicorp/Vagrant repository. You can also use a URL to a box on internet, it will be downloaded automatically. But we created our own box, so that we exactly know and control what's in it. It does not need to be downloaded, it's available right away.
Side note: manage your boxes
At start up this box is added to your local Vagrant box repository. You can see the boxes in your repository with the command
box list:
d:\Projects\vagrant\ol75>vagrant box list
OL7U4 (virtualbox, 0)
OL7U4-1.1b (virtualbox, 0)
Ubuntu16.0.4LTS (virtualbox, 0)
d:\Projects\vagrant\ol75>
These boxes are found in the
.vagrant.d/boxes folder your user profile:
My new ol75 box is not added yet, as you can see. It will be done at first
up. Since they're eating up costly space on my SSD drive, it's sensible to remove unused boxes. For instance, the
OL7U4 one is superseded by the
OL7U4-1.1b. The Ubuntu one I still use, although the Ubuntu version is a bit old. So, I should at least remove the OL7U4 one. It can be done with the command
box remove ${name}:
d:\Projects\vagrant\ol75>vagrant box remove OL7U4
Removing box 'OL7U4' (v0) with provider 'virtualbox'...
d:\Projects\vagrant\ol75>vagrant box list
OL7U4-1.1b (virtualbox, 0)
Ubuntu16.0.4LTS (virtualbox, 0)
d:\Projects\vagrant\ol75>
Support for multi-machine boxes
The other line I added was the
config.vm.define directive. This one allows to define multi-machine definitions in one Vagrant project. It can come in handy when you have a project where one VM is servicing your database, while another is doing your front-end application. You can have them started and provisioned automatically, where the provisioning can be done in stages. Or you can switch off auto-start for certain machines and start those explicitly. It's a nice-to-know, but I'll leave it for now. I use it for noting the machine in the provisioning logs.
Read more about multi-machine configs
here.
SSH Vagrant User
Below the
config.vm.* lines, within the
config block, you can add some config lines for the vagrant username/password:
config.ssh.username="vagrant"
config.ssh.password="vagrant"
config.ssh.port=2222
The password line should be optional, since we injected a key for the vagrant user. You'll see that Vagrant will replace that key. The
ssh.port will direct Vagrant to create a port-forwarding for the local port 2222 to the ssh port on the vm.
Provider config
Within the config block, below the line
config.fm.define add the following block:
config.vm.provider :virtualbox do |vb|
vb.name = VM_NAME
vb.gui = true
vb.memory = VM_MEMORY
vb.cpus = VM_CPUS
# Set clipboard and drag&drop bidirectional
vb.customize ["modifyvm", :id, "--clipboard", "bidirectional"]
vb.customize ["modifyvm", :id, "--draganddrop", "bidirectional"]
# Create a disk
unless File.exist?(VM_DISK2)
vb.customize [ "createmedium", "disk", "--filename", VM_DISK2, "--format", "vdi", "--size", VM_DISK2_SIZE , "--variant", "Standard" ]
end
# Add it to the VM.
vb.customize [ "storageattach", :id , "--storagectl", "SATA", "--port", "2", "--device", "0", "--type", "hdd", "--medium", VM_DISK2]
end
This block begins setting the following properties:
Property
|
Meaning
|
| vb.name |
Name of the VM to create. This is how the VM will appear in VirtualBox. |
| vb.gui |
This toggles the appearance of the UI of the VM. If false, it is started in the background. It is then only reachable through the network settings. Using the VirtualBox manager the GUI can then be brought to appear using the show button. |
| vb.memory |
This sets the memory available to the VM. In the global variables I have set it to 12GB (12*1024 MB) |
| vb.cpus |
Number of CPU cores availabe to the VM. In the globals I have set it to 4. |
Using the
customize provider command you can change the VM's configuration. It is in fact an
API to the VboxManage utility of VirtualBox.
With
modifyvm we set both the properties
--clipboard and
--draganddrop to
bidirectional. When the GUI is shown (
vb.gui = true) then it allows us to copy and paste into the VM for instance.
Then using the
createmedium command a standard
vdi disk is created. The name
VM_DISK2 is based on the variables
VMS_HOME and
VM_NAME. See the top of the file. It's convenient to have the file created in the same folder as the VM is created in. So check the VirtualBox preferences:
The
Default Machine Folder preference is used to create the VM in, in a subfolder indicated by the name of the VM. So make sure that the
VMS_HOME variable matches the value in that preference.
The value for
--format the
createmedium command I used is
vdi. This is the default
Virtual Disk Image format of VirtualBox. When exporting a VM into an OVA (
Open Virtual Appliance package, which is a
tar archive file) the VDI disks are converted to the
VMDK (Virtual Machine Disk).
The size is set with the
--size parameter, in my example set to the
VM_DISK2_SIZE that I created in the top of the file as 512GB (1024 * 512).
The
--variant Standard indicates a dynamically allocated file, that grows with the filling of it. So, you won't loose the complete filesize on diskspace. The 512GB limits the growth of the disk.
I want the disk to persist and not recreated at every startup, so I surrounded that command with the
unless File.exist?(VM_DISK2) block.
Using the
storageattach I add it to my SATA controller using the
--storagectl SATA parameters. It's added to
--port 2 and
--device 0, since it is my second drive. It needs to appear secondly in Linux. Then ofcourse the
--type is
hdd and the
--medium is
VM_DISK2.
You see a special variable
:id. This refers to the VM that is created in VirtualBox. Of course I want the disk attached to the proper VM.
Shared/Synced folders
Vagrant by default creates a folder-link, a so-called
Synced Folder, the wrapper around VirtualBox’s Shared Folder functionality. The default refers to the Vagrant project folder from which the VM is created and provisioned. Thus, in fact the folder where the
Vagrantfile resides. That folder is mounted in the VM as
/vagrant. So, navigating to the
/vagrant folder in the VM will show you the files from the vagrant-project folder and child folders. This is convenient, because subfolders in that folder, for instance a
scripts folder with provisioning scripts, are immediately available at startup.
Since we want to install software from the
Stage folder on our host, we need a mapping to that.
So find the following line:
# config.vm.synced_folder "../data", "/vagrant_data"
This is an example line for configuring additional synced folders.
Add a line below it as follows:
config.vm.synced_folder STAGE_HOST_FOLDER, STAGE_GUEST_FOLDER
This maps the folder on the host as denoted in the global variable
STAGE_HOST_FOLDER, and then mount that as the value from the global
STAGE_GUEST_FOLDER. Notice that I doing so I map the folder
/media/sf_Stage in the VM to the folder
d:/Projects/vagrant/Stage on the host. Wich is a sub-folder in my main vagrant project folder.
Provisioning
Having the VM configured, the provisioning part is to be configured. Vagrant allows for several provisioners like
Puppet, Chef, Ansible, Salt, and
Docker. But a
Shell snippet is provided in our
Vagrantfile. I'll expand that one. At the bottom of the file, right above the closing
end of our configure block, we'll find the snippet:
# config.vm.provision "shell", inline: <<-SHELL
# apt-get update
# apt-get install -y apache2
# SHELL
Replace it with the following block:
config.vm.provision "shell", inline: <<-SHELL
export SCRIPT_HOME=/vagrant/scripts
echo _______________________________________________________________________________
echo 0. Prepare Oracle Linux
$SCRIPT_HOME/0.PrepOEL.sh
echo _______________________________________________________________________________
echo 1. Create Filesystem
$SCRIPT_HOME/1.FileSystem.sh
echo _______________________________________________________________________________
echo 2. Create Oracle User
$SCRIPT_HOME/2.MakeOracleUser.sh
#
SHELL
This provides an inline script. I like that because it allows me to see directly what happens in helicopter view. But, I do not like to have all the detailed steps in here, so I only call sub-scripts within this block. You can also uses external scripts, even remote ones, as described in the
doc.
As can be seen the scripts I'll describe below have to be placed in the
scripts folder
as part of the vagrant project folder. Remember that this folder is mapped as a
synched folder to
/vagrant in the VM.
I have the following script, called
0.PrepOEL.sh, to update the Oracle Linux installation in the VM:
#!/bin/bash
SCRIPTPATH=$(dirname $0)
#
. $SCRIPTPATH/install_env.sh
echo Installing packages required by the software
sudo yum -q -y install compat-libcap1* compat-libstdc* libstdc* gcc-c++* ksh libaio-devel* dos2unix system-storage-manager
echo install Haveged
sudo rpm -ihv $STAGE_HOME/Linux/haveged-1.9.1-1.el7.x86_64.rpm
echo 'Adding entries into /etc/security/limits.conf for oracle user'
if grep -Fq oracle /etc/security/limits.conf
then
echo 'WARNING: Skipping, please verify!'
else
echo 'Adding'
sudo sh -c "sed -i '/End of file/i # Oracle account settings\noracle soft core unlimited\noracle hard core unlimited\noracle soft data unlimited\noracle hard data unlimited\noracle soft memlock 3500000\noracle hard memlock 3500000\noracle soft nofile 1048576\noracle hard nofile 1048576\noracle soft rss unlimited\noracle hard rss unlimited\noracle soft stack unlimited\noracle hard stack unlimited\noracle soft cpu unlimited\noracle hard cpu unlimited\noracle soft nproc unlimited\noracle hard nproc unlimited\n' /etc/security/limits.conf"
fi
echo 'Changing /etc/sysctl.conf'
if grep -Fq net.core.rmem_max /etc/sysctl.conf
then
echo 'WARNING: Skipping, please verify!'
else
echo 'Adding'
sudo sh -c "echo '
#ORACLE
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 4294967295
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 4194304
'>>/etc/sysctl.conf"
/sbin/sysctl -p
fi
It uses the following script,
install_env.sh:
#!/bin/bash
echo set Install environment
export STAGE_HOME=/media/sf_Stage
export SCRIPT_HOME=/vagrant/scripts
to set a few HOME variables.
It first install required packages using
sudo yum . These packages are required for most of the Oracle software setup, like database and/or FusionMiddleware.
Then from the
Linux folder in the
STAGE_HOME folder, the tool
Haveged is installed. You can download it from the
Oracle yum repository. So, I should be able to have it installed with
yum as well. One improvement point noted.
Why
Haveged you ask? On non-gui, terminal only virtualized systems, the entropy maybe low, which causes slow encryption/decryptions. Installing and configuring FusionMiddleware, for instance, or starting WebLogic maybe very slow.
Then it set some limits for the oracle user and kernel configs. These can be found in the install guides of the Oracle software.
To create the file system I use the script
1.FileSystem.sh:
#!/bin/bash
echo Create folder for mountpoint /app
sudo mkdir /app
echo Create a Logical Volume group and Volume on sdb
sudo ssm create -s 511GB -n disk01 --fstype xfs -p pool01 /dev/sdb /app
sudo ssm list
sudo sh -c "echo \"/dev/mapper/pool01-disk01 /app xfs defaults 0 0\" >> /etc/fstab"
This one creates a folder
/appfor the file mount. Then it uses
ssm (System Storage Manager) to create a 511GB (because of overhead I can't create a filesystem of 512GB) filesystem using a Logical Volume in a Logical Volume Group on the
sdb device in Linux. For more info on how this works, read my
blog article on it.
That leaves us to create an oracle user. For that I use the script
2.MakeOracleUser.sh:
#!/bin/bash
#
# Script to create a OS group and user
#
SCRIPTPATH=$(dirname $0)
ENV=${1:-dev}
function prop {
grep "${1}" $SCRIPTPATH/makeOracleUser.properties|cut -d'=' -f2
}
# As we are using the database as well, we need a group named dba
echo Creating group dba
sudo /usr/sbin/groupadd -g 2001 dba
# We also need a group named oinstall as Oracle Inventory group
echo create group oinstall
sudo /usr/sbin/groupadd -g 2000 oinstall
#
# Create the Oracle user
echo Create the oracle user
sudo /usr/sbin/useradd -u 2000 -g oinstall -G dba oracle
echo Setting the oracle password to...
sudo sh -c "echo $(prop 'oracle.password') |passwd oracle --stdin"
sudo chown oracle:oinstall /app
#
# Add Oracle to sudoers so he can perform admin tasks
echo Adding oracle user to sudo-ers.
sudo sh -c "echo 'oracle ALL=NOPASSWD: ALL' >> /etc/sudoers"
#
# Create oraInst.loc and grant to Oracle
echo Create oraInventory folder
sudo chown -R oracle:oinstall /app
sudo mkdir -p /app/oracle/oraInventory
sudo chown -R oracle:oinstall /app/oracle
echo Create oraInst.loc and grant to Oracle
sudo sh -c "echo \"inventory_loc=/app/oracle/oraInventory\" > /etc/oraInst.loc"
#sudo sh -c "echo \"\" > /etc/oraInst.loc"
sudo sh -c "echo \"inst_group=oinstall\" >> /etc/oraInst.loc"
sudo chown oracle:oinstall /etc/oraInst.loc
It uses a property file
makeOracleUser.properties for the oracle password:
oracle.password=welcome1
Using the
prop function this property is read.
The groups
dba and
oinstall are created. Then the
oracle user is created and it's password set. The filesystem mounted on
/app is assigned to the
oracle user to own. And then the user is added to tue
/etc/sudoers file.
Lastly the
oraInventory and the
oraInst.loc file are created.
Up, up and up it goes…!
If everything went alright, you’re now ready to fire up your VM.
So open a command window and navigate to your vagrant project folder, if not done already.
Then simply issue the following command:
d:\Projects\vagrant\ol75>vagrant up
And then you wait…. And watch carefully to see that Vagrant imports the box, creates the VM, and provisions it.
Some other helpfull commands
I'll finish with some other helpfull commands:
Command
|
Meaning
|
| vagrant up |
Start a VM, and provision it the first time. |
| vagrant halt |
Stop the VM. |
| vagrant suspend |
Remove the VM. |
| vagrant destroy |
Number of CPU cores availabe to the VM. In the globals I have set it to 4. |
| vagrant box list |
Lists the base boxes in your repository. |
| vagrant box remove |
Remove a listed base box from your repository. |
| vagrant package --base <VM Name> --output <box filename> |
Package the VM <VM Name> from the provider (VirtualBox) into a base box with given <box filename>. |
Next stop: installing software as an oracle (or othernon-vagrant) user user.
