Install ElasticSearch 1.3 on CentOS 6 for Graylog2 1.0

Graylog2 1.0 requires ElasticSearch 1.3 (or up). To fulfill this requirement, you will need to install ElasticSearch - I recommend on a separate server for environment growth. IOPS on your hard disk matters here.


First, make sure your CentOS 6 is fully patched:
#yum update


Then install Java 1.7:
#yum install java
Make sure that it prompts you to install Java 1.7 (look for the below text):
Installing:
 java-1.7.0-openjdk


Set up the ElasticSearch Repositories and install Elasticsearch 1.3:


Import they key:
#rpm --import https://packages.elasticsearch.org/GPG-KEY-elasticsearch


Create a new repository file in /etc/yum.repos.d/ named elasticsearch.repo with the following contents:
[elasticsearch-1.3]
name=Elasticsearch repository for 1.3.x packages
baseurl=http://packages.elasticsearch.org/elasticsearch/1.3/centos
gpgcheck=1
gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch
enabled=1


Install ElasticSearch:
#yum install elasticsearch


Add ElasticSearch to boot process:
#chkconfig --add elasticsearch

Stop the elasticsearch service so that we can update the cluster name:
#service elasticsearch stop



Edit the /etc/elasticsearch/elasticsearch.yml file to update your cluster.name variable. Ex:
cluster.name: graylog2_production

Update any additional settings needed and save the file. I recommend updating the path.data and path.logs to custom directories.



Start the elasticsearch service and set it to run on startup:
#service elasticsearch start
#chkconfig elasticsearch on



Check your logs to make sure that it started properly and joined the cluster (if there is an existing one).



For Graylog2, the recommended settings are also to increase the open file limit to at least 64000 as seen in the Configuring and tuning Elasticsearch documentation. I did this by increasing the max number of ulimit open file below.



Edit /etc/sysctl.conf and add the following line at the end:
fs.file-max = 65536



Save the file. Next edit /etc/security/limits.conf and add the following lines:
*               soft    nproc           65535
*               hard    nproc           65535
*               soft    nofile          65535
*               hard    nofile          65535



Save the file and restart the server.
#shutdown -r now



Once restarted, verify that the max open file ulimit has been increased.
# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 30507
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 65535
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 65535
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited



Additional recommended settings are to increase the ES_HEAP_SIZE. I did this by editing /etc/init.d/elasticsearch and adding the following line after checkJava under start():
ES_HEAP_SIZE=2g



They recommend that you leave 50% of your memory for other system functions, and I had 4 Gig of RAM, hence the 2g setting.



------
Dustin Shaw
VCP

Install ElasticSearch 0.90.10 on CentOS 6

Graylog2 0.20.x requires ElasticSearch 0.90.10. To fullfil this requirement, you will need to manually download and install the RPM for ElasticSearch.

Download ElasticSearch 0.90.10 from the ElasticSearch Downloads page here.

Save the file and upload it to your CentOS 6 server.

Install Java 1.7:
#yum install java-1.7.0-openjdk.x86_64

Install the RPM:
#rpm -ivh elasticsearch-0.90.10.noarch.rpm

Stop the elasticsearch service so that we can update the cluster name:
#service elasticsearch stop

Edit the /etc/elasticsearch/elasticsearch.yml file to update your cluster.name variable. Ex:
cluster.name: graylog2_production

Update any additional settings needed and save the file. I recommend updating the path.data and path.logs to custom directories.

Start the elasticsearch service and set it to run on startup:
#service elasticsearch start
#chkconfig elasticsearch on

Check your logs to make sure that it started properly and joined the cluster (if there is an existing one).

For Graylog2, the recommended settings are also to increase the open file limit to at least 64000 as seen in the Configuring and tuning ElasticSearch for Graylog2 >v0.20.0 documentation. I did this by increasing the max number of ulimit open file below.

Edit /etc/sysctl.conf and add the following line at the end:
fs.file-max = 65536

Save the file. Next edit /etc/security/limits.conf and add the following lines:
*               soft    nproc           65535
*               hard    nproc           65535
*               soft    nofile          65535
*               hard    nofile          65535

Save the file and restart the server.
#shutdown -r now

Once restarted, verify that the max open file ulimit has been increased.
# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 30507
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 65535
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 65535
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Additional recommended settings are to increase the ES_HEAP_SIZE. I did this by editing /etc/init.d/elasticsearch and adding the following line after checkJava under start():
ES_HEAP_SIZE=2g

They recommend that you leave 50% of your memory for other system functions, and I had 4 Gig of RAM, hence the 2g setting.

------
Dustin Shaw
VCP

Ship RHEL 6 or CentOS6 syslogs

Reference more for myself than for y'all, but you get the shared benefit.

To ship the logs from RHEL6 or Centos 6 to a remote syslog server, edit the following file: /etc/rsyslog.conf


At the bottom of the file, remove the comment from the remote-host entry, and update with your server name or IP. Example:
*.* @@192.168.40.15:514

Restart the rsyslog service:
service rsyslog restart

Make sure you are receiving the logs at your syslog server.

------
Dustin Shaw
VCP

Install VMware Tools on RHEL 6 or CentOS 6

Below is the process to install VMware Tools on RHEL 6 or CentOS 6. This guide is more here for me than anyone else, but I hope that you can benefit from it.


Install the Pre-Requisites:
yum install make gcc kernel-devel kernel-headers glibc-headers perl


Start the VMware Tools installation process on your VM:





Mount the VMware Tools installation media:
mkdir /mnt/cd
mount /dev/cdrom /mnt/cd
Expected warning:
mount: block device /dev/sr0 is write-protected, mounting read-only


Extract the installer:
cp /mnt/cd/VMwareTools-9.0.10-1481436.tar.gz /tmp/
umount /mnt/cd
cd /tmp
tar xvf VMwareTools-9.0.10-1481436.tar.gz
cd vmware-tools-distrib/


Install tools (accepting all defaults):
sudo ./vmware-install.pl -d

Reboot the VM to verify that the service starts up automatically as expected.
shutdown -r now


------
Dustin Shaw
VCP

Extend a volume in RedHat 6 or CentOS 6

To extend a partition on a VM on CentOS 6 or RHEL 6, do the following:

Extend the Partition in vSphere to the size that you need:

in the VM, refresh the Partition Tables by the following command:

echo 1 > /sys/block/sda/device/rescan
Run fdisk to create a new primary partition:

fdisk /dev/sda

Press p to print the number of partitions
Press n to create a new primary patition
Press p for primary
The partition number should automatically be selected, and press enter twice to accept the beginning and ending blocks on the free space.
Press w to write the changes.

Reboot the system: shutdown -r now

Verify that the new partition exists:
fdisk -l

Convert the new partition to a physical volume:
pvcreate /dev/sda3
Extend the Physical Volume (run vgdisplay to obtain the name of the Volume Group; default is VolGroup00):
vgextend VolGroup00 /dev/sda3

Extend the Logical Volume (run vgdisplay to obtain the free space and replace # below, and lvdisplay to obtain the name of the Logical Volume; default is /dev/VolGroup00/LogVol00)
lvextend -L+#G /dev/VolGroup00/LogVol00

Expand the ext3 online using the following command (substituting your proper Volume Group and Logical Volume names):
resize2fs /dev/VolGroup00/LogVol00

Confirm that your new space is available:
df -h



This was written using VMware's KB article on the subject as a reference.

------
Dustin Shaw
VCP