Daemon Service for Graphite

A daemon is a program running in the background, usually providing some sort of service.” enabling other work in the “foreground,” and is independent of control from a terminal. Typical daemons that provide e-mail, printing, telnet, FTP, and web access.

Daemons can either be started by a process, such as a system startup script, where there is no controlling terminal, or by a user at a terminal without “tying up” that terminal

Here we used this technic in Graphite to automate the servers of the Graphite. So we plan to start carbon and graphite server automatically using daemon.

The daemons referenced in /etc/init.d are configured to be run as Linux services. Services are programs that are started and stopped through the init scripts in the /etc/init.d directory. Many of these services are launched when the system is booted. The /sbin/service utility provides a consistent interface to executing the init scripts. The init scripts provide a consistent interface to managing a service by providing options that start, stop, restart, query status.

RRDTOOL for Graphing

RRDtool is the OpenSource industry standard, high performance data logging and graphing system for time series data. RRDtool can be easily integrated in shell scripts, perl, python, ruby, lua or tcl applications.

It has the similar features as Graphite graphing. This is very use full in for industrial level approaches .

Also,

• RRDtool stores data; that makes it a back-end tool. The RRDtool command set allows the creation of graphs; that makes it a front-end tool as well. Other databases just store data and can not create graphs.

• In case of linear databases, new data gets appended at the bottom of the database table. Thus its size keeps on increasing, whereas the size of an RRDtool database is determined at creation time. Imagine an RRDtool database as the perimeter of a circle. Data is added along the perimeter. When new data reaches the starting point, it overwrites existing data. This way, the size of an RRDtool database always remains constant. The name "Round Robin" stems from this behaviour.

• Other databases store the values as supplied. RRDtool can be configured to calculate the rate of change from the previous to the current value and store this information instead.

• Other databases get updated when values are supplied. The RRDtool database is structured in such a way that it needs data at predefined time intervals. If it does not get a new value during the interval, it stores an UNKNOWN value for that interval. So, when using the RRDtool database, it is imperative to use scripts that run at regular intervals to ensure a constant data flow to update the RRDtool database.

RRDtool assumes time-variable data in intervals of a certain length. This interval, usually named step, is specified upon creation of an RRD file and cannot be changed afterwards. Because data may not always be available at just the right time, RRDtool will automatically interpolate any submitted data to fit its internal time-steps.

The value for a specific step, that has been interpolated, is named a primary data point (PDP). Multiple PDPs may be consolidated according to a consolidation function (CF) to form a consolidated data point (CDP). Typical consolidation functions are average, minimum, maximum.

After the data have been consolidated, the resulting CDP is stored in a round-robin archive (RRA). A round-robin archive stores a fixed number of CDPs and specifies how many PDPs should be consolidated into one CDP and which CF to use.

Reference :

http://en.wikipedia.org/wiki/RRDtool

http://oss.oetiker.ch/rrdtool/index.en.html

Graphite Custom page Creation

We have some procedures to Create the custom page for Graphite. The brief steps are,

•    Define the URL path

•    Create the Sample page Directory

•    Create the Sample page HTML view

•    Link the HTML with specific folder

•    Call the Html from specific folder using Python

Step 1:

We have to set the URL path in urls.py file which is located in /opt/graphite/webapp/graphite/urls.py . In this file we have some set of line which contain the url paths . So inside of this line we have to add our own web path with the same pattern.

Example: If we have a Directory called Test (With my own functions) then we have to add a line in this file url.py with the same pattern as below,

('^Test/', include('graphite.Test.urls')),

^Test/                                  : This is the URL extension (What we type after localhost)

graphite.Test.urls : Calling the python URL file which is in graphite/test/

Note : URL extensions are Case sensitive .To avoid the complex make all links as lower case

Step 2:

Now we have to create the sample Directory. So to avoid the complex of the implementation we can copy the browser directory which is located in /opt/graphite/webapp/graphite/ and rename the browser directory as Test. Paste it in same location (/opt/graphite/webapp/graphite/) So the Test Directory will contain the same files, which are in default page. Now we can edit the Test folder which contain url.py and views.py files. Each file we have some small changes.

The /opt/graphite/webapp/graphite/test directory contain url.py and views.py. Now change the graphite.browser.views as graphite.test.views to access our own page.

Similar Change the browser.html page as test.html in views.py file.

/opt/graphite/webapp/graphite/templates/test.html

Step 3:

Now we have to Create the HTML page for test with that what we mention above .So for that we have to create html file in /opt/graphite/webapp/graphite/templates/ as test.html.

Step 4:

Design the Page with your own Specification.

So finally we have to work with only three files for custom page creation

• urls.py

• views.py

• test.html

Source Files

Done.

Graphite : Real Time Graphing Installation

Installed packages has been updated.
sudo apt-get update

Supported libraries has been installed using below command,Also this will installe the django as well
sudo apt-get install --assume-yes apache2 apache2-mpm-worker apache2-utils apache2.2-bin apache2.2-common libapr1 libaprutil1 libaprutil1-dbd-sqlite3 python3.2 libpython3.2 python3.2-minimal libapache2-mod-wsgi libaprutil1-ldap memcached python-cairo-dev python-django python-ldap python-memcache python-pysqlite2 sqlite3 erlang-os-mon erlang-snmp rabbitmq-server bzr expect ssh libapache2-mod-python python-setuptools

The applications has been downloaded,
Carbon
Graphite
Whisper

 To install whisper
 cd ~/whisper  
 sudo python setup.py install  

 To install Carbon
 cd ~/carbon  
 sudo python setup.py install  

 To configure Carbon  
 cd /opt/graphite/conf  
 sudo cp carbon.conf.example carbon.conf  
 sudo cp storage-schemas.conf.example storage-schemas.conf  

 To set duration of the Graphite
 sudo gedit storage-schemas.conf  
 Edit the data with below test,  
 [stats]  
 priority = 110  
 pattern = .*  
 retentions = 10:2160,60:10080,600:262974  
 Graphite has been installed after check the dependencies , 
 If you it give an error with dependencies then fix and 
 go forward .sudo python check-dependencies.py  
 sudo python setup.py install  

 Apache server has been configured for graphite,  
 cd ~/graphite/examples  
 sudo cp example-graphite-vhost.conf /etc/apache2/sites-available/default  
 sudo cp /opt/graphite/conf/graphite.wsgi.example /opt/graphite/conf/graphite.wsgi  
 sudo vim /etc/apache2/sites-available/default  
 sudo mkdir /etc/httpd  
 sudo mkdir /etc/httpd/wsgi  
 sudo /etc/init.d/apache2 reload  

 Initial database has been created for graphite,  
 cd /opt/graphite/webapp/graphite/  
 sudo python manage.py syncdb  

 Here give the user name and password for you database  
 sudo chown -R nobody:nobody /opt/graphite/storage/  
 sudo /etc/init.d/apache2 restart  

 cd /opt/graphite/webapp/graphite  
 sudo cp local_settings.py.example local_settings.py  

 Start the Carbon  
 cd /opt/graphite/  
 sudo ./bin/carbon-cache.py start  

 Send the data to the Graphite  
 cd ~/graphite/examples  
 sudo chmod +x example-client.py  
 sudo ./example-client.py  

 Go to web browser and launched localhost .  
 Done.   

Graphiti: An Alternate Frontend for Graphite

Graphite is a powerful front end for the real-time graphing engine. Graphiti provides a whole new way of creating and storing graphs. This was using in day-to-day at Paperless as a way to monitor the infrastructure and product metrics. 

There’s been a big push in the development community on the need and desire for metrics to drive the business, product, and development cycles. Therefore it will definitely help full to the organisation which are based on  analysis and multi task processes.

Accounting data is not real-time, but it is precise. It’s generated nightly by some very awesome and complex SQL functions and collations by our venerable analytics lead. These numbers are hard facts and they give the information  to make important big decisions.

Difference between D3.js and Graphiti
D3 allows you to bind arbitrary data to a Document Object Model (DOM), and then apply data-driven transformations to the document. This can use D3 to generate an HTML table from an array of numbers. Or, use the same data to create an interactive SVG bar chart with smooth transitions and interaction.

So D3.js contain many kind of visualisation view something like table view, tree maps, bubble charts and more.But Graphiti is not contain that kind of fancy things. It's only the contain the specialised graph visualisation .

Therefore Graphiti is more use full when you are going to deal with some kind of analysis and comparing projects 

Reference