My setup
Host OS: Ubuntu 12.04 LTS
KVM: QEMU emulator version 1.5.0 (Debian 1.5.0+dfsg-3ubuntu5.1)
VM OS: Ubuntu Quantal Quetzal 12.04 with libvirtd
Create a VM in KVM
With the following characteristics:
CPU: 1 vCPU
RAM: 1 GB
HDD: 14 GB
we are going to use Virtual Machine Manager, in particular, "Create a new virtual machine" to create a virtual machine (VM) with name metricsGrimoireVM, and choosing "local install media(ISO image or CDROM" in the step 1.
In the step 3, we are going to choose memory 1 GB and 1 vCPU.
In the step 4, it is necessary to create a disk image for storage, in this cases, 14 GB, enough to save several repositories. For instance, Linux uses around 2 GB. We use the command fallocate to create a fully-allocated (non-sparse) raw file:
root@gon:/media/janague/vms# fallocate -l 14G metricsGrimoireDisk.img
In the step 5, a configuration summary
Install Ubuntu Server 12.04 LTS
In the the Ubuntu Server Guide in chapter 2 is detailed installation process.
https://help.ubuntu.com/12.04/serverguide/serverguide.pdf
hostname: metricsGrimoireHost
Besides, it is important in the installation process to install openSSH server.
To make easier access, we set a static IP on metricsGrimoreHost, editing file /etc/network/interfaces
auto lo eth0 iface lo inet loopback iface eth0 inet static address 192.168.122.101 netmask 255.255.255.0 gateway 192.168.122.1
and in the host server in /etc/hosts file
# MSWL - Project Evaluation 192.168.122.101 metricsGrimoireHost
To create your public and private SSH keys to access in secure mode, use this link
https://help.ubuntu.com/community/SSH/OpenSSH/Keys
Install git
janague@metricsGrimoireHost:~$ sudo apt-get install git Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: git-man liberror-perl Suggested packages: git-daemon-run git-daemon-sysvinit git-doc git-el git-arch git-cvs git-svn git-email git-gui gitk gitweb The following NEW packages will be installed git git-man liberror-perl 0 upgraded, 3 newly installed, 0 to remove and 66 not upgraded. Need to get 6,741 kB of archives. After this operation, 15.2 MB of additional disk space will be used. ... Unpacking git (from .../git_1%3a1.7.9.5-1_amd64.deb) ... Processing triggers for man-db ... Setting up liberror-perl (0.17-1) ... Setting up git-man (1:1.7.9.5-1) ... Setting up git (1:1.7.9.5-1) ...
Install Python
janague@metricsGrimoireHost:~$ sudo apt-get install python3 Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: python3-minimal python3.2 python3.2-minimal Suggested packages: python3-doc python3-tk python3.2-doc binutils binfmt-support The following NEW packages will be installed python3 python3-minimal python3.2 python3.2-minimal 0 upgraded, 4 newly installed, 0 to remove and 66 not upgraded. Need to get 4,355 kB of archives. After this operation, 14.7 MB of additional disk space will be used.
Install MariaDB
Here are the commands to run to add MariaDB to your system:
1.- Install the repo manager
janague@metricsGrimoireHost:~$ sudo apt-get install python-software-properties Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: python-pycurl unattended-upgrades Suggested packages: libcurl4-gnutls-dev python-pycurl-dbg bsd-mailx The following NEW packages will be installed python-pycurl python-software-properties unattended-upgrades 0 upgraded, 3 newly installed, 0 to remove and 66 not upgraded. Need to get 97.6 kB of archives. After this operation, 657 kB of additional disk space will be used. ... Setting up python-pycurl (7.19.0-4ubuntu3) ... Setting up python-software-properties (0.82.7.6) ...
2.- Import the GnuPG signing key
janague@metricsGrimoireHost:~$ sudo apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0xcbcb082a1bb943db Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --secret-keyring /tmp/tmp.EjYqLFa95L --trustdb-name /etc/apt/trustdb.gpg --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0xcbcb082a1bb943db gpg: requesting key 1BB943DB from hkp server keyserver.ubuntu.com gpg: key 1BB943DB: public key "MariaDB Package Signing Key <package-signing-key@mariadb.org>" imported gpg: no ultimately trusted keys found gpg: Total number processed: 1 gpg: imported: 1 janague@metricsGrimoireHost:~$ sudo add-apt-repository 'deb http://ftp.igh.cnrs.fr/pub/mariadb/repo/5.5/ubuntu precise main'
4.- Refresh your system
janague@metricsGrimoireHost:~$ sudo apt-get update Hit http://es.archive.ubuntu.com precise Release.gpg Hit http://es.archive.ubuntu.com precise-updates Release.gpg Hit http://es.archive.ubuntu.com precise-backports Release.gpg Hit http://es.archive.ubuntu.com precise Release ... Hit http://es.archive.ubuntu.com precise-backports/multiverse Translation-en Hit http://es.archive.ubuntu.com precise-backports/restricted Translation-en Hit http://es.archive.ubuntu.com precise-backports/universe Translation-en Fetched 11.5 kB in 1s (6,567 B/s) Reading package lists... Done
5.- And finally install MariaDB
janague@metricsGrimoireHost:~$ sudo apt-get install mariadb-server Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: libaio1 libdbd-mysql-perl libdbi-perl libhtml-template-perl libmariadbclient18 libmysqlclient18 libnet-daemon-perl libplrpc-perl mariadb-client-5.5 mariadb-client-core-5.5 mariadb-common mariadb-server-5.5 mariadb-server-core-5.5 mysql-common Suggested packages: libipc-sharedcache-perl tinyca mailx mariadb-test The following NEW packages will be installed libaio1 libdbd-mysql-perl libdbi-perl libhtml-template-perl libmariadbclient18 libmysqlclient18 libnet-daemon-perl libplrpc-perl mariadb-client-5.5 mariadb-client-core-5.5 mariadb-common mariadb-server mariadb-server-5.5 mariadb-server-core-5.5 mysql-common 0 upgraded, 15 newly installed, 0 to remove and 66 not upgraded. Need to get 32.7 MB of archives. After this operation, 114 MB of additional disk space will be used. ... Setting up mariadb-server-core-5.5 (5.5.34+maria-1~precise) ... Setting up mariadb-server-5.5 (5.5.34+maria-1~precise) ... * Stopping MariaDB database server mysqld [ OK ] 131224 20:00:56 [Note] Plugin 'InnoDB' is disabled. 131224 20:00:56 [Note] Plugin 'FEEDBACK' is disabled. * Starting MariaDB database server mysqld [ OK ] * Checking for corrupt, not cleanly closed and upgrade needing tables. Setting up mariadb-server (5.5.34+maria-1~precise) ... Processing triggers for libc-bin ... ldconfig deferred processing now taking place
See Installing MariaDB .deb Files for more information.
You can also create a custom MariaDB sources.list file. To do so, copy and paste the following into a file under /etc/apt/sources.list.d/(we suggest naming the file MariaDB.list or something similar), or add it to the bottom of your /etc/apt/sources.list file.
# MariaDB 5.5 repository list - created 2013-12-24 18:52 UTC # http://mariadb.org/mariadb/repositories/ deb http://ftp.igh.cnrs.fr/pub/mariadb/repo/5.5/ubuntu precise main deb-src http://ftp.igh.cnrs.fr/pub/mariadb/repo/5.5/ubuntu precise main
References:
https://downloads.mariadb.org/mariadb/repositories/#mirror=cnrs&distro=Ubuntu&distro_release=precise&version=5.5
Create database and user for analysis
Using a root account on the applications as user is risky. It is more secure to create a normal user for the applications.
janague@metricsGrimoireHost:~$ mysql -u root -p Enter password: Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 30 Server version: 5.5.34-MariaDB-1~precise-log mariadb.org binary distribution Copyright (c) 2000, 2013, Oracle, Monty Program Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> create user 'janague'@'localhost' identified by 'janague123'; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | +--------------------+ 3 rows in set (0.00 sec) MariaDB [(none)]> create database CVSAnalytDB; Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | CVSAnalytDB | | mysql | | performance_schema | +--------------------+ 4 rows in set (0.00 sec) MariaDB [(none)]> grant all privileges on CVSAnalytDB.* to 'janague'@'localhost' identified by 'janague123'; Query OK, 0 rows affected (0.00 sec)
Install CVSAnalyt
Create a directory for the repositories with user different of root.
janague@metricsGrimoireHost:~$ mkdir repositories janague@metricsGrimoireHost:~$ cd repositories/
Clone CVSAnalyt repository
janague@metricsGrimoireHost:~/repositories$ git clone git://github.com/MetricsGrimoire/CVSAnalY.git Cloning into 'CVSAnalY'... remote: Counting objects: 3029, done. remote: Compressing objects: 100% (1225/1225), done. remote: Total 3029 (delta 1828), reused 2964 (delta 1776) Receiving objects: 100% (3029/3029), 1.95 MiB | 704 KiB/s, done. Resolving deltas: 100% (1828/1828), done.
You can install cvsanaly2 just by running setup.py script in CVSAnalY directory
janague@metricsGrimoireHost:~/repositories/CVSAnalY$ sudo python setup.py install Traceback (most recent call last): File "setup.py", line 35, in <module> from setuptools import setup ImportError: No module named setuptools
In this case, we have this error "No module named setuptools". To resolve the problem we need to install python-pip and setuptools
janague@metricsGrimoireHost:~/repositories/CVSAnalY$ sudo apt-get install python-pip Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: python-setuptools The following NEW packages will be installed python-pip python-setuptools 0 upgraded, 2 newly installed, 0 to remove and 66 not upgraded. Need to get 536 kB of archives. After this operation, 1,467 kB of additional disk space will be used. Do you want to continue [Y/n]? Y Get:1 http://es.archive.ubuntu.com/ubuntu/ precise/main python-setuptools all 0.6.24-1ubuntu1 [441 kB] Get:2 http://es.archive.ubuntu.com/ubuntu/ precise/universe python-pip all 1.0-1build1 [95.1 kB] Fetched 536 kB in 0s (619 kB/s) Selecting previously unselected package python-setuptools. (Reading database ... 52983 files and directories currently installed.) Unpacking python-setuptools (from .../python-setuptools_0.6.24-1ubuntu1_all.deb) ... Selecting previously unselected package python-pip. Unpacking python-pip (from .../python-pip_1.0-1build1_all.deb) ... ... janague@metricsGrimoireHost:~/repositories/CVSAnalY$ sudo pip install --upgrade setuptools Downloading/unpacking distribute Downloading distribute-0.7.3.zip (145Kb): 145Kb downloaded Running setup.py egg_info for package distribute ... Installing easy_install script to /usr/local/bin Installing easy_install-2.7 script to /usr/local/bin Successfully installed distribute setuptools Cleaning up...
Try again to install CVSAnalyt
janague@metricsGrimoireHost:~/repositories/CVSAnalY$ sudo python setup.py install running install running bdist_egg running egg_info creating cvsanaly2.egg-info writing requirements to cvsanaly2.egg-info/requires.txt writing cvsanaly2.egg-info/PKG-INFO ... Installed /usr/local/lib/python2.7/dist-packages/cvsanaly2-2.1.0-py2.7.egg Processing dependencies for cvsanaly2==2.1.0 Searching for RepositoryHandler==0.5.1 Best match: RepositoryHandler 0.5.1 Adding RepositoryHandler 0.5.1 to easy-install.pth file Using /usr/local/lib/python2.7/dist-packages Finished processing dependencies for cvsanaly2==2.1.0
Install dependence to RepositoryHandler
janague@metricsGrimoireHost:~/repositories$ git clone git://github.com/MetricsGrimoire/RepositoryHandler.git Cloning into 'RepositoryHandler'... remote: Counting objects: 649, done. remote: Compressing objects: 100% (223/223), done. remote: Total 649 (delta 423), reused 635 (delta 411) Receiving objects: 100% (649/649), 173.71 KiB | 230 KiB/s, done. Resolving deltas: 100% (423/423), done. Install RepositoryHandler janague@metricsGrimoireHost:~/repositories/RepositoryHandler$ sudo python setup.py install [sudo] password for janague: running install running build running build_py creating build creating build/lib.linux-x86_64-2.7 creating build/lib.linux-x86_64-2.7/repositoryhandler ... byte-compiling /usr/local/lib/python2.7/dist-packages/repositoryhandler/Downloader.py to Downloader.pyc running install_egg_info Removing /usr/local/lib/python2.7/dist-packages/RepositoryHandler-0.5.1.egg-info Writing /usr/local/lib/python2.7/dist-packages/RepositoryHandler-0.5.1.egg-info
To test CVSAnalyt installation, let's go to analyse CVSAnalyt repository. We are going to use de database CVSAnalytDB.
Before executing CVSAnalyt
janague@metricsGrimoireHost:~$ mysql -u janague -p -h localhost CVSAnalytDB Enter password: Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 32 Server version: 5.5.34-MariaDB-1~precise-log mariadb.org binary distribution Copyright (c) 2000, 2013, Oracle, Monty Program Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [CVSAnalytDB]> show tables; Empty set (0.00 sec)
Execute cvsanaly2 in CVSAnalyt repository
janague@metricsGrimoireHost:~/repositories/CVSAnalY$ cvsanaly2 -u janague -d CVSAnalytDB Traceback (most recent call last): File "/usr/local/bin/cvsanaly2", line 5, in <module> pkg_resources.run_script('cvsanaly2==2.1.0', 'cvsanaly2') File "/usr/local/lib/python2.7/dist-packages/pkg_resources.py", line 487, in run_script self.require(requires)[0].run_script(script_name, ns) File "/usr/local/lib/python2.7/dist-packages/pkg_resources.py", line 1337, in run_script execfile(script_filename, namespace, namespace) File "/usr/local/lib/python2.7/dist-packages/cvsanaly2-2.1.0-py2.7.egg/EGG-INFO/scripts/cvsanaly2", line 37, in <module> retval = pycvsanaly2.main.main (sys.argv[1:]) File "/usr/local/lib/python2.7/dist-packages/cvsanaly2-2.1.0-py2.7.egg/pycvsanaly2/main.py", line 275, in main config.db_hostname) File "/usr/local/lib/python2.7/dist-packages/cvsanaly2-2.1.0-py2.7.egg/pycvsanaly2/Database.py", line 609, in create_database db.connect ().close () File "/usr/local/lib/python2.7/dist-packages/cvsanaly2-2.1.0-py2.7.egg/pycvsanaly2/Database.py", line 462, in connect import MySQLdb ImportError: No module named MySQLdb
cvsanaly2 shows an error, because we have a dependence of MySQLdb module.
janague@metricsGrimoireHost:~/repositories/CVSAnalY$ sudo apt-get install python-mysqldb Reading package lists... Done Building dependency tree Reading state information... Done Suggested packages: python-egenix-mxdatetime python-mysqldb-dbg The following NEW packages will be installed python-mysqldb 0 upgraded, 1 newly installed, 0 to remove and 66 not upgraded. Need to get 64.0 kB of archives. After this operation, 221 kB of additional disk space will be used. Get:1 http://es.archive.ubuntu.com/ubuntu/ precise-updates/main python-mysqldb amd64 1.2.3-1ubuntu0.1 [64.0 kB] Fetched 64.0 kB in 0s (137 kB/s) Selecting previously unselected package python-mysqldb. (Reading database ... 53198 files and directories currently installed.) Unpacking python-mysqldb (from .../python-mysqldb_1.2.3-1ubuntu0.1_amd64.deb) ... Setting up python-mysqldb (1.2.3-1ubuntu0.1) ...
Execute again cvsanaly2 in CVSAnalyt repository
janague@metricsGrimoireHost:~/repositories/CVSAnalY$ cvsanaly2 -u janague -d CVSAnalytDB Password: Parsing log for /home/janague/repositories/CVSAnalY (git) Warning: Detected empty branch 'libresoft-utils', it'll be ignored Executing extensions
And now in the database it shows the following tables
MariaDB [CVSAnalytDB]> show tables; +-----------------------+ | Tables_in_CVSAnalytDB | +-----------------------+ | action_files | | actions | | actions_file_names | | branches | | file_copies | | file_links | | files | | people | | repositories | | scmlog | | tag_revisions | | tags | +-----------------------+ 12 rows in set (0.00 sec)
Some example queries
1.- Top of committers
MariaDB [CVSAnalytDB]> select p.name, count(distinct(s.id)) num_commits from people p, scmlog s where p.id=s.author_id group by p.name order by count(distinct(s.id)) desc; +--------------------------------+-------------+ | name | num_commits | +--------------------------------+-------------+ | Carlos Garcia Campos | 277 | | Jesus M. Gonzalez-Barahona | 60 | | Israel Herraiz | 39 | | Andy Grunwald | 31 | | Santiago Dueñas | 23 | | Alvaro Navarro | 17 | | Germán Póo-Caamaño | 17 | | Santiago Dueñas Domínguez | 17 | | Luis Cañas Díaz | 10 | | Daniel Izquierdo Cortazar | 9 | | Alvaro del Castillo | 7 | | Gregorio Robles | 6 | | Daniel Izquierdo | 4 | | Luis Cañas | 3 | | Luis Cañas-Díaz | 3 | | Felipe Ortega | 3 | | companheiro.vermelho@gmail.com | 2 | | Ilya Shakhat | 1 | | Juan Francisco Gato Luis | 1 | | Carlos Gonzalez | 1 | | Liliana Tovar | 1 | | Maëlick Claes | 1 | +--------------------------------+-------------+ 22 rows in set (0.00 sec)
2.- Sort list of developers with the number of touched files.
MariaDB [CVSAnalytDB]> select p.name, count(distinct(a.file_id)) num_files from people p, actions a, scmlog s where s.id = a.commit_id and s.author_id = p.id group by p.name order by count(distinct(a.id)) desc; +--------------------------------+-----------+ | name | num_files | +--------------------------------+-----------+ | Carlos Garcia Campos | 302 | | Alvaro Navarro | 138 | | Jesus M. Gonzalez-Barahona | 56 | | Israel Herraiz | 43 | | Andy Grunwald | 27 | | Luis Cañas Díaz | 43 | | Santiago Dueñas Domínguez | 5 | | Luis Cañas-Díaz | 32 | | Santiago Dueñas | 19 | | Germán Póo-Caamaño | 5 | | Liliana Tovar | 12 | | companheiro.vermelho@gmail.com | 12 | | Alvaro del Castillo | 8 | | Carlos Gonzalez | 9 | | Daniel Izquierdo Cortazar | 7 | | Gregorio Robles | 6 | | Juan Francisco Gato Luis | 7 | | Daniel Izquierdo | 4 | | Luis Cañas | 3 | | Felipe Ortega | 2 | | Ilya Shakhat | 1 | | Maëlick Claes | 1 | +--------------------------------+-----------+ 22 rows in set (0.01 sec)
3.- Sort list of contributors in 2013.
MariaDB [CVSAnalytDB]> select p.name, count(distinct(a.file_id)) num_files from people p, actions a, scmlog s where s.id = a.commit_id and s.author_id = p.id and year(s.date) = 2013 group by p.name order by count(distinct(a.id)) desc; +---------------------------+-----------+ | name | num_files | +---------------------------+-----------+ | Andy Grunwald | 27 | | Carlos Gonzalez | 9 | | Santiago Dueñas | 5 | | Alvaro del Castillo | 2 | | Daniel Izquierdo Cortazar | 2 | | Daniel Izquierdo | 1 | | Ilya Shakhat | 1 | | Maëlick Claes | 1 | +---------------------------+-----------+ 8 rows in set (0.00 sec)
References
http://metricsgrimoire.github.io/
https://github.com/MetricsGrimoire
https://github.com/MetricsGrimoire/CVSAnalY
http://www.linux-kvm.org
https://mariadb.org/
https://help.ubuntu.com/12.04/serverguide/serverguide.pdf
https://help.ubuntu.com/community/SSH/OpenSSH/Keys
No hay comentarios:
Publicar un comentario