err: Could not retrieve catalog from remote server: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed

So I am seeing below captioned error when I am trying to connect to a puppet master.


err: Could not retrieve catalog from remote server: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed

The reason for above error is because the agent node is trying to connect to a different master and then it failed to validate the certificate. To solve the problem, we have to execute below command and retry.

find /var/lib/puppet -type f | xargs rm -rf

Generate puppet server certificate

So I am getting error "err: Could not call sign: Could not find certificate request for puppetmaster" when I try to startup puppet server. I have to generate a SSL cert for the puppet server before going on.

root@puppetmaster:/etc/puppet# puppet cert generate puppetmaster
notice: puppetmaster has a waiting certificate request
notice: Signed certificate request for puppetmaster
notice: Removing file Puppet::SSL::CertificateRequest puppetmaster at '/var/lib/puppet/ssl/ca/requests/puppetmaster.pem'
notice: Removing file Puppet::SSL::CertificateRequest puppetmaster at '/var/lib/puppet/ssl/certificate_requests/puppetmaster.pem'

Ubuntu, E: Unable to locate package

So I am trying to install packages on a newly installed Ubuntu box from aptitude but somehow it failed to locate the package.

# apt-get install gcc
Reading package lists... Done
Building dependency tree      
Reading state information... Done
E: Unable to locate package gcc
# aptitude search gcc
#

I am pretty sure the box could connect to the internet so it is quite weird it failed to locate the package.

In fact the issue is pretty straight forward, the local aptitude database didn't contain the software entries and this require an update of the database.

# apt-get update
Ign http://us.archive.ubuntu.com precise InRelease                                                        
Ign http://us.archive.ubuntu.com precise-updates InRelease                                                
Ign http://us.archive.ubuntu.com precise-backports InRelease
Ign http://security.ubuntu.com precise-security InRelease
Get:1 http://us.archive.ubuntu.com precise Release.gpg [198 B]
Get:2 http://us.archive.ubuntu.com precise-updates Release.gpg [198 B]
Get:3 http://security.ubuntu.com precise-security Release.gpg [198 B]
...
...
Get:87 http://us.archive.ubuntu.com precise-backports/restricted Translation-en [14 B]                                                        
Get:88 http://us.archive.ubuntu.com precise-backports/universe Translation-en [8,555 B]                                                       
Fetched 24.8 MB in 32s (769 kB/s)                                                                                                             
Reading package lists... Done

Now the issue is resolved. :-)

# apt-get install gcc
Reading package lists... Done
Building dependency tree      
Reading state information... Done
The following extra packages will be installed:
  binutils cpp cpp-4.6 gcc-4.6 libc-dev-bin libc6-dev libgomp1 libmpc2 libmpfr4 libquadmath0 linux-libc-dev manpages-dev
Suggested packages:
... 
...

Openstack Keystone (diablo): Got: ImportError('No module named MySQLdb',)

If one is seeing this on a Ubuntu / Debian box during start of keystone after migrating DB from sqlite to MySQL, simply installing the associated python libraries would fix the issue.

root@keystone:~/openstack-keystone-79a9fde# ERROR: Unable to load keystone-legacy-auth from configuration file /etc/keystone/keystone.conf.
Got: ImportError('No module named MySQLdb',)


root@keystone:~/openstack-keystone-79a9fde# apt-get install python-mysqldb
Reading package lists... Done
Building dependency tree      
Reading state information... Done
The following extra packages will be installed:
  python-support
Suggested packages:
  python-egenix-mxdatetime python-mysqldb-dbg
The following NEW packages will be installed:
  python-mysqldb python-support
0 upgraded, 2 newly installed, 0 to remove and 58 not upgraded.
Need to get 109 kB of archives.
After this operation, 578 kB of additional disk space will be used.
Do you want to continue [Y/n]? y
Get:1 http://us.archive.ubuntu.com/ubuntu/ oneiric/main python-support all 1.0.13ubuntu1 [26.6 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu/ oneiric/main python-mysqldb amd64 1.2.3-0ubuntu1 [82.5 kB]
Fetched 109 kB in 0s (184 kB/s)     
Selecting previously deselected package python-support.
(Reading database ... 55750 files and directories currently installed.)
Unpacking python-support (from .../python-support_1.0.13ubuntu1_all.deb) ...
Selecting previously deselected package python-mysqldb.
Unpacking python-mysqldb (from .../python-mysqldb_1.2.3-0ubuntu1_amd64.deb) ...
Processing triggers for man-db ...
Setting up python-support (1.0.13ubuntu1) ...
Setting up python-mysqldb (1.2.3-0ubuntu1) ...
Processing triggers for python-support ...

root@keystone:~/openstack-keystone-79a9fde# keystone &
[1] 13882
root@keystone:~/openstack-keystone-79a9fde# Starting the RAX-KEY extension
Starting the Legacy Authentication component
Service API listening on 0.0.0.0:5000
Admin API listening on 0.0.0.0:35357

Disk benchmarking by bonnie++ in Linux

Bonnie++ would possibly be come with your install, you could either get it by "yum install bonnie++" (Fedora/CentOS) or "apt-get install bonnie++" (Ubuntu/Debian) or install it via source (link here)

So once bonnie++ is installed you could start the test by doing.

root@host:/tmp# bonnie++ -m test-box -u root -x 3 -d /tmp/ -s 1024 -r 512 | bon_csv2html > result.html
Using uid:0, gid:0.
Writing a byte at a time...Can't process: format_version,bonnie_version,name,file_size,io_chunk_size,putc,putc_cpu,put_block,put_block_cpu,rewrite,rewrite_cpu,getc,getc_cpu,get_block,get_block_cpu,seeks,seeks_cpu,num_files,max_size,min_size,num_dirs,file_chunk_size,seq_create,seq_create_cpu,seq_stat,seq_stat_cpu,seq_del,seq_del_cpu,ran_create,ran_create_cpu,ran_stat,ran_stat_cpu,ran_del,ran_del_cpu,putc_latency,put_block_latency,rewrite_latency,getc_latency,get_block_latency,seeks_latency,seq_create_latency,seq_stat_latency,seq_del_latency,ran_create_latency,ran_stat_latency,ran_del_latency
done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.


-m test-box : -m is the option to control name presented in report.



-u : root means you want it be executed with root privilege. If you want to use something else, just make sure that particular user have the write privileges to the directory specified in option -d.

-d : means the directory you want to be running on, which means the file system have to be mounted prior to the run.

-x 3 : means it will be executed 3 times so that we can pick a fair value.

bon_csv2html > result.html : Bonnie++ would only generate result in CSV format which is no good for presentation purpose. bon_csv2html do the dirty job for you to convert CSV to HTML. We add the redirection here to save the output to a static file for future retrieval. If you prefer TXT report instead of HTML report, you would want to go with bon_csv2txt.

Cyberduck authentication failure against Openstack Swift with swauth

During testing of Openstack Swift, I found that there isn't much GUI clients out there support Openstack Swift and the best option I could go with is Cyberduck.

Downloading it and then follow by installation on my windows test VM, everything appears to be good and smooth, except it keep saying "Login failed."

So I am pretty sure my username (account:username) and API Key (basically the password of the user) is correct however I am still not able to get in. I did some test with s3curl.pl and it is authenticating successfully without any issue.

Tried googling but there isnt article explaining this but somehow I found this article from cyberduck trac page.  It looks like I have to modify some parameter on Cyberduck to allow it to work with Swift with swauth authentication module.

So as per the article suggested, I added below line to user.config and restarted Cyberduck. Now authentication seems to be working.

<setting name="cf.authentication.context" value="/auth/v1.0" />

AWS storage gateway: WORKING STORAGE NOT CONFIGURED

As continuing the test on AWS Storage gateway, I found that there is an implicit requirement of the AWS storage VM, i.e. the VM have to be assigned with a publicly accessible IP address, or at least the IP address could be reached by AWS network.

The logic behind is that when someone trying to manage the AWS storage VM via AWS web console, the instruction will have to be passed over to the VM (possibly via port 80 of the AWS VM, but I didnt confirm it yet) via public network. In any case AWS failed to reach the VM, it will not able to proceed with the instruction.

The above idea was tested against an internal VM I was playing with yesterday. The VM is sit on private network (e..g 192.168.x.x) with outgoing NAT enable but not incoming NAT enable. I could successfully proceed with the VM activation but no volumes could be added from AWS console. The newly added volumes keep showing "WORKING STORAGE NOT CONFIGURED" on AWS console which basically means that it is not creating at all. Usually, creating a new volume should not take too long at all.

Here is the screenshot though,

Apart from volumes creation failure, I also tried adding new virtual disk to the storage VM and see if AWS could see the new virtual disk. However, the answer is no. So what I could pretty sure here is that AWS will have to talk to VM and it just wont be able to put the Storage VM on an internal network segment which is not accessible from public.