Raed's Journey Blog

In a lovely day, I woke up and went to work feeling nothing but happiness. And as I usually start my days, I opened the office door, sat down on my uncomfortable chair, opened putty session to my outdated RHEL 4.7 machine and my browser to our Red Hat Network Satellite server web page.

"My machine has lots and lots of messing security patches. I think I'll update every thing.", I said.
next thing, my fingers were typing "up2date". Once Enter was pressed, a huge smile acquired my face.
"I will tell them I am updating my machine and can't work for 20-30 minutes", I thought.

seeing the command trying to fetch packages. fetching, fetching, fetching, .... "Segmentation Fault". WTH .... I did not believe this thing. I tried searching and searching around and around. I found a couple of nice posts (can't remember them, I closed them once I solved the problem :P) and tried what they suggest. One of them used strace to trace the root of the problem. Guess what big guy, i DON'T KNOW A THING ABOUT STRACE. I am not a quitter tho. So, I ran "strace -f -o /tmp/strace up2date thunderbird". in less than a second, the command line appeared asking me for another command. I said "What??? my machine is da*# fast.". 

I opened /tmp/strace, and guess what. It was empty. I removed the "-o /tmp/strace" and tried again. An output there was. weird huh??

I checked the output and there was a weird path, "/var/spool/up2date". "lemme check it", I said.

Once I did, i found out that there were old headers in there. So, as a noobie usually does, I deleted them. I ran "up2date" again and this time it came with another error message. something related to not having enough space. 

I found later that /tmp was full. Therefore, trynig to write to /tmp did not work (no warning message though). I had to delete everything there and ,,, that solved the problem.

just to sum up:

• /tmp was full. cleaned it. (you might need to use "lsof /tmp to see /tmp (deleted??) opened files and kill the PIDs)
• clean /var/spool/up2date/

sorry, had to write this post really fast. I need to do some remote installation :).

Three days ago, I faced a tomcat problem with Red Hat Network Satellite Server. The problem was that whenever I tried to start the Red Hat Satellite Server service it said

[root@RHS run]# service rhn-satellite start
Starting rhn-satellite...
Starting Jabber services                                   [  OK  ]
Starting osa-dispatcher:                                   [  OK  ]
Starting RHN Taskomatic...
Starting tomcat5:                                          [  OK  ]
Starting satellite-httpd:                                  [  OK  ]
Starting rhn-search...
Done.

So, everything should work perfectly. Not really.

[root@RHS run]# service rhn-satellite status
jabberd router (pid 3002) is running...
osa-dispatcher (pid  3074) is running...
RHN Taskomatic is running (3433).
lock file found but no process running for pid 3888
httpd (pid 3956 3955 3954 3953 3952 3951 3950 3949 3922) is running...
rhn-search is running (3947).

I tried several solutions in the Internet just to solve the problem but I could not find a solution that really solved it. I had to compare our two Red Hat Network Satellite to "detect" the main cause.

First, I needed to get my tomcat packages. I ran

rpm -qa | grep tomcat | grep -v rhn-oracle-jdbc-tomcat5-1.0-13.el5 | xargs rpm -ql > /tmp/tomcatfiles

which gave me all tomcat files I have. Then I executed

for file in `cat /tmp/tomcatfiles`; do ll $file; done > /tmp/tomcatFilesPermission

 

to see all my files with their permissions (I did not want to miss anything :)). When I compared the results from my two Red Hat Network Satellite, the one with "lock file" was missing some jar files under /var/lib/tomcat5/common/lib. When I copied them

scp -r /var/lib/tomcat5/common/lib NewRedHatSatelliteServer:/var/lib/tomcat5/common/lib

I went to the Satellite Server with the problem and started the service

[root@RHS common]# service rhn-satellite start
Starting rhn-satellite...
Starting Jabber services                                   [  OK  ]
Starting osa-dispatcher:                                   [  OK  ]
Starting RHN Taskomatic...
Starting tomcat5:                                          [  OK  ]
Starting satellite-httpd:                                  [  OK  ]
Starting rhn-search...
Done.
[root@RHS common]# service rhn-satellite status
jabberd router (pid 11539) is running...
osa-dispatcher (pid  11563) is running...
RHN Taskomatic is running (11588).
/etc/init.d/tomcat5 is already running (12141)
httpd (pid 12200 12199 12198 12197 12196 12195 12194 12193 12166) is running...
rhn-search is running (12192).

everything worked just fine :)

CiscoPress tweeted that "CCIE Routing and Switching Exam Certification Guide, 4th Edition" is going to be available on December 3rd, 2009. CCIEs and CCIE seekers, hurry up an pre-order the guide.