events/0 events/1 high load

Okay so what happens is i got a quad core xeon server with 16 GB Ram, nothing special, running on 100 mbit uplink.

I had to upload lots of offsite data to this server, and i first noticed i could not go past 3MB/s on speeds. up/down total. I made a mess at the datacenter, because i thought i was capped. The server also became almost unresponsive. The overall connectivity seemed really saturated or capped, until i tried unpacking some archives on the drive, and it took me like … 1 hour for a 5 gb archive, the idea punched me in the face. It could be a disk performance issue.

I had to start debugging, cuz i did not want to look like a moron asking for the moon in the sky so …

Debugging:

– i noticed the events/0 events/1 process was sky high load 90 almost all the time.

– suddenly i came with the idea to do a hdd test:

[[email protected] d1]# hdparm -cuda /dev/hdc

/dev/hdc:
IO_support = 0 (default 16-bit)
unmaskirq = 0 (off)
using_dma = 0 (off)
readahead = 256 (on)

the performance therefore goes:

[[email protected] d1]# hdparm -tT /dev/hdc

/dev/hdc:
Timing cached reads: 10444 MB in 2.00 seconds = 5225.62 MB/sec
Timing buffered disk reads: 2 MB in 4.15 seconds = 493.55 kB/sec

dma is not enabled on the hard drive, and i tried to enable from the system,
but it would not work

[[email protected] d1]# hdparm -d1 /dev/hdc

/dev/hdc:
setting using_dma to 1 (on)
HDIO_SET_DMA failed: Operation not permitted
using_dma = 0 (off)

Things to consider: DMA not enabled, all i/o requests go through the kernel, hence the crappy read/write speeds.

What to do?

Check if DMA is enabled, if not it’s a network issue. If DMA is not enabled, the drive will simply act like a saturated internet connection. How to fix the permission denied DMA issue – ask the dc techs to check your bios. It’s most likely bios if you cannot enable it from the os.

[[email protected] ~]# hdparm -tT /dev/sda

/dev/sda:
Timing cached reads:   10020 MB in  2.00 seconds = 5014.39 MB/sec
Timing buffered disk reads:  158 MB in  3.01 seconds =  52.52 MB/sec

Enjoy

x264 compile – isomedia.h not found

Right, so after installing gpac, then you try installing x264. Figure it out that you need to install the gpac libs before compiling x264:

gcc -Wall -I. -O4 -ffast-math -D__X264__ -DHAVE_MALLOC_H -DHAVE_MMXEXT -DHAVE_SSE2 -DARCH_X86 -DSYS_LINUX -DHAVE_PTHREAD -s -fomit-frame-pointer -c -o x264.o x264.c gcc -Wall -I. -O4 -ffast-math -D__X264__ -DHAVE_MALLOC_H -DHAVE_MMXEXT -DHAVE_SSE2 -DARCH_X86 -DSYS_LINUX -DHAVE_PTHREAD -s -fomit-frame-pointer -c -o muxers.o muxers.c

muxers.c:43:27: error: gpac/isomedia.h: No such file or directory
muxers.c:550: error: syntax error before âGF_ISOFileâ
muxers.c:550: warning: no semicolon at end of struct or union
muxers.c:551: warning: type defaults to âintâ in declaration of âp_configâ
muxers.c:551: warning: data definition has no type or storage class
muxers.c:552: error: syntax error before â*â token
muxers.c:552: warning: type defaults to âintâ in declaration of âp_sampleâ
muxers.c:552: warning: data definition has no type or storage class
muxers.c:561: error: syntax error before â}â token
muxers.c:561: warning: type defaults to âintâ in declaration of âmp4_tâ
muxers.c:561: warning: data definition has no type or storage class
muxers.c:564: error: syntax error before â*â token
muxers.c: In function ârecompute_bitrate_mp4â:
muxers.c:566: error: âu32â undeclared (first use in this function)
muxers.c:566: error: (Each undeclared identifier is reported only once
muxers.c:566: error: for each function it appears in.)
muxers.c:566: error: syntax error before âiâ
muxers.c:567: error: âu64â undeclared (first use in this function)
muxers.c:568: error: âDoubleâ undeclared (first use in this function)
muxers.c:569: error: âGF_ESDâ undeclared (first use in this function)
muxers.c:569: error: âesdâ undeclared (first use in this function)
muxers.c:571: warning: implicit declaration of function âgf_isom_get_esdâ
muxers.c:571: error: âp_fileâ undeclared (first use in this function)
muxers.c:576: error: ârateâ undeclared (first use in this function)
muxers.c:576: error: âtime_wndâ undeclared (first use in this function)
muxers.c:578: error: âtimescaleâ undeclared (first use in this function)

muxers.c:578: warning: implicit declaration of function âgf_isom_get_media_timescaleâ

muxers.c:579: error: âcountâ undeclared (first use in this function)

muxers.c:579: warning: implicit declaration of function âgf_isom_get_sample_countâ

muxers.c:580: error: âiâ undeclared (first use in this function)
muxers.c:581: error: âGF_ISOSampleâ undeclared (first use in this function)
muxers.c:581: error: âsampâ undeclared (first use in this function)

muxers.c:581: warning: implicit declaration of function âgf_isom_get_sample_infoâ

muxers.c:581: error: âdiâ undeclared (first use in this function)
muxers.c:581: error: âoffsetâ undeclared (first use in this function)
muxers.c:594: warning: implicit declaration of function âgf_isom_sample_delâ
muxers.c:597: error: âbrâ undeclared (first use in this function)
muxers.c:597: error: âs64â undeclared (first use in this function)
muxers.c:597: error: syntax error before âgf_isom_get_media_durationâ

muxers.c:604: warning: implicit declaration of function âgf_isom_change_mpeg4_descriptionâ

muxers.c:605: warning: implicit declaration of function âgf_odf_desc_delâ
muxers.c:605: error: âGF_Descriptorâ undeclared (first use in this function)
muxers.c:605: error: syntax error before â)â token
muxers.c: In function âclose_file_mp4â:
muxers.c:611: error: âp_mp4â undeclared (first use in this function)
muxers.c:611: error: syntax error before â)â token
muxers.c:617: warning: implicit declaration of function âgf_odf_avc_cfg_delâ

muxers.c:630: warning: implicit declaration of function âgf_isom_set_pl_indicationâ muxers.c:630: error: âGF_ISOM_PL_VISUALâ undeclared (first use in this function) muxers.c:631: warning: implicit declaration of function âgf_isom_set_storage_modeâ muxers.c:631: error: âGF_ISOM_STORE_FLATâ undeclared (first use in this function)

muxers.c:632: warning: implicit declaration of function âgf_isom_closeâ
muxers.c: In function âopen_file_mp4â:
muxers.c:642: error: âp_mp4â undeclared (first use in this function)
muxers.c:646: error: syntax error before â)â token
muxers.c:650: warning: implicit declaration of function âgf_isom_openâ

muxers.c:650: error: âGF_ISOM_OPEN_WRITEâ undeclared (first use in this function)

muxers.c:652: warning: implicit declaration of function âgf_isom_sample_newâ

muxers.c:658: warning: implicit declaration of function âgf_isom_set_brand_infoâ muxers.c:658: error: âGF_ISOM_BRAND_AVC1â undeclared (first use in this function)

muxers.c: In function âset_param_mp4â:
muxers.c:668: error: âp_mp4â undeclared (first use in this function)
muxers.c:668: error: syntax error before â)â token
muxers.c:670: warning: implicit declaration of function âgf_isom_new_trackâ

muxers.c:670: error: âGF_ISOM_MEDIA_VISUALâ undeclared (first use in this function)

muxers.c:673: warning: implicit declaration of function âgf_odf_avc_cfg_newâ

muxers.c:674: warning: implicit declaration of function âgf_isom_avc_config_newâ muxers.c:677: warning: implicit declaration of function âgf_isom_set_track_enabledâ muxers.c:679: warning: implicit declaration of function âgf_isom_set_visual_infoâ

muxers.c: In function âwrite_nalu_mp4â:
muxers.c:699: error: âp_mp4â undeclared (first use in this function)
muxers.c:699: error: syntax error before â)â token

muxers.c:700: error: âGF_AVCConfigSlotâ undeclared (first use in this function)

muxers.c:700: error: âp_slotâ undeclared (first use in this function)
muxers.c:714: error: syntax error before â)â token
muxers.c:718: warning: implicit declaration of function âgf_list_addâ
muxers.c:728: error: syntax error before â)â token

muxers.c:736: warning: implicit declaration of function âgf_isom_avc_config_updateâ

muxers.c: In function âset_eop_mp4â:
muxers.c:759: error: âp_mp4â undeclared (first use in this function)
muxers.c:759: error: syntax error before â)â token
muxers.c:767: warning: implicit declaration of function âgf_isom_add_sampleâ
make: *** [muxers.o] Error 1

So the thing is :
cd /path/to/gpac
make install-lib
make install
cd /path/to/x264
start recompiling x264 and it works.

gpac osmozilla – compile error

While compiling gpac, on several systems, you may get the following error:

osmozilla.cpp:793: error: ânsrefcntâ does not name a type
osmozilla.cpp:799: error: ânsrefcntâ does not name a type
osmozilla.cpp:811: error: ânsresultâ does not name a type
make[2]: *** [osmozilla.o] Error 1
make[2]: Leaving directory `/root/video-stuff/gpac/applications/osmozilla’
make[1]: *** [apps] Error 2
make[1]: Leaving directory `/root/video-stuff/gpac/applications’
make: *** [apps] Error 2

Fix is simple:

edit applications/Makefile and comment out the following lines:

#INSTDIRS+=osmozilla
#APPDIRS+=osmozilla

Should get you running

pkg_delete: package doesn’t have a prefix

—> Deinstalling ‘curl-7.16.3’
pkg_delete: package ‘curl-7.16.3’ doesn’t have a prefix
** Listing the failed packages (*:skipped / !:failed)
! curl-7.16.3 (pkg_delete failed)
—> Packages processed: 0 done, 0 ignored, 0 skipped and 1 failed

======================

ok i came across this error, when i was installing php5-extensions, and suddenly the box froze. curl was installed half only and i could not deinstall the package simply by pkg_deinstall or pkg_delete.

after a few minutes, i came across to a simple solution:

pkg_deinstall -pf curl-7.16.3

this will make the pain go away.

it’s obvious that we’re talking about a freebsd server right?

SSH keys how to

Been having this issue with creating daily backups, on a remote server, now i got the chance to work it out pretty easy via rsync/ssh keys.

Easy way to do it:

What must be done, then , is to generate a public/private key pair, and copy the public part into the appropiate place on the server side.
For doing this, on the user’s home directory, on the client machine, type

local> ssh-keygen -t dsa -f .ssh/id_dsa

-t tells the type of encription
-f tells where to store the public/private key pairs. In this case, the .ssh directory on home is being used

A password will be asked; leave this part blank, just pressing <enter>
Now, go the .ssh directory, and you will find two new files: id_dsa and id_dsa.pub. The last one is the public part. Now, copy the public key to the server machine

local> cd .ssh
local> scp id_dsa.pub [email protected]:~/.ssh/id_dsa.pub

Of course, this time you will need to enter the password.
Now, login into the server machine and go to the .ssh directory on the server side

local> ssh [email protected]
remote> cd .ssh

Now, add the client’s public key to the know public keys on the server

remote> cat id_dsa.pub >> authorized_keys2
remote> chmod 640 authorized_keys2
remote> rm id_dsa.pub

remote> exit

and that’s all.
Next time you log into the remote server, no password will be asked.
Note that this sytem will work while none of the machines change its IP address and for the specific user, so it is still safe.

Compiling PHP 5.2.6 / cannot find -lltdl

When compiling php from source, some CentOS 64 bit users have reported getting an error like this:

Code:
/usr/bin/ld: cannot find -lltdl
collect2: ld returned 1 exit status

Now, this error can be triggered in a couple different places and if you do a general Google search you’ll get all kinds of advice about installing this or that module. All great advice, but not your solution. Here’s what you need to do:

  1. Verify that the libtool and libtool-ltdl packages are installed.
  2. Symlink libltdl.so to libltdl.so.x.x.x

Step one is easy.

Code:
[[email protected] ~]# yum install libtool-ltdl.x86_64 libtool.x86_64

yum list |grep libtool |grep installed

libtool-ltdl.x86_64                      1.5.22-6.1             installed
libtool-ltdl-devel.x86_64                1.5.22-6.1             installed

However, note that if you have libtool already installed, if you ‘yum info libtool*,’ yum wont list any of the related packages. I don’t know why this is or how to fix it (or if you even want it fixed) — I simply didn’t investigate. Just know that you’ll have to manually execute the aforementioned statement. The ‘.x86_64’ might be redundant if libtool has already been installed. Make sure to install libtool-ltdl though!

Step two took me a while to figure out. Apparently PHP looks for the libltdl library only at (for x64 at least) /usr/lib64/libltdl.so. The symlink to this file is not included in the libtool packages (for what reason I don’t know — but apparently this has been true and been stumping people since early 2006[1]) so you’ll have to make it yourself. Execute:

Code:
[[email protected] ~]# cd /usr/lib64
[[email protected] lib64]# ln -s libltdl.so.3.1.4 libltdl.so

And that’s it. PHP should configure and compile without error. Well, without this particular error at least.

I hope the time you save reading this will make up for my wasted evening.

Exim useful commands

Just in case i need to remember them at some point 🙂

/usr/sbin/exim -v -Mrm (MAIL ID HERE) ==> REMOVE MAILS BY ID

/usr/sbin/exim -bp ==> LIST QUEUED MAILS

/usr/sbin/exim -bpc ==> OUTPUT NUMBER OF QUEUED MAILS

/usr/sbin/exim -bpr | grep ‘*** frozen ***’ | awk ‘{print $3}’ | xargs exim -Mrm ==> DELETE FROZEN MAILS

/usr/sbin/exim -qff -v -C /etc/exim.conf & ==> DELIVER FORCEFULLY EMAILS

/usr/sbin/exiqgrep -i -f (MAIL ADDRESS HERE) | xargs exim -Mf ==> FREEZE MAILS FROM SENDER

/usr/sbin/exiqgrep -i -f (MAIL ADDRESS HERE) | xargs exim -Mrm ==> REMOVE MAILS FROM SENDER

exim -M email-id ==> Force delivery of one message.

exim -qf ==> Force another queue run.

exim -qff ==> Force another queue run and attempt to flush the frozen message.

exim -Mvl messageID ==> View the log for the message.

exim -Mvb messageID ==> View the body of the message.

exim -Mvh messageID ==> View the header of the message

exim -Mrm messageID ==> Remove message without sending any error message

exim -Mg messageID ==> Giveup and fail message to bounce the message to the Sender

exim -bpr | grep “<” | wc -l ==> How many mails on the Queue?

exim -bpr | grep frozen | wc -l ==> How many Frozen mails on the queue

ModSecurity: Request body is larger than the configured limit

Some users may experience problems when uploading large files via a webpage. The resultant errors often look like this:

Request body is larger than the configured limit (134217728).

This is because the request body’s default maximum size is set lower than the size that was submitted. A detailed explanation of how and why this directive is used can be found here: http://httpd.apache.org/docs/2.0/mod/core.html#limitrequestbody

To work around the issue create a .htaccess file in the same directory as the page causing the issue. Insert one of the following lines:

LimitRequestBody 0 ## unlimited up to 2GB
LimitRequestBody 2097152 ## 2 MB, decent default for many cases.

and save this file. If you increased the size correctly the page should now be working.

HowTo :: enable FancyIndexing on Apache v2.x

IndexOptions
IndexOptions controls the appearance of server generated directing listings, by adding icons, file descriptions, and so on. If Options Indexes is set, the Web server generates a directory listing when the Web server receives an HTTP request for a directory without an index.

First, the Web server looks in the requested directory for a file matching the names isted in the DirectoryIndex directive (usually, index.html). If an index.html file is not found, Apache HTTP Server creates an HTML directory listing of the requested directory. The appearance of this directory listing is controlled, in part, by the IndexOptions directive.

FancyIndexing
This means that a user can re-sort a directory listing by clicking on column headers. Another click on the same header will switch from ascending to descending order. FancyIndexing also shows different icons for different files, based upon file extensions.

The default configuration on a cPanel powered server turns off FancyIndexing. To enable this feature, do the following:

  1. Makse sure the file httpd-autoindex.conf exists in /etc/httpd/conf/extra directory
  2. Add the following directives to your /etc/httpd/conf/httpd.conf file:
      <IfModule mod_autoindex.c>
      IndexOptions FancyIndexing IconHeight=16 IconWidth=16
      </ifModule>
  3. Run this command to restart httpd:
    • /scripts/restartsrv httpd

error while loading shared libraries: libavdevice.so.52: cannot open shared object file: No such file or directory

error while loading shared libraries: libavdevice.so.52: cannot open shared object file: No such file or directory

right, this error usually comes when ldconfig does not know where to search for the libraries, so, in order to help it out,  on my fedora/centos systems, i did the following:

cd /etc/ld.so.conf.d

add another file:  custom-libs.conf

inside, put :

/usr/local/lib

save,

then do ldconfig, et voila.

Enjoy