Kind of a repository for daily findings

It's certainly not a blog, more of a place to gather all the random bits and pieces of information that one finds daily.
Bits, pieces, tricks, syntaxes to remember.
Stuff that's maybe useful for someone (or at least to aid my own memory).

Unless otherwise noted, everything applies to Debian GNU/Linux - because that's what I like and use.

Maybe some day the google-ads will at least pay for some beers.

Index

Easy OpenVPN setup with pre-shared keys

More often than not, I just need a quick&dirty tunnel from A to B, and I certainly don't want to deal with CAs (they have their uses, though).
OpenVPN makes this easy enough (though not easy enough to find in the docs or on the website (which seems mighty commercial nowadays)).

This is for Debian GNU/Linux "lenny" (5.0++), though IIRC was the same for "sarge" and "etch". 

aptitude install openvpn

Server

Client

Hint

If it doesn't start, check /var/log/syslog and if it can't open /dev/net/tun:

Xen

If you want to set up a DomU with a Debian distro different than what runs in your Dom0 (newer or older, doesn't matter), you can tell xen-create-image which distro to set up with: 
xen-create-image --dist lenny
If it complains that it hasn't got a clue about it, look in /usr/lib/xen-tools/ and make another symlink, like 
ln -s debian.d lenny.d

Also the logfile xen-create-image produces is rather useful - especially if it hangs and tells something like "don't wanna remove an essential package" (just press enter a couple times in the console where you started xen-create-image).

My standard image creation is: 

http_proxy="http://proxy.invalid:8080/" xen-create-image  --dist lenny --lvm large_volume_group --ip 10.249.48.67 --gateway 10.249.48.1 --netmask 255.255.255.0 --mem 128Mb --swap 256Mb --size 2Gb --hostname newxen
(yes, it (probably wget started from debootstrap) honors http_proxy!)

Starting a degraded array quickly with mdadm

mdadm --assemble --uuid=55a7b946:ff8ae0c1:f012f582:26f0f5d7 /dev/md0 /dev/sdb1
The magic is the --uuid bit - if you don't have it written down, you can find it out via file -s, for example (other host): 
file -s /dev/sdb1
/dev/sdb1: Linux rev 1.0 ext3 filesystem data, UUID=b451e67b-3d22-404b-91cb-56cfe84723f (needs journal recovery) (large files)
Substitue "-" with ":" as appropriate.

dmesg in a loop

If you're like me, you want to keep half an eye on dmesg, especially when you're connecting USB devices etc. often.

So I usually have at least one Eterm constantly open with:: 

while true; do dmesg -c; sleep 10; done

lm-sensors

The PC of my sigificant other has a no-name mainboard, which features a SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 3a) chip that mbmon doesn't recognize.
But lm-sensors does, and identifies it (via sensors-detect, thankfully) as 
Driver `f71882fg' (should be inserted):
  Detects correctly:
  * ISA bus, address 0x225
    Chip `Fintek F71882FG/F71883FG Super IO Sensors' (confidence: 9)

Driver `k8temp' (should be inserted):
  Detects correctly:
  * Chip `AMD K8 thermal sensors' (confidence: 9)
So, just load these modules via /etc/modules, and then, voila: 
root@dim:~# sensors
acpitz-virtual-0
Adapter: Virtual device
temp1:       +63.0�C  (crit = +70.0�C)                  

k8temp-pci-00c3
Adapter: PCI adapter
Core0 Temp:  +61.0�C                                    
Core0 Temp:  +62.0�C                                    
Core1 Temp:  +59.0�C                                    
Core1 Temp:  +63.0�C
So, no fan-speeds to read out, but CPU temperature (per core, even) is a nice start.

On Ubuntu (current as of June 9, 2010 - I can never be bothered to remember their naming/versions as I only run it on the SOs PC) gkrellm was the application with the least resistance for watching the temp(s) with a GUI.

Various tools to find info about hardware

From "what kinda DIMM is in which socket" to "which USB devices are there".

AMD/Intel CPUs and frequency scaling

I have an "AMD Athlon(tm) II X2" dual-core CPU in my main workstation, which is capable of frequency scaling and thus can save quite some electricity (25-35W on my workstation at home - linux-image-2.6.30-bpo.2-686-bigmem from http://backports.org/ at the time of writing this).

For this to work the following modules need to be loaded: 

powernow_k8            12228  0 
cpufreq_conservative     6256  0 
cpufreq_powersave       1292  0 
cpufreq_stats           3520  0 
cpufreq_userspace       2740  0 
processor              34508  1 powernow_k8
And at least "cpufrequtils" installed (I also have "cpufreqd", but I'm not sure it's necessary, and no interest right now to find out).
I initialize it via the following snippet: 
#!/bin/bash

echo -n "Setting lowest possible CPU-speed to 800 MHz..."
/usr/bin/cpufreq-set -c 0 -g ondemand
/usr/bin/cpufreq-set -c 0 -d 800
/usr/bin/cpufreq-set -c 1 -g ondemand
/usr/bin/cpufreq-set -c 1 -d 800
echo "done."

exit
For the "Intel(R) Core(TM)2 CPU" in the workstation at, well, work, it works much the same, only difference in setup is that it's governed by acpi-cpufreq instead of powernow-k8 (look in /lib/modules/`uname -r`/kernel/arch/x86/kernel/cpu/cpufreq to see what's available) and that the possible speeds are different (use cpufreq-info to find out).
Or, at least, that was the difference: with Debians linux-image-2.6.26-2-686 the ACPI method doesn't work anymore (Error inserting processor ... No such device), though it did with the one before (2.6.22? 2.6.24?). If I have some spare time I'll try and see if 2.6.30/2.6.32 behave again.

HD MKV (Matroska) to AVI w/ MPEG4

A friend recorded some videos for me, "helpfully" in some HD mode that my "Xtreamer" hardware player can't play. 
General
Complete name                    : 2007.07.25.mkv
Format                           : Matroska
File size                        : 1.43 GiB
Duration                         : 1h 2mn
Overall bit rate                 : 3 301 Kbps
Encoded date                     : UTC 2007-07-26 06:36:48
Writing application              : mkvmerge v2.0.2 ('You're My Flame') built on Apr 26 2007 10:34:00
Writing library                  : libebml v0.7.7 + libmatroska v0.8.1

Video
ID                               : 1
Format                           : AVC
Format/Info                      : Advanced Video Codec
Format profile                   : [email protected]
Format settings, CABAC           : Yes
Format settings, ReFrames        : 6 frames
Muxing mode                      : Container [email protected]
Codec ID                         : V_MPEG4/ISO/AVC
Duration                         : 1h 2mn
Bit rate                         : 2 852 Kbps
Nominal bit rate                 : 2 915 Kbps
Width                            : 1 280 pixels
Height                           : 720 pixels
Display aspect ratio             : 16:9
Frame rate                       : 25.000 fps
Resolution                       : 8 bits
Colorimetry                      : 4:2:0
Scan type                        : Progressive
Bits/(Pixel*Frame)               : 0.124
Stream size                      : 1.24 GiB (86%)
Writing library                  : x264 core 55 svn-663
Encoding settings                : cabac=1 / ref=3 / deblock=1:-2:-1 / analyse=0x3:0x133 / me=umh / subme=6 / brdo=1 / mixed_ref=0 / me_range=16 / chroma_me=0 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=0 / threads=3 / nr=0 / decimate=1 / mbaff=0 / bframes=3 / b_pyramid=1 / b_adapt=1 / b_bias=0 / direct=3 / wpredb=1 / bime=1 / keyint=250 / keyint_min=25 / scenecut=40(pre) / rc=2pass / bitrate=2915 / ratetol=1.0 / rceq='blurCplx^(1-qComp)' / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / pb_ratio=1.30
Language                         : English

Audio
ID                               : 2
Format                           : AC-3
Format/Info                      : Audio Coding 3
Format_Settings_ModeExtension    : CM (complete main)
Codec ID                         : A_AC3
Duration                         : 1h 2mn
Bit rate mode                    : Constant
Bit rate                         : 384 Kbps
Channel(s)                       : 6 channels
Channel positions                : Front: L C R, Side: L R, LFE
Sampling rate                    : 48.0 KHz
Stream size                      : 170 MiB (12%)
Language                         : English
After much fiddling around with transcode and mencoder and everything else on this earth, I found something that actually works in this case (don't get me wrong, transcode is great - only not with AC3 audio!). 
OLDIFS=$IFS
export IFS=$'\n'       # spaces in filenames are ok now
for FILE in `ls -1 *.mkv`; do
  NEWNAME=`echo "$FILE" | sed s/"\.mkv$"//g`
  ffmpeg -i "$FILE" -s 512x328 -ac 2 -ab 128k "00_$NEWNAME.avi"
done
export IFS=$OLDIFS
That now puts out 
00_2007.07.25.avi: RIFF (little-endian) data, AVI, 512 x 328, 25.00 fps, video: FFMpeg MPEG-4, audio: MPEG-1 Layer 1 or 2 (stereo, 48000 Hz)
This scales down to PAL resolution (the -s), sets 128kbit for audio, mixes down to 2 channels (eg stereo), and outputs in MPEG4/mp2. You can leave out the -s if you want the same resolution in your output as in your input (HD video at 1920�1080 or so takes up senseless amounts of diskspace if your TV can't display it anyway).

Another, maybe better, way

At least my "Xtreamer" player has problems with "[email protected]" format profile (use mediainfo to get detailed information about most video files).
Re-muxing the .mkv-file via mkvmerge -i oldfile.mkv newfile.mkv gets the format profile thingie to "[email protected]", it's lossless (I'm told) and fast - the downside, of course is that you get no space-saving.

Also, for problems with "classic" Xvid-in-AVI, one can re-mux with 'MPEG4 Modifier' (see http://moitah.net/), add and then re-remove (or was it the other way around) the "packed bitstream" bit (I couldn't find the Linux version, and was in a hurry, so I took the easy way out via wine): 

wine /data/win/mmcl.exe --pack oldfile.avi newfile.avi
(and then again, just with "--unpack").

iotop

If you ever wondered which darn process was using up all the I/O (hence "iowait" in top), iotop is your friend. 
Total DISK READ: 11.68 K/s | Total DISK WRITE: 70.07 K/s
  PID USER      DISK READ  DISK WRITE   SWAPIN    IO>    COMMAND                
 3459 proxy      11.68 K/s       0 B/s  0.00 %  1.90 % (squid) -D -YC
 2118 root           0 B/s   23.36 K/s  0.00 %  0.49 % [kjournald]
 2608 root           0 B/s   46.71 K/s  0.00 %  0.00 % syslogd -r
    2 root           0 B/s       0 B/s  0.00 %  0.00 % [kthreadd]
    3 root           0 B/s       0 B/s  0.00 %  0.00 % [migration/0]
...
Especially useful when all you see in classic top is a.) iowait and b.) kcryptd (for those with sensibly encrypted filesystems).

Problems with "Azalia (Intel HDA)" (snd-hda-intel) sound on an "Asus M4A88TD-M Evo" mainboard

I recently had to get a new mainboard, and with this I have trouble with the on-board sound device about every other boot:
Sep  8 09:45:51 fsck kernel: [    6.280194] hda-intel: Error creating card!
Sep  8 09:45:51 fsck kernel: [    6.280239] HDA Intel: probe of 0000:00:14.2 failed with error -16
(Re-)inserting the snd-hda-intel module doesn't help, but alsa force-reload snd-hda-intel does. <shrug>

Judging from other problems I have with this board, also about every other boot (but not necessarily at the same time as the sound problem) I can only conclude that this board is pure unadulterated shit (despite the hefty price-tag!), and should be avoided at all costs.

Sometimes nothing will get sound of the fucking Azalia thingie, then it's reboot time (with 30 seconds of hard power-off).

GRUB problems with Knoppix and/or chroot-ed Debian Lenny

After a recent hardware failure I was forced to restore my whole system from backup. So after I had everything on a new disk, I booted up Knoppix, chrooted into the system, and ran grub-install /dev/sda. But while this seemed to install GRUB just fine, and I got the menu all right at boot-time, there was something wrong with the root device (or, what, third-stage?), and the kernel never loaded the needed SATA modules from the RAM-disk (error message was something like "ramdisk creation failed"), and thus couldn't find its root device.
What helped here was to install GRUB neither from the chrooted-into Debian Lenny nor from Knoppix, but from a Debian Lenny "netinst" CD.
No idea why.

Converting .FLV (downloaded YouTube) video

ffmpeg -i file.flv file.mpg (or file.avi, if MPEG-4 is more to your liking), that's it.
It's not all that fast (about 50 FPS on my Athlon Phenom 3.1 GHz), but it works and the result looks just as bad in mplayer as the "original" flash video.

Converting DVDs for standalone player consumption the easy way

HandBrakeCLI is the way to go, f'rex: 
HandBrakeCLI -i /dev/dvd -C 2 -t 1 -f mkv -e ffmpeg -S 600 -a 2,1 -E lame -c 1 -o "$FILE1"
For title 1, chapter 2, with 2 audio tracks (which in my case are german and english, and setting them with 2,1 re-sorts them to first english, then german), 600MB MKV output.
The result plays perfectly fine (with both languages, the first one is default) on my "Xtreamer" player (hint: for mplayer, use -aid 0/1 for language selection.
(I could play around with space requirements, but in my experience with files from other sources, 350MB is not enough for decent quality for 45 minutes of video with one audio-stream, and 700 MB is too damn much, so I guesstimated 600 MB for one video with two audio-streams.)
Oh, and HandBrakeCLI --help gives actually helpful, and not too detailed, information - for more options go just use good old transcode or DVD::Rip (Debian: dvdrip).

Also, for whatever reason, all the other "easy DVD converting" software packages I tried briefly, none of them would actually use more than one CPU core, but with this it uses 180-190% of CPU (according to top), which means 2 threads, each eating one CPU core. I like that - 6 minutes for a 45 minute video, and that's incl. ripping.

Anjuta - a C development IDE that sucks a bit less than Eclipse/KDevelop

Because I needed to write more than 40 lines of C code for a pet project of mine I had a long hard look at what's available for IDEs nowadays. 20 years ago I got hooked on Turbo C/Turbo Pascal, and have been looking for something as similar as possible ever since, but no such luck.
The upsides of the Turbo IDE for me were mainly: For the last couple years, everyone not already somehow hooked en Emacs (which I loathe, because I'm squarely on the "vi" side of the fence) has been talking about how great Eclipse is. Im my experience, though, it's akin to the Emacs of ages past (Eight Megs And Constantly Swapping), plus the initial learning curve is steep. Way easier to continue debugging with lots of printf()s. Don't get me wrong, it's probably perfectly fine for huge projects that have a gazillion people working on it, but I don't care about that.
Then someone mentioned KDevelop, which installed straight forward, but needs too much knowledge of the whole automake/autoconf mess for small projects, plus I never could get the gdb interface to cooperate.

But Anjuta mostly got it right, once you get the hang of not using your own pre-existing files, but start from a fresh "project" and hang your code from the generated "main.c", not one of your own files. The debugger Just Works, watches and breakpoints Just Work, just the editor isn't great (but at least it ain't Emacs), but syntax-highlighting does what it's supposed to do.
Still, what I really want is Turbo C, not something Knome-ified, but whatever.

Amarok: "xine couldn't initialize audio drivers"/"xine konnte keine Audiotreiber initialisieren"

If you get this in Amarok, but sound otherwise works fine:
just try & install amarok-engine-yauap and use that instead of the "xine"-engine.

Don't ask me why, I don't know and don't care.

Getting Amarok to play back FLAC

If you're using the "xine" backend, you probably won't get FLAC playback.
Install a different backend, like phonon-backend-vlc, then go to Settings->Configure Amarok->Playback->Configure Phonon->Backend and move the "VLC" thing to the top. Now all of a sudden it'll recognize FLAC files in the collection and will also happily play them.

(Amarok 2.3.1, phonon-backend-xine 4:4.6.0really4.4.2-1, phonon-backend-vlc 0.2.0-2 when testing.)

Western Digital (WDC) "Caviar Green" disks and Idle Timeout ("IntelliPark")

"Modern" WD disks are quite aggressive with regard to trying and save power - so aggressive, in fact, that they'll go and park the read-/write-heads after just 8 seconds.
This often means trouble, as some filesystems per default flush their buffers every 10 or 30 seconds, and certain applications, such as mplayer under some circumstances, read data every 10 seconds, thus nearly instantly re-waking the drive.
And those drives are only spec'd for 300-1,000k load/unload cycles for their heads.

In my setup, this lead to ~600 load/unload cycles per day, which would've meant I'd reach the 300k in ~500 days -> 1.5 years. And with my luck with disk drives, it would've instantly fallen over dead right then.

Affected models, according to WD: WD20EADS, WD20EARS, WD15EADS, WD15EARS, WD10EADS, WD10EARS, WD8000AARS, WD7500AADS, WD7500AARS, WD6400AADS, WD6400AARS, WD5000AADS, WD5000AARS

You can check with smartctl (from package smartmontools): 

smartctl -a /dev/sdX | egrep "(Power_On_Hours|Load_Cycle_Count)"
and knowing how much cycles/hour resp. /day you get is just a bit of easy math away.

WD claims that you can disable this timeout via hdparm -B, but on my box this just throws an error, and has no noticable effect otherwise.
But if you click further on that page, you'll get to a tool called "RE2GP Idle Mode Update Utility" or "wdidle3". Of course, this tool is for DOS - DOS, for fucks sake! - which means you'll get to fiddle around with FreeDOS.
I'll give you the short route for how I finally, after many a detour and senseless reboots, got the WD tool to run:

As an aside: hddtemp allegedly wakes up drives - smartctl -a | grep Temp allegedly doesn't. And smartd (which comes with package smartmontools wakes 'em up, too, so fiddle with the config file (/etc/smartd.conf) so as to not throw away the above effort.

Temperature-controlled fan on a QNAP TS-41x

According to http://www.cyrius.com/debian/kirkwood/qnap/ts-41x/tips.html qcontrol can't vary the speed of the system fan depending on either system or harddisk temperature, so I hacked together a fugly little script to do just that. YMMV, YHBW.
Here it is: fancontrol.sh

exmh and glimpse "not initialized"

Remember that at least Debians exmh looks for glimpse in /usr/bin/glimpse (and for glimpseindex, too).

Enabling/installing SSH on an Xtreamer/XtreamerPro

The answer is: install XMP (Xtreamer Mod Pack), see http://blog.mameso.com/2010/03/xtreamer-xtreamer-mod-packxmp-xtreamer-web-sdk/. Maybe now Auntie 'oogle will find the answer to this frequently-asked question.

From my experience, you will have to repeat this installation after every firmware upgrade.

Another hint: top is an ipkg update; ipkg install procps away. Please also set a root password after first login.

Re-encoding h/x.264-in-mkv so the Xtreamer/Pro is happy

Seems the Xtreamer/XtreamerPro boxen are decidedly unhappy when an x/h.264 encoded video has more than 4 "ReFrames" (whatever that is).
Here's the HandBrakeCLI commandline that sets this right: 
                   nice HandBrakeCLI -i "$FILE" -o "$FIX"       \
                                     -q "$QUALITY" -f mkv       \
                                     -C "$MAXCPU" -e x264       \
                                     -a "$AUDIOS" -E "$COPIES"  \
                                     -s "$SUBS"                 \
                                     --markers -2 -T

I've also put this in a little shell wrapper so I can just go over the whole collection in one go (keeping all audio and subtitle tracks intact), and delete the sources if the re-encode looks fine: mkv-fix.sh (help is inline).
Be warned, this works for me, it might not work for you, it might destroy your illegitimately gotten flicks, yadda, yadda. Use at your own risk.
Files will end up having Fixed- prefixed to their name.

Re-encoding movies so they'll play fine on Android phones

Tested with Android 2.3.5 on an HTC Wildfire S.

HandBrakeCLI commandline: 
HandBrakeCLI -i "$SOURCE" \
             -o "$TARGET" \
             -Z "iPod Legacy" \
             -X 480 -Y 320 \
             -2 -T \
             -b 500 -B 128

As usual, I've put this in a little shell helper: mkv-mp4.sh.txt. Use at your own risk, yadda, yadda.

The options are a compromise between filesize and quality - good enough for the tiny little screen. Mind that the built-in video player can't cope with subtitles or multiple audio tracks, so you should make sure to select the right audio track (with -a).

Re-set a sensible X11 resolution

If something (glaring at games here) has set your screen resolution to a nice&steady 640x480, and after exiting left it there, run xrandr -s 2048x1152 (or whatever resolution you desire).

(I can never remember this, and invariably start with apropos resolution, and looking at the man-pages of xset/xsetmode first.)

In case of XFCE4 session trouble

If you (like me) like to have a separate X-server with XFCE4, for the occasional game or so, and suddenly it displays a terminal with some debug output, and won't start any terminal emulator (though other programs run fine, like gamix): startx /usr/bin/xfce4-session -- :1. Fucker, for whatever reason, now needs to be called with full path, even when /usr/bin/ is in $PATH.

dm-cache setup hints

I want to use dm-cache to utililze an SSD as a cache for a normal spinning-rust device. I've built a 3.9-series kernel with the relevant options enabled.
But the docs were incomprehensible - so here's what I used to get it going:
dmsetup create dmcachehome --table "0 3586532418 cache /dev/sdc1 /dev/sdc2 /dev/sda3 512 0 mq 4 sequential_threshold 1024 random_threshold 8"
The first big number (3586532418) is the size of the origin device in sectors - find out via blockdev --getsz /dev/sda3.
The separation of metadata/cachedevice is senseless for my situation, but I just need to live with it, I guess, so I made 2 partitions on my SSD - 10GB+110GB - don't make the "metadata" device too big, otherwise you'll end up with something like this (I had 20GB for metadata in that try): 
[30520.720233] device-mapper: cache: Metadata device sdc1 is larger than 33423360 sectors: excess space will not be used.
[30520.775940] device-mapper: space map common: space map too large
[30520.776059] device-mapper: transaction manager: couldn't create metadata space map
[30520.776174] device-mapper: cache metadata: tm_create_with_sm failed
[30520.857791] device-mapper: table: 253:0: cache: Error creating metadata object
[30520.857959] device-mapper: ioctl: error adding target to table
I have no idea whatsoever how optimal sizing between metadata/cache is determined, I found the above via trial&error.

It is perfectly possible to use partitions etc., one needn't use whole devices - for metadata/cache/origin, any block device (sdc, sdc1, md0, /dev/disk/by-uuid/281230...) will do (haven't tried with container files, though).

You will not need to re-create a filesystem if you've already got one on your "origin" - in my example above there was already a filesystem (/home) on dev/sda3. And it's perfectly fine to use it directly (via mount /dev/sda3 /home) if you don't want it cached anymore, just dmsetup remove dmcachehome first.

Anyway, if you've got it right, you can then mount /dev/mapper/dmcachehome /home (substitute your own device names/mount points).

Some other dm-cache resources:

Mind, cache.txt and cache-policies.txt are also in your own kernel sources (3.9.x) (Documentation/device-mapper/cache.txt / cache-policies.txt), and should be more relevant to your situation than the random versions off the 'net mentioned above.

Why there's no STARTTLS on my MX

For SMTP at least, I find the notion of STARTTLS inherently broken. You have no control if someone'll use it, if that someone will check your certificate, and if you check someone elses cert, what're you supposed to do if it's self-signed, not for the correct hostname, or with a dangerously weark cipher/keylength? And the next hop (hey, there's smarthosts, .forwards, the works) might not bother. Or the one after that. Or your correspondent uses POP/IMAP w/o SSL for mailbox access.
Let's face it, for something inherently NOT end-to-end like SMTP, it makes no sense. I'll much rather use PGP/GnuPG, get quite closer to real end-to-end encryption and get at least some semblance of control over ciphers used.

The only sensible argument for SMTP+STARTTLS, IMO, is to -maybe!- annoy some Three Letter Agencies. This is worthwhile all in itself, but places it rather near the bottom of the "when I get a round tuit" pile.

BackupPC complaining about "getStats"

If you, like me for example, have recently moved BackupPC and the spool to a different machine, and now it sits there going Can't call method "getStats" on an undefined value at /usr/share/backuppc/bin/BackupPC_dump (line number may differ): do you have all necessary stuff installed∓available?
Me, I was missing libfile-rsyncp-perl ....
$Id: index.html,v 1.74 2014/11/23 08:24:21 waldner Exp $