Docker on CentOS 7 - proxies, yum repo, LVM, ...

By default CentOS 7 (and RHEL) use a not-so-good filesystem for Docker.

Here's the instructions to use Copy-on-Write LVM volumes instead.  This will give a vastly superior experience when using Docker on CentOS 7.

## Setup LVM

rm -rf /var/lib/docker

pvcreate /dev/

vgcreate docker /dev/

lvcreate -l 95%VG -n data docker  ## creates a large data partition

lvcreate -l 100%FREE -n meta docker  ## creates a small metadata partition

You'll need to reconfigure the service too.  This is done by a "drop-in" file for systemd.
If you need proxies for the Docker Engine, configure them here too.

## Reconfigure the service

mkdir /etc/systemd/system/docker.service.d

tee /etc/systemd/system/docker.service.d/custom.conf <<-'EOF'

[Service]

ExecStart=

ExecStart=/usr/bin/docker daemon -H fd:// --storage-driver=devicemapper --storage-opt dm.datadev=/dev/docker/data --storage-opt dm.metadatadev=/dev/docker/meta --iptables=false

Environment='HTTP_PROXY=http://192.0.2.22:8080' 'NO_PROXY=localhost'

EOF

## if this is modified later, do this: systemctl daemon-reload

Now add the Docker Inc. yum repo & install docker:

## Docker repo & install rpm

tee /etc/yum.repos.d/docker.repo <<-'EOF'

[dockerrepo]

name=Docker Repository

baseurl=https://yum.dockerproject.org/repo/main/centos/$releasever/

enabled=1

gpgcheck=1

gpgkey=https://yum.dockerproject.org/gpg

EOF

yum install docker-engine -y

## Systemd service

systemctl enable docker

service docker start

Using parted on large (>2TB) drives

Making aligned partitions, etc.

parted -a optimal /dev/sdx
mklabel gpt
mkpart primary 0% 100%

Use %'s to avoid alignment issues.

Alternatively:
unit bytes
mkpart ...

if you want to get into the guts of things.


Bonus: you can optionally "mark" partitions as their function.
- set 1 lvm on  [for LVM]
- set 1 raid on  [for MD RAID]

Mounting VMDK files in Linux

This is pretty awesome - useful for recovery.

From: http://cromoteca.com/en/blog/mountflatvmwarediskimagesunderlinux/
Here's my extra notes.

First, mount the flat vmdk file as loop0, eg:
losetup /dev/loop0 /srv/data/deadserver-flat.vmdk

Now, investigate the structure with parted, eg:
# parted /dev/loop0
GNU Parted 2.1
Using /dev/loop0
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) unit bytes
(parted) print
Model:  (file)
Disk /dev/loop0: 12884901888B
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start       End           Size          Type     File system  Flags
 1      32256B      131604479B    131572224B    primary  ext3         boot
 2      131604480B  238533119B    106928640B    primary               raid
 3      238533120B  12880788479B  12642255360B  primary               lvm

The figures in the start column can be used as offsets into the loop device to get to the partitions stuck in the vmdk file.

Next, mount the partition from within the vmdk as another loop device, eg:
losetup -o 238533120 /dev/loop2 /dev/loop0

Finally, confirm it worked:
# blkid /dev/loop2
/dev/loop2: UUID="oc9FXU-tSLN-jdy9-1Zpy-XpA1-qgkI-Ng2l4w" TYPE="LVM2_member"

Cool.  Now you can use loop2 as a block-device.

NSS_LDAP timeout towards Active Directory

Suffering from LDAP timeouts on Unix, where it thought it had a TCP connection to Active Directory LDAP - but it didn't really?

ie:
"nss_ldap: reconnected to LDAP server ldap://ad.in.company.com/"
or...
"nscd: nss_ldap: could not search LDAP server - Server is unavailable"

Here's one possible solution: increase the idle timeout in the Active Directory LDAP policies. By default it is 900 seconds.
(ref: http://support.microsoft.com/kb/315071)

> ntdsutil.exe
: LDAP Policies
: connections
: connect to server ad.in.company.com
: q
: Show Values
: set MaxConnIdleTime to 129600
: Show Values
: Commit Changes
: Show Values
: q
: q

This effects all DC's (as far as I can tell).


Now create a cron job in Unix to query the Active Directory (i.e. getent passwd) at an interval of less than MaxConnIdleTime.

Cross platform shell script / batch file (Windows NT & Unix)

Here's a neat little hack I've come up with to script across both Windows and Unix.

It requires a ".bat" extension under Windows, and the execute bit set under Unix.

It uses the GOTO command in Windows to skip the Unix part, and it abuses the stderr redirect in Unix to effectively ignore the Windows IF and GOTO commands on line 1.


if %OS% == Windows_NT goto WINDOWS
then
:
## Hack to make a cross-OS compatible script
fi 2> /dev/null

# ------------------------------------------------------------------------------
# Unix execution
# ------------------------------------------------------------------------------

NAME=$0
NAME=${NAME%.bat}
NAME=${NAME##*/}

echo "My name is ${NAME}, and I work in Unix environments"

exit

# ------------------------------------------------------------------------------
# Windows execution
# ------------------------------------------------------------------------------

:WINDOWS
@echo off

set NAME=%~n0

echo My name is %NAME%, and I work in Windows NT environments"

# ------------------------------------------------------------------------------

esxi raw disk pass through

As a follow up to this post, I have migrated to ESXi.

Here's how to pass through drives as raw in ESXi:

(from the SSH tech support console)

# fdisk -l

Have a look at all the drives on the system, then check how they map to "vml" descriptors:

# ls /dev/disks/ -l

Hopefully you can see something like this:

vml.01000000002020202020202020202020203956533030524859535433313530 -> t10.ATA_____ST31500341AS________________________________________9VS00RHY

Now create a spot for the mappings... such as: /vmfs/volumes/system/rdms/

# cd /vmfs/volumes/system/
# mkdir rdms
# cd rdms

Then create the mapping:

# vmkfstools -z /vmfs/devices/disks/vml.010000000020202020202020202020202039565330344d4254535433313530 rdm02.vmdk -a lsilogic

You should end up with a nicely formatted .vmdk file:

# cat rdm01.vmdk
# Disk DescriptorFile
version=1
encoding="UTF-8"
CID=fffffffe
parentCID=ffffffff
isNativeSnapshot="no"
createType="vmfsPassthroughRawDeviceMap"

# Extent description
RW 2930277168 VMFSRDM "rdm01-rdmp.vmdk"

# The Disk Data Base
#DDB

ddb.virtualHWVersion = "7"
ddb.longContentID = "b2fb0907ebc87f39d99073f1fffffffe"
ddb.uuid = "60 00 C2 9b 29 b5 6d ec-bd 59 29 42 64 db 45 46"
ddb.geometry.cylinders = "182401"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.adapterType = "lsilogic"


Now head back to the GUI and add the harddrive to the desired VM.

( Thank you http://www.vm-help.com/esx40i/SATA_RDMs.php )

Windows Home Premium (Vista) username troubles

The trouble with Windows Vista Home Premium is that it doesn't allow access to the local users and groups snap-in. This means that usernames expected by network devices (other computers, i.e. file servers) cannot be specified for users whilst keeping "pretty names" such as "Joe Blogs" (username: joeb).

Solution is to use: Netplwiz.exe in system32.

Samba / Winbind auth to ADS 2008 R2

I'd been having problems connecting CentOS 5.5 to a Active Directory 2008 R2 domain.

1) use the samba3x packages

Winbind would refuse to show entries in getent passwd, and Samba would not allow domain users access to shares. There were problems observable in a packet trace... for example Wireshark would decode STATUS_LOGON_FAILURE in SMB transactions.

Anyway, it turns out that you MUST specify a "idmap config " for the domain! Otherwise UID/GID mappings fail and then users cannot login.

2) use idmap config configuration options

Resulting configuration looks like this:

[global]
log level = 0

workgroup = ad
password server = ad-box-1.ad.local ad-box-2.ad.local
realm = AD.LOCAL
security = ads
encrypt passwords = yes

idmap uid = 1500-999999
idmap gid = 1500-999999
idmap backend = ad

idmap config AD : backend = ad
idmap config AD : range = 1500-999999

template shell = /bin/bash

ldap ssl ads = no

preferred master = no
local master = no
domain master = no

winbind use default domain = true
winbind offline logon = true
winbind nss info = rfc2307
winbind enum users = yes
winbind enum groups = yes
winbind separator = +

server string = File server


[temp]
comment = samba test server tmp directory
path = /tmp
valid users = @AD+Administrators
read only = No
writeable = Yes
guest ok = No
browseable = Yes

verify timezone data in unix

With daylight savings approaching...

> /usr/sbin/zdump -v /etc/localtime | grep 2010

/etc/localtime Sat Apr 3 15:59:59 2010 UTC = Sun Apr 4 02:59:59 2010 EST isdst=1 gmtoff=39600
/etc/localtime Sat Apr 3 16:00:00 2010 UTC = Sun Apr 4 02:00:00 2010 EST isdst=0 gmtoff=36000
/etc/localtime Sat Oct 2 15:59:59 2010 UTC = Sun Oct 3 01:59:59 2010 EST isdst=0 gmtoff=36000
/etc/localtime Sat Oct 2 16:00:00 2010 UTC = Sun Oct 3 03:00:00 2010 EST isdst=1 gmtoff=39600

> /usr/sbin/zdump -v /etc/localtime | grep 2011

/etc/localtime Sat Apr 2 15:59:59 2011 UTC = Sun Apr 3 02:59:59 2011 EST isdst=1 gmtoff=39600
/etc/localtime Sat Apr 2 16:00:00 2011 UTC = Sun Apr 3 02:00:00 2011 EST isdst=0 gmtoff=36000
/etc/localtime Sat Oct 1 15:59:59 2011 UTC = Sun Oct 2 01:59:59 2011 EST isdst=0 gmtoff=36000
/etc/localtime Sat Oct 1 16:00:00 2011 UTC = Sun Oct 2 03:00:00 2011 EST isdst=1 gmtoff=39600


Perfect : )

GnuCash reports with zero values?

If you do a report in GnuCash for accounts with currency x (say AUD), but the GnuCash default for reports is currency y (say USD), then you will get an error message about zero values, or a report with all zeros.

The solution is to alter the report's currency to match the account's currency.

Why? Because the accounts really do have zero balance in currency y - but that's probably not what you intended to report on. Really you wanted a report on currency x.

Note: you can change the default currency for reports in the preferences.

openSUSE distribution upgrades

To go from openSUSE 10.3 -> 11.0 -> 11.1 -> 11.2 -> 11.3 etc. you can use zypper. This does it all from the Internet, so it's nice and easy (in theory).

At a very high level this is the procedure:

(for 11.1 -> 11.2)

zypper ar http://download.opensuse.org/distribution/11.2/repo/oss/ openSUSE-11.2-oss
zypper ar http://download.opensuse.org/distribution/11.2/repo/non-oss/ openSUSE-11.2-nonoss
zypper ar http://download.opensuse.org/source/distribution/11.2/repo/oss/ openSUSE-11.2-srcoss
zypper ar http://download.opensuse.org/update/11.2/ openSUSE-11.2-update


I generally upgrade the tools before doing the "dup" command, as I had a range of issues doing 10.2 to 10.3 upgrades:

zypper in zypper rpm

Then I go forth and:

zypper dup


I did have tonnes of problems going from 10.2 to 10.3 due to changes in libzypper. I ended up force installing a whole bunch of rpm's to make it work. Messy, but got there in the end.

Ethernet link drop when unlocking Vista

Oddly, my Vista laptop was dropping the wired Ethernet link each time that I unlocked the computer. Very frustrating as all my apps would have to reconnect, etc.

After some quick poking around in the power settings, I've uncovered in the properties of the Intel 82567LM network card:
- Reduce link speed during battery operation
- Reduce link speed during system idle


Uncheck these, and the issue goes away : )



It has to be said, Intel does have some stupid "default" settings on this card. Here's a registry hack to be able to see 802.1p/q headers, which for some reason Intel, by default, doesn't want us to see... bizarre.

http://www.intel.com/support/network/sb/CS-005897.htm

vmware raw disks

Here's an odd setup... but I want to have only one computer turned on 24/7 in my house, this is what I'm doing:

Hardware:

• Asus P5Q3 with 4gb RAM & Intel core 2 duo
• 4 x 1.5TB drives (1.36 REAL terrabytes... damn you harddrive industry)
  

Vista 64-bit on "bare metal"... why Vista? because of the TV Tuner cards (Windows 2003 is no good for this task).

Responsibilities:

• MediaPortal TV server
• Printer Server
• VMware host (VMware Server 2.0)

On this I run 3 x OpenSuSE 11 systems hosted in VMware server, each for a different purpose:

• Asterisk PBX
• Fileserver
• General use host for fun

The crazyness of this system comes about through use of Vista as the base OS, and the fileserver as Linux. I want the linux fileserver to have direct access to each drive.

The drives are partitioned like so:

64gb NTFS partition
1333gb Data partition (ext3) (md software raid5)


Annoyingly VMware server 2 'apparently' removes the feature of raw disks... that is, drives that show up in the VMware without the need to create an intermediate VMware file-based disk.

To get around this, I installed the demo of VMware Workstation 6.5 and created the VMware machines.

Here's the definition of the rawdisk's in the .vmdk files:

# Disk DescriptorFile
version=1
encoding="windows-1252"
CID=a1a071cd
parentCID=ffffffff
createType="fullDevice"

# Extent description
RW 2930277168 FLAT "\\.\PhysicalDrive0" 0

# The Disk Data Base
#DDB

ddb.toolsVersion = "7458"
ddb.adapterType = "lsilogic"
ddb.geometry.sectors = "63"
ddb.geometry.heads = "16"
ddb.geometry.cylinders = "16383"
ddb.uuid = "60 00 C2 9a 4e e3 4f 90-5f 74 f9 c8 0f 2b c1 f0"
ddb.virtualHWVersion = "7"


That makes the drive appear as a SCSI drive in the VMware, despite it being an IDE drive.


To give another example:

# Disk DescriptorFile
version=1
encoding="windows-1252"
CID=b362ecc3
parentCID=ffffffff
createType="fullDevice"

# Extent description
RW 2930277168 FLAT "\\.\PhysicalDrive3" 0

# The Disk Data Base
#DDB

ddb.toolsVersion = "7458"
ddb.adapterType = "lsilogic"
ddb.geometry.sectors = "63"
ddb.geometry.heads = "16"
ddb.geometry.cylinders = "16383"
ddb.uuid = "60 00 C2 9b 2f e0 b6 75-8a da b5 7c 6c eb 0a 96"
ddb.virtualHWVersion = "7"



Then in the .vmx file:

scsi1.present = "TRUE"
scsi1.virtualDev = "lsilogic"

scsi1:0.present = "TRUE"
scsi1:0.fileName = "vm_fileserver_d0.vmdk"
scsi1:0.deviceType = "rawDisk"
scsi1:1.present = "TRUE"
scsi1:1.fileName = "vm_fileserver_d1.vmdk"
scsi1:1.deviceType = "rawDisk"
scsi1:2.fileName = "vm_fileserver_d2.vmdk"
scsi1:2.present = "TRUE"
scsi1:2.deviceType = "rawDisk"
scsi1:3.present = "TRUE"
scsi1:3.fileName = "vm_fileserver_d3.vmdk"
scsi1:3.deviceType = "rawDisk"

SSL countries don't match?

Problem:

server:~/myCA # openssl ca -out certs/server.cert.pem -days 1461 -keyfile private/myCA.key.pem -extensions v3_ca_has_san -config ./openssl.cnf -infiles requests/server.req.pem

Using configuration from ./openssl.cnf
Enter pass phrase for private/myCA.key.pem:
Check that the request matches the signature
Signature ok
The countryName field needed to be the same in the
CA certificate (AU) and the request (US)

server:~/myCA #

Uh-oh? Why can't I sign a certificate with my CA setup in Australia, for a server in the USA?

Simple, the CA's openssl.cnf is required to "match" the country name (and other parameters).
(i.e. requested cert's much match parameters in the signing CA)

Alter these in the CA's openssl.cnf to "supplied" instead:

[ policy_match ]
countryName = supplied
stateOrProvinceName = supplied
organizationName = supplied

SSL keys for a webserver

Here's how I generate SSL keys for a webserver.

In particular I want name-based vhosts with SSL - this can only be done with multiple DNS names in the certificate...

Most of my openssl setup has been guided by:
http://www.phildev.net/ssl/opensslconf.xhtml
http://www.phildev.net/ssl/creating_ca.xhtml
http://www.phildev.net/ssl/managing_ca.xhtml

with a little info from here:
http://wiki.cacert.org/wiki/VhostTaskForce#A1.Way.3ASubjectAltNameOnly


step 1.... get openssl.cnf as you want it

step 2.... add multiple dns names as required
[alt_names]
DNS.1 = example.net
DNS.2 = www.example.net
DNS.3 = example.com
DNS.4 = www.example.com
DNS.5 = mms.example.com
DNS.6 = fancy.example.org


step 3.... generate key:
openssl genrsa -out example.net.key.pem 4096


step 4.... generate cert
openssl req -config ./openssl.cnf -new -key example.net.key.pem -out example.net.req.pem


use one of the names as the CN..
usually I'd say use the DNS A record as the CN (i.e. if example.net -> a specific IP, and that IP's PTR is example.net)... and then put all the C-names in as DNS (as well as the A-rec) (i.e. the DNS aliases, www.example.net = example.net)


step 5.... transfer the req to the CA and sign it
openssl ca -out certs/example.net.cert.pem -days 1461 -keyfile private/theCA.key.pem -extensions v3_ca_has_san -config ./openssl.cnf -infiles requests/example.net.req.pem




how does it end up?

One CN:
Subject: C=AU, ST=Victoria, O=example, OU=example dot net, CN=example.net/emailAddress=security@example.net

Multiple DNS's
X509v3 Subject Alternative Name:
DNS:example.net, DNS:www.example.net[...]

Wii and the Edimax 7206-APg

Tonight I upgraded the Edimax 7206-APg from the somewhat unstable 1.22 firmware, to the newer 1.31. I am surprised they are still developing for this particular model - 1.26 firmware was available for a long long time before 1.31 came out this year.

Anyway, I tried setting "Authentication Type" to shared key, instead of open system... not that it should make a different in a WPA2 environment, but I had a theory that a laptop I have might stop dropping out and then saying "insecure" if I changed it.

On shared key however, the Wii refused to connect (51030). So I tried the auto setting. No good either. Back to open system - now it works.

While thinking about the Wii + wireless, another annoying thing about the Wii is that it requires a B+G wireless environment. Pure G does not work. It has to be both B and G. Frustrating.

My observations are that the Wii starts on 802.11b and then negotiates its' way up the speeds until 11mbit is going, at which point it negotiates its' way onto 802.11g and faster again through all the speeds. Quite bizarre. Why not start at 55mbit and work its' way down if it has to?...