Here's the slides from the presentation I did for the FrontRange PHP Users Group

Front Range PHP NoSQL Databases

View more presentations from Jon Meredith.

Setting up GPG under OS X

Here's a good how to for setting up GPG under OS X - link.

Documenting PHP Extension

I've been trying to use phpdoc with a PHP Extension I'm writing for work and it isn't obvious. Thankfully I found this post about Kristen Chodorow's phpdoc pain

It turns out you have to get phpdoc from svn and use that.

To checkout the doc tree (had to upgrade from the stock svn on OS X leopard first - thanks homebrew!)

svn co http://svn.php.net/repository --depth empty phpdoc
svn co http://svn.php.net/repository/phpdoc/en/trunk --depth infinity phpdoc/en
svn co http://svn.php.net/repository/phpdoc/doc-base/trunk --depth infinity phpdoc/doc-base

And install phd (I had to install sqlite3 first - yes I'm using ports for that)

sudo port install php5-sqlite +debug
sudo pear install doc.php.net/phd-beta

Create an XML template

/path/to/svn/phpdoc/php -d debug_mode=1 -d extension_dir=modules/ -d extension=myext.so /Users/jmeredith/src/php/phpdoc/doc-base/scripts/docgen/docgen.php --output tmp --extension myext

Now you have to merge that into the main php docs

cp -rp tmp /Users/jmeredith/src/php/phpdoc/en/reference/myext

Edit phpdoc/manual.xml.in and add a reference for your extension (I added mine under database vendors)

    &reference.magneto.book;

In phpdoc/en/reference add a file called entities.myext.xml - I'm still tinkering with what needs to go in there, base it on one of the other files in the same directory.

Finally, run phd

phd -d /Users/jmeredith/src/php/phpdoc/doc-base/.manual.book.myext.xml

It creates files in a directory named output. Now all I need to do is work out what to put where to get decent output.

Using NIF for E V I L

A couple of weeks ago at work we had a problem with memory growth in the erlang VM. We could see binaries that were being kept around using process_info(Pid, binary), but the only way we could work out what was in the binary was by attaching gdb and walking the structures.

Using gdb to walk memory structures inside the VM is a pain. Along comes NIF with the R13B03 release which gives me a good excuse to have a go with that instead. After a quick look at the tutorial by Paul Joseph Davis I present sNIFfer - a very naughty peek at the VM internals with no regard to locking or thread safety.

Using the module you can go from an address listed by process_info back to (a copy of) the binary again.

Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false]

Eshell V5.7.4  (abort with ^G)
1> sniffer:start().
ok
2> B= <<"abc">>.
<<"abc">>
3> process_info(self(), binary).
{binary,[{4303709336,3,3}]}
4> sniffer:get_binary(4303709336).   
<<"abc">>

Here is the C code

// sniffer.c
#include 
#include 
#include "erl_nif.h"

#ifdef HAVE_CONFIG_H
#  include "config.h"
#endif

#include "sys.h"
#include "erl_vm.h"
#include "global.h"

static int
load(ErlNifEnv* env, void** priv, ERL_NIF_TERM load_info)
{
    return 0;
}

static int
reload(ErlNifEnv* env, void** priv, ERL_NIF_TERM load_info)
{
    return 0;
}

static int
upgrade(ErlNifEnv* env, void** priv, void** old_priv,
          ERL_NIF_TERM load_info)
{
    return 0;
}

static void
unload(ErlNifEnv* env, void* priv)
{
    return;
}

static ERL_NIF_TERM
get_binary(ErlNifEnv* env, ERL_NIF_TERM a1)
{
  unsigned long mem_loc;
  if (!enif_get_ulong(env, a1, &mem_loc))
    {
      return enif_make_badarg(env);
    }
  else
    {
      Binary* bin_ptr = (Binary*) mem_loc;
      ErlNifBinary nif_bin;

      enif_alloc_binary(env, bin_ptr->orig_size, &nif_bin);
      memcpy(nif_bin.data, bin_ptr->orig_bytes, bin_ptr->orig_size);
      return enif_make_binary(env, &nif_bin);
    }
}

static ErlNifFunc sniffer_funcs[] =
{
    {"get_binary", 1, get_binary}
};

ERL_NIF_INIT(sniffer, sniffer_funcs, load, reload, upgrade, unload)

The erlang module

%% sniffer.erl
-module(sniffer).
-export([start/0, get_binary/1]).

start() ->
    erlang:load_nif("sniffer", 0).

get_binary(_Val) ->
    nif_error(?LINE).    

nif_error(Line) ->
    exit({nif_not_loaded,module,?MODULE,line,Line}).

And a makefile - you'll need to update ERL_TOP to point to your VM source tree.

# Makefile
ERL_TOP=/Users/jmeredith/git/erlang0d
include $(ERL_TOP)/make/target.mk

INCLUDES = \
  -I$(ERL_TOP)/erts/$(TARGET) \
  -I$(ERL_TOP)/erts/emulator/$(TARGET) \
  -I$(ERL_TOP)/erts/emulator/$(TARGET)/opt/smp \
  -I$(ERL_TOP)/erts/emulator/beam/ \
  -I$(ERL_TOP)/erts/emulator/sys/unix \
  -I$(ERL_TOP)/erts/include/$(TARGET) \
  -I$(ERL_TOP)/erts/include/internal \
  -no-cpp-precomp  -DHAVE_CONFIG_H 

# OS X Snow Leopard flags.
GCCFLAGS = -m64 -O3 -fPIC -bundle -flat_namespace -undefined suppress -fno-common -Wall

# Linux Flags
#GCCFLAGS = -O3 -fPIC -shared -fno-common -Wall

CFLAGS = $(GCCFLAGS) $(INCLUDES)
LDFLAGS = $(GCCFLAGS) $(LIBS)

OBJECTS = sniffer.o

DRIVER = sniffer.so
BEAM = sniffer.beam

all: $(DRIVER) $(BEAM)

clean: 
 rm -f *.o *.beam $(DRIVER)

$(DRIVER): $(OBJECTS)
 gcc -o $@ $^ $(LDFLAGS)

$(BEAM): sniffer.erl
 erlc $^

How to get protobuf-c compiled for linking to PHP extensions under OS X

I'm writing a PHP extension at the moment that uses the C variant of Google protocol buffers - here. Unfortunately the PHP build I'm using uses 32-bit code for the commandline php, but 64-bit when running under apache. Here is the magic recipe.


First, build the standard C++/Java protocol buffers code

jons-macpro:protobuf-2.1.0 jmeredith$ cat BUILDIT 
ARCH='-arch i386 -arch x86_64' 
./configure --prefix=/Users/jmeredith/Applications/protobuf CFLAGS="$ARCH" CXXFLAGS="$ARCH" --disable-dependency-tracking && \
make  && \
make install

Then build protobuf-c which uses protobuf

jons-macpro:protobuf-c-0.11 jmeredith$ cat BUILDIT 
## Make sure protoc is in your path or this will die
ARCH="-arch i386 -arch x86_64"
./configure --prefix=/Users/jmeredith/Applications/protobuf-c \
  CXXFLAGS="-I/Users/jmeredith/Applications/protobuf/include $ARCH" \
  CFLAGS="$ARCH"  \
  LDFLAGS="-L/Users/jmeredith/Applications/protobuf/lib" \
  --disable-dependency-tracking && \
make && \
make install

Add this to the extension config.m4

  dnl # protobuf-c
  if test "$PHP_PROTOBUFC" != "no"; then
    PHP_ADD_INCLUDE($PHP_PROTOBUFC/include)
    PHP_ADD_LIBRARY_WITH_PATH(protobuf-c, $PHP_PROTOBUFC/lib, MAGNETO_SHARED_LIBADD)
  fi

And finally, to build your PHP extension

   phpize
   CFLAGS='-arch i386 -arch x86_64' ./configure --enable-yourextension --with-protobufc=/path/to/pb-c && \
   make && \
   sudo make install

Star Wars Ascii Art

I'm writing it here lest I ever forget this gem. Telnet to towel.blinkenlights.nl for the show...

Debugging .NET HTTPS apps

We're currently going through a security audit at work hardening a third party application. One of the improvements is to switch from HTTP to HTTPS which makes verifying the rest of the changes is tricky.

The way I did it was to download/installed Fiddler (http://www.fiddlertool.com/) which is an inspecting proxy. I followed the instructions to enable https decryption, and told Windows to trust the fiddler root certificate (Moved from Personal->Certificates to Trusted Root Certificate Authorities->Certificates).

Checked IE worked and it was fine, then I modified the application.exe.config file under the Docs and Settings\user\Local Settings\Application Data\Vendor\App and added

<configuration>
  <system.net>
    <defaultProxy>
      <proxy proxyaddress="http://localhost:8888" />
    </defaultProxy>
  </system.net>
</configuration>

I thought I was being very clever, but it turns out I couldn't get it to work. So I switched to using HTTPS instead.

Fiddlertool is pretty cool though :)

Dora the Explorer World Adventure workaround

My daughter was heartbroken when we bought this cheap and it didn't work under IE7, so with a little investigation I came up with this workaround rather than 'downgrade to IE6' as Activision recommend.

Download the 'standalone' IE6 from http://browsers.evolt.org/?ie/32bit/standalone

Uncompress it into C:\Program Files\Activision Value\Dora World Adventure

Then bring up a command prompt (Start -> All Programs -> Accessories under Windows XP) and type

        cd "C:\Program Files\Activision Value\Dora World Adventure" 
        copy IEXPLORE.EXE.local DoraAdventure.exe.local 
        exit 

And voila, it should just work. I'd expect this would work for any of your games that rely on IE6 for their presentation engine

iptables rules to lock out multiple failed ssh attempts

Two simple lines to run at boot:

iptables -I INPUT -p tcp --dport 22 -i eth0 -m state --state NEW -m recent --set &

iptables -I INPUT -p tcp --dport 22 -i eth0 -m state --state NEW -m recent --update --seconds 60 --hitcount 4 -j DROP &

Minimal Debian 4.0r0 install on VMWare

Did the install and unchecked all uses except 'base system'. To install the VMware tools, had to add the following packages

psmisc - for killall
linux-headers-`uname -r`
gcc (with deps binutils, cpp, cpp-4.1, gcc-4.1 libssp0)
make

Accepting the defaults for vmware-config-tools seemed to do the trick. Except for the HGFS module - that didn't load on reboot.

Dualboot VMware

I got fed up with running out of disk space so I stuck an old EIDE disk (PATA?) in my Dell to stick my virtual machines on (using the advice from the previous post).

Having two different drive types worked out well. I want to play with Xen when I get a chance, so I thought I'd try setting up CentOS 5 for dual-booting and for running inside a VM when I'm running Windows.

After a quick read of the old docs for VMware 4.5 I found by Googling (http://www.vmware.com/support/ws45/doc/disks_dualboot_ws.html) I created a custom machine configuration that exposed the physical disk to the virtual machine.



Installed CentOS using VMware (because I couldn't get my machine to boot of the DVD-ROM for some reason) without a hitch, rebooted and hit F12 for the boot menu, selected the Master Primary IDE disk (rather than the SATA Master) and it booted right into CentOS.

There are a couple of issues to resolve - the graphics and audio drivers change depending on which environment CentOS is booted in. I need to add something to the init process to switch around the configuration files depending on the boot configuration. I suppose there could be a mechanism built into the OS...

VMWare Workstation Tweaks

I found a useful little article on VMWare disk performance

http://www.virtualization.info/2005/11/how-to-improve-disk-io-performances.html

It recommends disabling anti-virus auto-scanning of the VMDK and VMEM files - I also followed the .ISO recommendation as I attach a lot of them for installation/liveCD testing.

Wot no updates

So much for blogging - haven't updated this in weeks. I'm working on a couple of projects - replacing my phone service with Asterisk running on my Linksys router and I'm toying with integrating Bacula with the Amazon S3 service so I can stop worrying about backups.

I'm also trying out the CentOS 4.92 beta. Running the hgfs driver for shared folders with SELinux enabled caused the kernel to panic. Once I disable it, it worked fine. 'dmesg' had an error in it about hgfs not supporting labelling so I thought I'd get rid of it.

SQLServer 2005 trigger debugging

They've removed the T-SQL debugger in SQLServer 2005. Luckily I have 1 day left of my Visual Studio 2005 trial.

The way to debug a trigger is to create a stored procedure that exercises the trigger (mine just did an update on the table), set a breakpoint in the trigger where you want, and then step into the *stored procedure* by right clicking on it. Bobs your uncle, VS grinds away and you get to your breakpoint (eventually).

Fedora setup

Ok, on to setting up FC6. I've installed the base FC6 and picked out the development and MySQL packages. On configuration, I disabled the firewall and SELinux during testing. We'll decide what do about those on deployment.

Next I instaled the VMware tools (thanks for the mouse pointer help here http://www.thoughtpolice.co.uk/vmware/howto/fedora-core-6-vmware-tools-install.html and the vmxnet diver help here http://www.vmware.com/community/thread.jspa?messageID=556834).

To make sure the two are synchronised I'm planning to setup NTP on the second node pointing to the first one (I found instructions here http://wiki.novell.com/index.php/SUSE_Linux_Enterprise_Server) - but I want to get MySQL configured and running on the master first before I clone the system.

Testing MySQL Clustering

At work we're setting up a pool of LAMP servers and they will need some persistent storage. We're using MySQL in house for our other relational needs so I'm having a go at getting a 2-node high-availability/shared nothing/load balanced database cluster working.

I can't access the actual hardware yet, so I'm playing in a couple of virtual machines. The first step is to get them installed and stick MySQL on there. The LAMP servers are all using FC6 (for better or worse - maybe CentOS would have been better) so I'll use that. First step, getting the first node working so I can clone it and try out replication.

Built myself a packet sniffer

I had an old Pentium IV 2.0Ghz sitting around with 256Mb of broken RAM and after pondering what to do with it, I decided to build a packet sniffer. I got the machine very cheap ($20) because it's previous owner couldn't understand why it kept crashing. Running memtest86 found problems with the 0x0105cxxx addresses so I decided to stick a copy of Fedora Core 6 on it and add the BadRAM patch (http://rick.vanrein.org/linux/badram/) - which was harder than expected. I tried installing with a 'mem' limit under the threshold (about 80Mb), but it wouldn't install, so I ended up pinching some working memory out of a different system to build it. Building the kernel RPM took a long time and I think the BadRAM patches conflict with some of the others in the Fedora SPEC file as I had to manually tweak one of the files between the prep and build stage to get it to work. I'll dig deeper next time Fedora release an SRPM for the kernel.

Anyway, I eventually got the kernel build and installed with the BadRAM parameters set correctly (so now I have 256Mb-12kb memory available) and it runs like a charm. I put a couple of extra NICs in the machine, installed Wireshark/ntop and then stuck it between my Cablemodem and my WRT-54GS to see what was coming over the wire. I left Wireshark capturing packets overnight to find out and the answer is - a lot of ARP packets. I got 60Mb of them and very little else sent over my cable modem in 8 hours. The next step is to work out why....

openSUSE 10.2 and VMWare Workstation

I'm interested in comparing openSUSE and SLES so I've installed openSUSE 10.2 under VMware 5.5.3. To get it to work I had to use the text mode installation (I think it was either F2 or F3 then choose text mode). After a few false starts in graphical mode, that fixed it.

My brain dump

It seems like I keep losing notes I make about everything, so I thought I'd try keeping it in a blog - like a lab notebook, but searchable.