Here's the slides from the presentation I did for the FrontRange PHP Users Group

Front Range PHP NoSQL Databases

View more presentations from Jon Meredith.

Setting up GPG under OS X

Here's a good how to for setting up GPG under OS X - link.

Documenting PHP Extension

I've been trying to use phpdoc with a PHP Extension I'm writing for work and it isn't obvious. Thankfully I found this post about Kristen Chodorow's phpdoc pain

It turns out you have to get phpdoc from svn and use that.

To checkout the doc tree (had to upgrade from the stock svn on OS X leopard first - thanks homebrew!)

svn co http://svn.php.net/repository --depth empty phpdoc
svn co http://svn.php.net/repository/phpdoc/en/trunk --depth infinity phpdoc/en
svn co http://svn.php.net/repository/phpdoc/doc-base/trunk --depth infinity phpdoc/doc-base

And install phd (I had to install sqlite3 first - yes I'm using ports for that)

sudo port install php5-sqlite +debug
sudo pear install doc.php.net/phd-beta

Create an XML template

/path/to/svn/phpdoc/php -d debug_mode=1 -d extension_dir=modules/ -d extension=myext.so /Users/jmeredith/src/php/phpdoc/doc-base/scripts/docgen/docgen.php --output tmp --extension myext

Now you have to merge that into the main php docs

cp -rp tmp /Users/jmeredith/src/php/phpdoc/en/reference/myext

Edit phpdoc/manual.xml.in and add a reference for your extension (I added mine under database vendors)

    &reference.magneto.book;

In phpdoc/en/reference add a file called entities.myext.xml - I'm still tinkering with what needs to go in there, base it on one of the other files in the same directory.

Finally, run phd

phd -d /Users/jmeredith/src/php/phpdoc/doc-base/.manual.book.myext.xml

It creates files in a directory named output. Now all I need to do is work out what to put where to get decent output.

Using NIF for E V I L

A couple of weeks ago at work we had a problem with memory growth in the erlang VM. We could see binaries that were being kept around using process_info(Pid, binary), but the only way we could work out what was in the binary was by attaching gdb and walking the structures.

Using gdb to walk memory structures inside the VM is a pain. Along comes NIF with the R13B03 release which gives me a good excuse to have a go with that instead. After a quick look at the tutorial by Paul Joseph Davis I present sNIFfer - a very naughty peek at the VM internals with no regard to locking or thread safety.

Using the module you can go from an address listed by process_info back to (a copy of) the binary again.

Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false]

Eshell V5.7.4  (abort with ^G)
1> sniffer:start().
ok
2> B= <<"abc">>.
<<"abc">>
3> process_info(self(), binary).
{binary,[{4303709336,3,3}]}
4> sniffer:get_binary(4303709336).   
<<"abc">>

Here is the C code

// sniffer.c
#include 
#include 
#include "erl_nif.h"

#ifdef HAVE_CONFIG_H
#  include "config.h"
#endif

#include "sys.h"
#include "erl_vm.h"
#include "global.h"

static int
load(ErlNifEnv* env, void** priv, ERL_NIF_TERM load_info)
{
    return 0;
}

static int
reload(ErlNifEnv* env, void** priv, ERL_NIF_TERM load_info)
{
    return 0;
}

static int
upgrade(ErlNifEnv* env, void** priv, void** old_priv,
          ERL_NIF_TERM load_info)
{
    return 0;
}

static void
unload(ErlNifEnv* env, void* priv)
{
    return;
}

static ERL_NIF_TERM
get_binary(ErlNifEnv* env, ERL_NIF_TERM a1)
{
  unsigned long mem_loc;
  if (!enif_get_ulong(env, a1, &mem_loc))
    {
      return enif_make_badarg(env);
    }
  else
    {
      Binary* bin_ptr = (Binary*) mem_loc;
      ErlNifBinary nif_bin;

      enif_alloc_binary(env, bin_ptr->orig_size, &nif_bin);
      memcpy(nif_bin.data, bin_ptr->orig_bytes, bin_ptr->orig_size);
      return enif_make_binary(env, &nif_bin);
    }
}

static ErlNifFunc sniffer_funcs[] =
{
    {"get_binary", 1, get_binary}
};

ERL_NIF_INIT(sniffer, sniffer_funcs, load, reload, upgrade, unload)

The erlang module

%% sniffer.erl
-module(sniffer).
-export([start/0, get_binary/1]).

start() ->
    erlang:load_nif("sniffer", 0).

get_binary(_Val) ->
    nif_error(?LINE).    

nif_error(Line) ->
    exit({nif_not_loaded,module,?MODULE,line,Line}).

And a makefile - you'll need to update ERL_TOP to point to your VM source tree.

# Makefile
ERL_TOP=/Users/jmeredith/git/erlang0d
include $(ERL_TOP)/make/target.mk

INCLUDES = \
  -I$(ERL_TOP)/erts/$(TARGET) \
  -I$(ERL_TOP)/erts/emulator/$(TARGET) \
  -I$(ERL_TOP)/erts/emulator/$(TARGET)/opt/smp \
  -I$(ERL_TOP)/erts/emulator/beam/ \
  -I$(ERL_TOP)/erts/emulator/sys/unix \
  -I$(ERL_TOP)/erts/include/$(TARGET) \
  -I$(ERL_TOP)/erts/include/internal \
  -no-cpp-precomp  -DHAVE_CONFIG_H 

# OS X Snow Leopard flags.
GCCFLAGS = -m64 -O3 -fPIC -bundle -flat_namespace -undefined suppress -fno-common -Wall

# Linux Flags
#GCCFLAGS = -O3 -fPIC -shared -fno-common -Wall

CFLAGS = $(GCCFLAGS) $(INCLUDES)
LDFLAGS = $(GCCFLAGS) $(LIBS)

OBJECTS = sniffer.o

DRIVER = sniffer.so
BEAM = sniffer.beam

all: $(DRIVER) $(BEAM)

clean: 
 rm -f *.o *.beam $(DRIVER)

$(DRIVER): $(OBJECTS)
 gcc -o $@ $^ $(LDFLAGS)

$(BEAM): sniffer.erl
 erlc $^

How to get protobuf-c compiled for linking to PHP extensions under OS X

I'm writing a PHP extension at the moment that uses the C variant of Google protocol buffers - here. Unfortunately the PHP build I'm using uses 32-bit code for the commandline php, but 64-bit when running under apache. Here is the magic recipe.


First, build the standard C++/Java protocol buffers code

jons-macpro:protobuf-2.1.0 jmeredith$ cat BUILDIT 
ARCH='-arch i386 -arch x86_64' 
./configure --prefix=/Users/jmeredith/Applications/protobuf CFLAGS="$ARCH" CXXFLAGS="$ARCH" --disable-dependency-tracking && \
make  && \
make install

Then build protobuf-c which uses protobuf

jons-macpro:protobuf-c-0.11 jmeredith$ cat BUILDIT 
## Make sure protoc is in your path or this will die
ARCH="-arch i386 -arch x86_64"
./configure --prefix=/Users/jmeredith/Applications/protobuf-c \
  CXXFLAGS="-I/Users/jmeredith/Applications/protobuf/include $ARCH" \
  CFLAGS="$ARCH"  \
  LDFLAGS="-L/Users/jmeredith/Applications/protobuf/lib" \
  --disable-dependency-tracking && \
make && \
make install

Add this to the extension config.m4

  dnl # protobuf-c
  if test "$PHP_PROTOBUFC" != "no"; then
    PHP_ADD_INCLUDE($PHP_PROTOBUFC/include)
    PHP_ADD_LIBRARY_WITH_PATH(protobuf-c, $PHP_PROTOBUFC/lib, MAGNETO_SHARED_LIBADD)
  fi

And finally, to build your PHP extension

   phpize
   CFLAGS='-arch i386 -arch x86_64' ./configure --enable-yourextension --with-protobufc=/path/to/pb-c && \
   make && \
   sudo make install

Star Wars Ascii Art

I'm writing it here lest I ever forget this gem. Telnet to towel.blinkenlights.nl for the show...

Debugging .NET HTTPS apps

We're currently going through a security audit at work hardening a third party application. One of the improvements is to switch from HTTP to HTTPS which makes verifying the rest of the changes is tricky.

The way I did it was to download/installed Fiddler (http://www.fiddlertool.com/) which is an inspecting proxy. I followed the instructions to enable https decryption, and told Windows to trust the fiddler root certificate (Moved from Personal->Certificates to Trusted Root Certificate Authorities->Certificates).

Checked IE worked and it was fine, then I modified the application.exe.config file under the Docs and Settings\user\Local Settings\Application Data\Vendor\App and added

<configuration>
  <system.net>
    <defaultProxy>
      <proxy proxyaddress="http://localhost:8888" />
    </defaultProxy>
  </system.net>
</configuration>

I thought I was being very clever, but it turns out I couldn't get it to work. So I switched to using HTTPS instead.

Fiddlertool is pretty cool though :)