Mar 25

Creating a Binary File Using a Ruby DSL

So I used ruby to convert a simple hex string into a binary file in Converting hex to Binary in 4 Languages. Today I was trying to create a mixed ascii/binary file at work and created a little Domain Specific Language that has good possibilities.

NOTE: There is an expanded version of this script here: A Better Binary File Generator DSL in Ruby

The Example

#!/usr/bin/ruby
class Hex2Bin
    attr_accessor :data;
 
    def initialize
        @data = ""
        @invert = false
    end
 
    def invert
        @invert = !@invert
        yield
        @invert = !@invert
    end
 
    def binary(binary_str)
        if(@invert) then
            binary_str = binary_str.unpack("C*").collect{|x| ~x}.pack("C*")
        end
        @data << binary_str
    end
 
    def hex(hex_str)
        binary( hex_str.gsub(/\s/,'').to_a.pack("H*") )
    end
 
    def ascii(ascii_string)
        binary( ascii_string )
    end
 
    def byte(value)
        binary([value].pack("C1"))
    end
 
    def get_binding()
        return binding
    end
end
context = Hex2Bin.new();
eval(STDIN.read, context.get_binding)
STDOUT.write(context.data);

This lets me write the following file:

#frame1
ascii "####2008-001-00:10:01.123;"
hex   "FF AB 00 00 00 00 00 00"
invert {
    hex   "FF AB 01 00 00 00 00 00"
}
hex   "FF AB 02 00"
 
#frame2
ascii "####2008-001-23:10:01.123;"
hex   "00 00 00 00"
hex   "FF AB 03 00 00 00 00 00"
hex   "FF AB 04 00 00 00 00 00"
 
#frame3
ascii "####2008-001-00:10:01.123;"
hex   "00 00 00 00"
hex   "00 00 00 00 00 00 00 00"
hex   "00 00 00 00 00 00 00 FF"
 
#frame4
ascii "####2008-001-00:20:01.123;"
hex   "AB 05 00 00 00 00 00"
hex   "00 00 00 00 00 00 00 00"
hex   "00 00 00 00 00"

How Does It Work

For details on writing a DSL in ruby please refer to the previous article: Ruby Shell as Domain Specific Language

This line:

    binary_str = binary_str.unpack("C*").collect{|x| ~x}.pack("C*")

Is worth commenting on. This is the sort of thing ruby proponents call intuitive and I will merely call convenient. Basically, unpack turns the binary data string into an array of integers. Collect takes this array and applies a block of code to each element and returns a new array containing the modified values returned by that block. It is worth noting here that ruby doesn’t require return and will simply return the last value in a block of code so ~x is equivalent to return ~x.

Anyway, once I have the new array of inverted byte values pack puts it back into a binary string, now inverted.

The other thing worth mentioning is this:

    def invert
        @invert = !@invert
        yield
        @invert = !@invert
    end

So what is going on here? The yield directive is how you can write code that uses blocks in the same way as collect from the previous example. Basically yield runs the block and returns when it is done.

In this case I put the object into invert mode and then execute any code in the block passed to the invert method. This allows the following syntax in my little language:

invert {
    hex   "FF AB 01 00 00 00 00 00"
}

Why Would I do This?

This isn’t really all that special so why do I like this solution so much? Well first of all it is really easy to add to (like if I want to put TCPIP headers in my file). If I wanted to do this with a custom input format I’d have to add new code to parse a new directive, blah blah blah.

Also I get a lot for free. I get # comments, variables, string concatenation, etc if that is where I want to go. I also really like the ability to use ruby closures to tag a subsection of data as inverted. In general all the reasons that a DSL is better than a custom language from scratch.

The Author

Michael Smit is a software engineer in Seattle, Washington who works for amazon

Comments are off for this post

Comments are closed.