omarskalli / fixy

Library for generating fixed width documents

fixy

Library for generating fixed width flat file documents.

Installation

Add this line to your application's Gemfile:

gem 'fixy'

And then execute:

bundle

Or install it yourself as:

gem install fixy

Then proceed to creating your records, and documents as described in the paragraphs below.

Overview

A fixed-width document (Fixy::Document) is composed of multiple single-line records (Fixy::Record).

Record definition

Every record is defined through a specific format, which defines the following aspects:

• Record length (how many characters in the line)
• Line ending (optional, defaults to "\n")
• Required formatters (e.g. Alphanumeric, Rate, Amount)
• Field declaration:
  • Field human readable name
  • Field size (how many characters for the field)
  • Field range (start/end column for the field)
  • Field format (e.g. Alphanumeric, Rate, Amount)
• Field definition

Below is an example of a record for defining a person's first and last name:

class PersonRecord < Fixy::Record

	# Include formatters
	
  include Fixy::Formatter::Alphanumeric

	# Define record length
	
  set_record_length 20

  # Fields Declaration:
  # -----------------------------------------------------------
  #       name          size      Range             Format        
  # ------------------------------------------------------------

  field :first_name,     10,     '1-10' ,      :alphanumeric
  field :last_name ,     10,     '11-20',      :alphanumeric

	# Any required data for the record can be 
	# provided through the initializer
			
	def initialize(first_name, last_name)
	  @first_name = first_name
	  @last_name  = last_name
	end
	
	# Fields Definition:
	# 1) Using a Proc 
	
  field_value :first_name, -> { @first_name }

	# 2) Using a method definition. 
	#    This is most interesting when complex logic is involved.
  
  def last_name
    @last_name
  end
end

You can also specify the field definition and field value together by passing a block to field.

field(:first_name, 10, '1-10', :alphanumeric) { @first_name }

If a record requires a specific line ending, you can specify it as part of the Record definition.

  set_line_ending Fixy::Record::LINE_ENDING_CRLF

Given a record definition, you can generate a single line (e.g. for testing purposes):

PersonRecord.new('Sarah', 'Kerrigan').generate
	
# This will output the following 20 characters long record
#
#  "Sarah     Kerrigan  \n"
#

Most of the time however, you will not have to call generate directly, as the document will take care of that part.

Parsing existing records

There is limited support for parsing existing records and documents. Because the formatters are currently uni-directionals, formatted values are extracted.

PersonRecord.parse "Sarah     Kerrigan  "

# This will generate the following hash
# {
#     :fields => [
#         { :name => :first_name, :value => "Sarah     "},
#         { :name => :last_name,  :value => "Kerrigan  "}
#     ],
#     :record => "Sarah     Kerrigan  \n"
# }

Document definition

A document is composed of a multitude of records (instances of a Fixy::Record). Because some document specification require earlier records to contain a count of upcoming records, both appending and prepending records is supported during a document definition. Below is an example of a document, based on the record defined in the previous section.

class PeopleDocument < Fixy::Document
  def build
    append_record  PersonRecord.new('Sarah', 'Kerrigan')
    append_record  PersonRecord.new('Jim', 'Raynor')
    prepend_record PersonRecord.new('Arcturus', 'Mengsk')
  end
end

Document definition using existing records

Occasionally, it is useful to generate a document using existing records. This is particularly handy when generating debug documents (detailed in the next section).

class ParsedPeopleDocument < Fixy::Document
  def build
    parse_record IdentityRecord, 'Arcturus  Mengsk    '
    parse_record IdentityRecord, 'Sarah     Kerrigan  '
    parse_record IdentityRecord, 'Jim       Raynor    '
  end
end

Generating a document

With records and documents defined, generating documents is a breeze:

** Generating to string **

PeopleDocument.new.generate

The output would be: "Arcturus Mengsk \nSarah Kerrigan \nJim Raynor "

** Generating to file **

PeopleDocument.new.generate_to_file("output.txt")

** Generating HTML Debug version **

This is most useful when getting an error such as: Unexpected character at line 20, column 95. The HTML output makes it really easy to make sense out of any fixed width document, and quickly identify issues.

PeopleDocument.new.generate_to_file("output.html", true)

Creating custom formatters

Currently, there aren't many formatters included in this release, and you will most likely have to write your own. To create a new formatter of type type (e.g. amount), you simply need a method called format_<type>(input, length). The argument input is the value being formatted, and length is the number of characters to fill. It is important to make sure length characters are returned by the formatter!

An example for formatter definition:

module Fixy
  module Formatter
    module Numeric
      def format_numeric(input, length)
        input = input.to_s
        raise ArgumentError, "Invalid Input (only digits are accepted)" unless input =~ /^\d+$/
        raise ArgumentError, "Not enough length (input: #{input}, length: #{length})" if input.length > length
        input.rjust(length, '0')
      end
    end
  end
end

Contributing

1. Fork it
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create new Pull Request

fixy's Issues

Custom line endings aren't working yet

I tried using the new cr/lf line ending as documented:

set_line_ending Fixy::Record::LINE_ENDING_CRLF

And.... no change to the line endings. Looking into the code a bit, the line ending seems to be held in an instance variable and accessed from a class method, which could explain the problem... and I'm not sure what the intention was, so I'm reporting it rather than trying to fix it. Thanks!

Line endings suggestion

Should be exist a feature that let you choose what's line ending append at the end of the row because some formats are different like American express that does not accept \r\n (carriage return)
in order to support different validations, this specific one is to

Acquisition Participant (AP)
Sponsored Merchant File
Technical File Specification

from American express apr 2017

Record parsing and generate

Recent record.rb changes no longer use the "decorator" method, instead parse and generate have

decorator = debug ? Fixy::Decorator::Debug : Fixy::Decorator::Default

Please revert to calling the decorator method to allow other decorators to be used by overriding it.

numeric and alphanumeric formatters?

Hello,

nice work here -- but your web page mentions numeric formatter, and that is not
included in the git source... just wondering if it an oversight, or should I write it ?

surely they are needed by almost anyone making fixed records.

Alphanumeric formatter does not work with accents

Hi,

given the use of bytesize, accents confuse the algorithm.

Below an implementation that does not break positioting:

module Alphanumeric
  def format_alphanumeric(input, byte_width)
    input = input.to_s
    String.new(input).ljust(byte_width, " ")
  end
end

I will also paste all my formatters below in case someone may need them.

module Fixy
  module Formatter

    # Justifies an alpha to the left
    module Alphanumeric
      def format_alphanumeric(input, byte_width)
        input = input.to_s
        String.new(input).ljust(byte_width, " ")
      end
    end

    # Formats a number 93 to "00000093"
    module Numeric
      def format_numeric(input, byte_width)
        String.new(input.to_s).rjust(byte_width, "0")
      end
    end

    # Formats datetime to YYYYMMDDHHMMSS
    module DateTime
      def format_datetime(input, byte_width)
        input.strftime("%Y%m%d%H%M%S")
      end
    end

    # Formats date to YYYYMMDD
    module Date
      def format_date(input, byte_width)
        ::Date.strptime(input, "%d/%m/%Y").strftime("%Y%m%d")
      end
    end

    # Formats date to YYMMDD
    module ShortDate
      def format_short_date(input, byte_width)
        ::Date.strptime(input, "%d/%m/%Y").strftime("%y%m%d")
      end
    end

    # Formats floating point "9.34" to "000000934"
    module Amount
      def format_amount(input, byte_width)
         input = "%0.2f" % input
         input.gsub(".", "").rjust(byte_width, "0")
      end
    end

  end
end