Parse-O-Matic: Binary Conversion Commands

Overview

The binary conversion commands deal with transformation of data between a computer ’ s representation (e.g. 10110111)and human-readable format (e.g.plain text).

A computer program that uses the ASCII character set will internally represent the letter A with the number 65 (or,more accurately,the binary value 01000001).This is not normally an issue,since a program designed to work with ASCII characters will show you the letter A on the screen.However,if the data is stored in the EBCDIC character set then the letter A will be represented by a different number.In such case you may need to convert the EBCDIC representation to the ASCII representation. Fortunately,this is quite easy to do,and a sample script to perform this conversion is available in the Pyroto,Inc.Knowledge Base (available via our web site,at www.Parse-O-Matic.com).

A more difficult problem arises when an input file contains numbers in “raw binary ”.That is to say,numbers in the file do not appear in plain text (e.g.'123').Rather,they are represented in a form that is familiar to the computer,so the number 123 might be represented as 01111011 (hexadecimal $7B).

Further complicating the issue is the fact that computers can represent numbers in various ways.123 can also be represented by 0111101100000000.This looks very similar – after all,,it is the same 8 bits as shown previously,followed by 8 zero bits – but in this case the number is being represented as a 2--byte value instead of a 1-byte value.The specific representation used by a number can be very important.If you translate a number using the wrong technique you could end up showing incorrect values,such as misinterpreting 255 as -1.

A final twist to this problem is that the various representations for numbers do not always have the same names.The word “byte ” always means “8 bits ” ,but even here we can run into trouble.A “byte ” is sometimes known as an “octet ” ,and sometimes it is assumed that one of the bits (the high bit)is not used,oris used for a purpose other than representing data (i.e.it is a “parity bit ”).The term “word ” can refer to one byte,two bytes,four,eight bytes or more,depending on the context.

For this reason,the binary conversion commands do not refer to data representations using traditional terminology such as “byte ”,“word ” and “integer ”.Rather,they use “Parse-O-Matic Conversion Codes ” to avoid confusion.For example,“I1U ” means “Integer,1 Byte,Unsigned ”.This can only refer to an 8-bit value that holds a value from 0 to 255.A complete list of the Parse-O-Matic Conversion Codes is shown below.

Parse-O-Matic Conversion Codes

For the reasons given in the Overview (above),Parse-O-Matic refers to data representations using “Conversion Codes ” rather than standard terms such as “byte ”,“word ”,“integer ”,“long integer ” and so on..

Here is a list of the conversion codes:

————— Code —————	——————————————————————————————————— Definition ———————————————————————————————————	————————————————————————————————————— Some Conventional Names (see Note) —————————————————————————————————————
I1U	Integer,1 Byte,Unsigned	Byte,Octet
I1S	Integer,1 Byte,Signed	ShortInt,Byte
I2U	Integer,2 Bytes,Unsigned	HalfWord,Word
I2S	Integer,2 Bytes,Signed	Integer,HalfWord
I4U	Integer,4 Bytes,Unsigned	DoubleWord,LongWord,Word
I4S	Integer,4 Bytes,Signed	Integer,LongInt,Cardinal
I8S	Integer,8 Bytes,Signed	DoubleWord,Int64,QuadWord
R4S	Real,4 Bytes,Signed	Real,Single
R6S	Real,6 Bytes,Signed	Real,Real48
R8S	Real,8 Bytes,Signed	Double,Real
R10S	Real,10 Bytes,Signed	Comp,Extended
R8$	Real (4 places),8 Bytes,Currency	Currency
HEX	Hexadecimal text (e.g.'F0')	Hex string
BIN	Binary text ('1111_0000')	(Used only in Parse-O-Matic)
BIC	Binary text compressed ('11110000')	Binary string
—————	———————————————————————————————————	—————————————————————————————————————

Note:

The conventional names should not be taken too seriously.A "word",for
example,might refer to 1,2,4,8 or more bytes,depending on the
context.Different computers and different computer languages may use
the same term to refer to completely different things.

These codes are not supported by all conversion commands.For example,you cannot convert from BIC format to I1U format.(In actual conversion applications,that particular transformation would almost never be required.)

You may occasionally encounter data representations that are not yet supported by Parse-O-Matic.For example,at the moment we do not translate the COMP data types used by COBOL programs.If you encounter an unsupported data type you can inquire about our schedule for adding the feature,and in the meantime you can use the CalcBinary command to transform the data into a form that is supported.

BinaryToText

Format	v1 =BinaryToText v2 v3 [v4 ]
Examples	MyByte =BinaryToText $Data [20 ] 'I1U'
Purpose	Returns the text representation of raw binary data
Parameters	v1 =Variable being set v2 =Value being converted v3 =Parse-O-Matic Conversion Code (see “Parse-O-Matic Conversion Codes ” ,in the “Overview ” section) v4 =Control setting (decimal places for real number conversions)
Defaults	v4 =2
Similar Cmds	TextToBinary
Notes	Please see the “Overview ” section for background details

All computer data is,of course,binary data at some level.The BinaryToText command is therefore a data format converter.For example,you can transform the value $FF into the string '255'or '-1',depending on the conversion code you use.$FF would produce '255'if you used the conversion code 'I1U'(Integer,1 byte, Unsigned)and '-1'if you used 'I1S'(Integer,1 Byte,Signed).

When we speak of conversion to ‘text ’ ,we are referring to the fact that all variables in Parse-O-Matic Scripting are expressed as human-readable text.To provide the ability to develop scripts quickly,there are no “data types ” such as Integer,,Real and so on,and no need to “declare ” the variables you are using..So the Parse-O-Matic Engine decides that ‘1234 ’ is an integer number if it used in a context where that matters,, such as the Calc command.Similarly,it decides that '1234.56'is a real number if it is fed into the the
CalcReal command.

The BinaryToText command provides you with the ability to translate from “typed ” data that you find in a raw binary input file into the generalized “text ” format used by Parse-O-Matic Scripting.This means that the resulting value can be fed into Parse-O-Matic commands,or send to an input file.

The sample script ScrPSTMain (which is included with the Parse-O-Matic Power Tool)provides many examples (with explanatory comments)of data format conversion using the BinaryToText command.

CalcBinary

Format	v1 =CalcBinary v2 v3 v4
Examples	ShiftedByte =CalcBinary $Data [20 ] 'SHL'1
Purpose	Returns the result of a binary operation (e.g.XOR,SHL)
Parameters	v1 =Variable being set v2 =A value upon which the operation is being performed v3 =The name of the operation v4 =The second value for the operation
Notes	Unlike Calc and CalcReal,the operation name (v3)must be in quotes

The CalcBinary command lets you manipulate data at the bit level.This can be useful for data format conversions that are not currently supported by the BinaryToText command.It is also useful for data decryption,CRC generation and so on.

In keeping with Parse-O-Matic ’s avoidance of data types (i.e.everything looks like text),you can perform the CalcBinary operations on data of any length.Thus,you could perform the ROR operation on a single byte,or hundreds of bytes.

Here is a summary of the operations supported by the CalcBinary command:

————— Name —————	————————————————— Description —————————————————	————————————————————————————————————— Notes —————————————————————————————————————
AND	Logical And	v2 and v4 must be the same length
NAND	Logical Not-And	v2 and v4 must be the same length
OR	Logical Or	v2 and v4 must be the same length
SHL	Shift Bits Left	v4 specifies number of bits to shift
SHR	Shift Bits Right	v4 specifies number of bits to shift
XOR	Exclusive Or	v2 and v4 must be the same length
ROL	Rotate Bits Left	v4 specifies number of bits to rotate
ROR	Rotate Bits Right	v4 specifies number of bits to rotate
—————	—————————————————	—————————————————————————————————————

If you want to perform a simple “NOT ” operation (i.e.flipping bits from 0 to 1 and vice-versa),use the NAND operation,pairing $FF with every byte you want flipped and $00 with every byte you do not want flipped.

The SHL and SHR commands are similar to the ROL and ROR commands,except that the latter commands “recycle ” the shifted bits to the other end of the data..In the case of SHL and SHR,on the other hand,bits shifted left or right are lost,with the “new ” bits being set to 0. The sample script ScrPSTMain (which is included with the Parse-O-Matic Power Tool)provides many examples (with explanatory comments)of the CalcBinary command in action.

TextToBinary

Format	v1 =TextToBinary v2 v3
Examples	RawIntegerSigned =TextToBinary 'I2S'-1234
Purpose	Returns the value encoded as the specified data type
Parameters	v1 =Variable being set v2 =Parse-O-Matic Conversion Code (see “Parse-O-Matic Conversion Codes ” ,in the “Overview ” section) v3 =The value being converted
Restrictions	Conversion to the BIN,BIC,HEX and R8$data types is not supported
Similar Cmds	BinaryToText
Notes	Please see the “Overview ” section for background details,and the “BinaryToText ” command for a discussion of how Parse-O-Matic manages to avoid requiring data types in scripts.

The TextToBinary command is the flip side of the BinaryToText command.You will typically use
TextToBinary command if you are creating a raw binary output file which must contain “typed ” data such as Signed Integer.

The sample script ScrPSTMain (which is included with the Parse-O-Matic Power Tool)provides many examples (with explanatory comments)of the TextToBinary command.

(This page is part of the online user manual for Parse-O-Matic. Parse-O-Matic is a programmable parsing tool that can extract, manipulate, convert or mine existing data sources and turn them into importable data. For more information on Parse-O-Matic products and conversion services, please visit www.ParseOMatic.com)