CIT 594 Module 7 Programming Assignment

CSV Slicer

In this assignment you will read files in a format known as “comma separated values” (CSV), interpret the formatting and output the content in the structure represented by the file.

Q1703105484

Learning Objectives

In completing this assignment, you will:

• Implement a method to extract content from an input CSV
• Read and understand a formal specification for the CSV format
• Use a “state machine”

Background and Getting Started

Applications have long used delimited text files to store and transmit tables. The simplicity of the format means these files are human readable and editable as well as relatively easy to support in code. The breadth of support from nearly any general purpose application or tabular data loading library continues to make these files particularly popular for publishing data despite their limitations and inefficiencies. You will see more of this in later assignments where you will be required to support reading publicly available datasets.

One of the more common choices for delimiters are commas to separate fields within a row of the table, and line breaks to separate rows of the table. The common name for these comma based file formats are “comma separated values” (CSV).

To support values in the fields that include delimiter characters (e.g., if a comma is part of the data, not just a field separator) requires some added complexity. There is no single, universally-accepted specification for CSV files, so we will focus on one specific format that is widely accepted for this assignment, RFC 4180.

To get started, carefully read sections 1 and 2 of RFC 4180: RFC 4180 - Common Format and MIME Type for Comma-Separated Values (CSV) Files

That document uses a precise syntax to express an exact formal specification. If you are unsure of how to interpret the ANBF grammar you can reference RFC 2234.

CSV Format For This Assignment

You will write a method to process CSV files character-by-character. The exact format is a relaxation of RFC 4180.

Adjustments and clarifications of the descriptive rules:

1. Formatting characters describe structure; they are not part of the content. For example,  the comma that separates two fields should not be included in either field. For example (easy3.csv):

"example of using "" in a field",1

Should result in a single row that is the equivalent an array constructed with the Java expres- sion:

new String[]{ "example of using \\" in a field", "1" }

1. The same applies for the escape character in an escaped field. Two double quotes in the middle of an escaped field count as a single double quote character in the content.
2. You may ignore rule number 4 (for this assignment). Instead, follow these rules:
   1. No special treatment should be applied to the first row. You should not treat a header any differently than a record.
   2. You will not need to check if all rows have the same number of fields.

1. 1. Commas at the end of a line signify empty fields. For example, "a,b,c," results in a row with four fields: ["a", "b", "c", ""]1
   2. An empty line terminated by a line break is a valid row if it’s outside an escaped field. Inside an escaped field, it is just part of the content of that field.
2. Clarification of rule 2: There is no additional record if the file ends at the start of a line. That includes but is not limited to example in rule 1 where the file ends with CRLF EOF

Adjustments to the grammar:

CRLF = [CR] LF

TEXTDATA =     x00-09  /   x0B-0C /    x0E-21  /   x23-2B  /   x2D-7F

The first rule change makes the carriage return (CR) optional everywhere CRLF is used. This is a relaxation seen in many places to adjust for the fact that many systems and applications choose to omit the carriage return and only use a single line feed character for line breaks.

The second rule change expands TEXTDATA2 to all characters that are not comma, double quote, carriage return, and line feed.

Note: these are relaxations, therefore all documents that are valid for the strict interpretation of RFC 4180 are valid for this format.

For the purposes of the assignment there are five classes of characters for you to consider:

1The line in rule 4 in RFC about "The last field in the record must not be followed by  a comma" is referring       to their rule that each line should contain the same number of fields.  A comma at the end of a line would insert       an additional, empty field. The statement indicates that an extra comma that would increase the number of fields  beyond the expected limit should not be dropped or ignored.

2We will not test with characters above %x7F (127), but you are welcome to include %xFF                                                                                                                   7FFFFFFF

(Integer.MAX_VALUE) in TEXTD