Skip to content

Commit

Permalink
CSV writer helper added.
Browse files Browse the repository at this point in the history
  • Loading branch information
cinar committed Dec 21, 2023
1 parent b2c4837 commit b8b737e
Show file tree
Hide file tree
Showing 6 changed files with 623 additions and 7 deletions.
42 changes: 36 additions & 6 deletions helper/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ The information provided on this project is strictly for informational purposes
- [Constants](<#constants>)
- [func Abs\[T Number\]\(c \<\-chan T\) \<\-chan T](<#Abs>)
- [func Add\[T Number\]\(ac, bc \<\-chan T\) \<\-chan T](<#Add>)
- [func AppendOrWriteToCsvFile\[T any\]\(fileName string, hasHeader bool, rows \<\-chan \*T\) error](<#AppendOrWriteToCsvFile>)
- [func Apply\[T Number\]\(c \<\-chan T, f func\(T\) T\) \<\-chan T](<#Apply>)
- [func Buffered\[T any\]\(c \<\-chan T, size int\) \<\-chan T](<#Buffered>)
- [func ChanToSlice\[T any\]\(c \<\-chan T\) \[\]T](<#ChanToSlice>)
Expand Down Expand Up @@ -74,8 +75,10 @@ The information provided on this project is strictly for informational purposes
- [type BstNode](<#BstNode>)
- [type Csv](<#Csv>)
- [func NewCsv\[T any\]\(hasHeader bool\) \(\*Csv\[T\], error\)](<#NewCsv>)
- [func \(c \*Csv\[T\]\) AppendToFile\(fileName string, rows \<\-chan \*T\) error](<#Csv[T].AppendToFile>)
- [func \(c \*Csv\[T\]\) ReadFromFile\(fileName string\) \(\<\-chan \*T, error\)](<#Csv[T].ReadFromFile>)
- [func \(c \*Csv\[T\]\) ReadFromReader\(reader io.Reader\) \<\-chan \*T](<#Csv[T].ReadFromReader>)
- [func \(c \*Csv\[T\]\) WriteToFile\(fileName string, rows \<\-chan \*T\) error](<#Csv[T].WriteToFile>)
- [type Float](<#Float>)
- [type Integer](<#Integer>)
- [type Number](<#Number>)
Expand Down Expand Up @@ -148,6 +151,15 @@ actual := helper.ChanToSlice(helper.Add(ac, bc))
fmt.Println(actual) // [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
```

<a name="AppendOrWriteToCsvFile"></a>
## func [AppendOrWriteToCsvFile](<https://github.com/cinar/indicator/blob/v2/helper/csv.go#L287>)

```go
func AppendOrWriteToCsvFile[T any](fileName string, hasHeader bool, rows <-chan *T) error
```

AppendOrWriteToCsvFile writes the provided rows of data to the specified file, appending to the existing file if it exists or creating a new one if it doesn't. In append mode, the function assumes that the existing file's column order matches the field order of the given row struct to ensure consistent data structure.

<a name="Apply"></a>
## func [Apply](<https://github.com/cinar/indicator/blob/v2/helper/apply.go#L17>)

Expand Down Expand Up @@ -233,7 +245,7 @@ fmt.Println(helper.ChanToSlice(actual)) // [400, 150, 60, -60, -87.5, -50, 200,
```

<a name="CheckEquals"></a>
## func [CheckEquals](<https://github.com/cinar/indicator/blob/v2/helper/check.go#L13>)
## func [CheckEquals](<https://github.com/cinar/indicator/blob/v2/helper/check.go#L14>)

```go
func CheckEquals[T comparable](inputs ...<-chan T) error
Expand Down Expand Up @@ -541,7 +553,7 @@ fmt.Println(helper.ChanToSlice(squared)) // [4, 9, 25, 100]
```

<a name="ReadFromCsvFile"></a>
## func [ReadFromCsvFile](<https://github.com/cinar/indicator/blob/v2/helper/csv.go#L191>)
## func [ReadFromCsvFile](<https://github.com/cinar/indicator/blob/v2/helper/csv.go#L274>)

```go
func ReadFromCsvFile[T any](fileName string, hasHeader bool) (<-chan *T, error)
Expand Down Expand Up @@ -804,7 +816,7 @@ type BstNode[T Number] struct {
```

<a name="Csv"></a>
## type [Csv](<https://github.com/cinar/indicator/blob/v2/helper/csv.go#L39-L46>)
## type [Csv](<https://github.com/cinar/indicator/blob/v2/helper/csv.go#L40-L47>)

Csv represents the configuration for CSV reader and writer.

Expand All @@ -815,16 +827,25 @@ type Csv[T any] struct {
```

<a name="NewCsv"></a>
### func [NewCsv](<https://github.com/cinar/indicator/blob/v2/helper/csv.go#L50>)
### func [NewCsv](<https://github.com/cinar/indicator/blob/v2/helper/csv.go#L51>)

```go
func NewCsv[T any](hasHeader bool) (*Csv[T], error)
```

NewCsv function initializes a new CSV instance. The parameter hasHeader indicates whether the CSV contains a header row.

<a name="Csv[T].AppendToFile"></a>
### func \(\*Csv\[T\]\) [AppendToFile](<https://github.com/cinar/indicator/blob/v2/helper/csv.go#L170>)

```go
func (c *Csv[T]) AppendToFile(fileName string, rows <-chan *T) error
```

AppendToFile appends the provided rows of data to the end of the specified file, creating the file if it doesn't exist. In append mode, the function assumes that the existing file's column order matches the field order of the given row struct to ensure consistent data structure.

<a name="Csv[T].ReadFromFile"></a>
### func \(\*Csv\[T\]\) [ReadFromFile](<https://github.com/cinar/indicator/blob/v2/helper/csv.go#L145>)
### func \(\*Csv\[T\]\) [ReadFromFile](<https://github.com/cinar/indicator/blob/v2/helper/csv.go#L146>)

```go
func (c *Csv[T]) ReadFromFile(fileName string) (<-chan *T, error)
Expand All @@ -833,14 +854,23 @@ func (c *Csv[T]) ReadFromFile(fileName string) (<-chan *T, error)
ReadFromFile parses the CSV data from the provided file name, maps the data to corresponding struct fields, and delivers the resulting snapshots through the channel.

<a name="Csv[T].ReadFromReader"></a>
### func \(\*Csv\[T\]\) [ReadFromReader](<https://github.com/cinar/indicator/blob/v2/helper/csv.go#L90>)
### func \(\*Csv\[T\]\) [ReadFromReader](<https://github.com/cinar/indicator/blob/v2/helper/csv.go#L91>)

```go
func (c *Csv[T]) ReadFromReader(reader io.Reader) <-chan *T
```

ReadFromReader parses the CSV data from the provided reader, maps the data to corresponding struct fields, and delivers the resulting it through the channel.

<a name="Csv[T].WriteToFile"></a>
### func \(\*Csv\[T\]\) [WriteToFile](<https://github.com/cinar/indicator/blob/v2/helper/csv.go#L176>)

```go
func (c *Csv[T]) WriteToFile(fileName string, rows <-chan *T) error
```

WriteToFile creates a new file with the given name and writes the provided rows of data to it, overwriting any existing content.

<a name="Float"></a>
## type [Float](<https://github.com/cinar/indicator/blob/v2/helper/helper.go#L27-L29>)

Expand Down
3 changes: 2 additions & 1 deletion helper/check.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ package helper
import (
"errors"
"fmt"
"reflect"
)

// CheckEquals determines whether the two channels are equal.
Expand All @@ -30,7 +31,7 @@ func CheckEquals[T comparable](inputs ...<-chan T) error {
return nil
}

if actual != expected {
if !reflect.DeepEqual(actual, expected) {
return fmt.Errorf("index %d pair %d actual %v expected %v", i, j/2, actual, expected)
}
}
Expand Down
105 changes: 105 additions & 0 deletions helper/csv.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"encoding/csv"
"errors"
"io"
"io/fs"
"log"
"os"
"path/filepath"
Expand Down Expand Up @@ -162,6 +163,20 @@ func (c *Csv[T]) ReadFromFile(fileName string) (<-chan *T, error) {
return rows, nil
}

// AppendToFile appends the provided rows of data to the end of the specified file, creating
// the file if it doesn't exist. In append mode, the function assumes that the existing
// file's column order matches the field order of the given row struct to ensure consistent
// data structure.
func (c *Csv[T]) AppendToFile(fileName string, rows <-chan *T) error {
return c.writeToFile(fileName, os.O_APPEND, rows)
}

// WriteToFile creates a new file with the given name and writes the provided rows
// of data to it, overwriting any existing content.
func (c *Csv[T]) WriteToFile(fileName string, rows <-chan *T) error {
return c.writeToFile(fileName, os.O_CREATE, rows)
}

// updateColumnIndexes aligns column indices to match the order of column headers.
func (c *Csv[T]) updateColumnIndexes(csvReader *csv.Reader) error {
headers, err := csvReader.Read()
Expand All @@ -186,6 +201,74 @@ func (c *Csv[T]) updateColumnIndexes(csvReader *csv.Reader) error {
return nil
}

// writeToFile writes the provided rows of data to a file with the given name, using the
// specified flag mode for precise control over file opening and writing behavior.
func (c *Csv[T]) writeToFile(fileName string, flag int, rows <-chan *T) error {
file, err := os.OpenFile(filepath.Clean(fileName), flag|os.O_WRONLY, 0644)
if err != nil {
return err
}

defer file.Close()

writeHeader := c.hasHeader && (flag == os.O_CREATE)

err = c.writeToWriter(file, writeHeader, rows)
if err != nil {
return err
}

return file.Sync()
}

// writeToWriter writes the provided rows of data to the specified writer, with the option
// to include or exclude headers for flexibility in data presentation.
func (c *Csv[T]) writeToWriter(writer io.Writer, writeHeader bool, rows <-chan *T) error {
csvWriter := csv.NewWriter(writer)

if writeHeader {
err := c.writeHeaderToCsvWriter(csvWriter)
if err != nil {
return err
}
}

record := make([]string, len(c.columns))

for row := range rows {
rowValue := reflect.ValueOf(row).Elem()

for i, column := range c.columns {
stringValue, err := getReflectValue(rowValue.Field(column.FieldIndex), column.Format)
if err != nil {
return err
}

record[i] = stringValue
}

err := csvWriter.Write(record)
if err != nil {
return err
}
}

csvWriter.Flush()

return csvWriter.Error()
}

// writeHeaderToCsvWriter writes the column headers for the CSV data to the specified CSV writer.
func (c *Csv[T]) writeHeaderToCsvWriter(csvWriter *csv.Writer) error {
header := make([]string, len(c.columns))

for i, column := range c.columns {
header[i] = column.Header
}

return csvWriter.Write(header)
}

// ReadFromCsvFile creates a CSV instance, parses CSV data from the provided filename,
// maps the data to corresponding struct fields, and delivers it through the channel.
func ReadFromCsvFile[T any](fileName string, hasHeader bool) (<-chan *T, error) {
Expand All @@ -196,3 +279,25 @@ func ReadFromCsvFile[T any](fileName string, hasHeader bool) (<-chan *T, error)

return csv.ReadFromFile(fileName)
}

// AppendOrWriteToCsvFile writes the provided rows of data to the specified file, appending to
// the existing file if it exists or creating a new one if it doesn't. In append mode, the
// function assumes that the existing file's column order matches the field order of the
// given row struct to ensure consistent data structure.
func AppendOrWriteToCsvFile[T any](fileName string, hasHeader bool, rows <-chan *T) error {
csv, err := NewCsv[T](hasHeader)
if err != nil {
return err
}

stat, err := os.Stat(filepath.Clean(fileName))
if err != nil {
if !errors.Is(err, fs.ErrNotExist) {
return err
}
} else if stat.Size() > 0 {
return csv.AppendToFile(fileName, rows)
}

return csv.WriteToFile(fileName, rows)
}
Loading

0 comments on commit b8b737e

Please sign in to comment.