I am working with an input csv file that has a varying number of columns which I am trying to convert into a list. The issue I am facing is that my test is parsing the csv file and creating a list with additional elements around the csv columns. Ideally, I would like the list to only contain the csv elements, but right now it also includes empty quoted elements. I need some assistance in figuring out what options I might be missing when using the csv reader.
Here is an example of the output:
$ python cond.py
opening conditions file conditions.lst
parser 0 input line:
"string1:", "string1b,string1c,"
output list elements:
['string1:']
['', '']
['']
['string1b,string1c,']
[]
parser 1 input line:
"stringa:", "stringb,stringc,"
output list elements:
['stringa:']
['', '']
['']
['stringb,stringc,']
[]
parser 2 input line:
"string3:", "string3next=abc", "string3b","string3c:", "string3d"
output list elements:
['string3:']
['', '']
['']
['string3next=abc']
['', '']
['']
['string3b']
['', '']
['string3c:']
['', '']
['']
['string3d']
[]
Input file:
$ cat conditions.lst
"string1:", "string1b,string1c,"
"stringa:", "stringb,stringc,"
"string3:", "string3next=abc", "string3b","string3c:", "string3d"
Python cond.py file:
$ cat cond.py
from __future__ import print_function
#from csv import reader
import re
import sys
import csv
# variables
conditionsFile = "conditions.lst"
parserConditions = []
numOfParsers = 0
print("opening conditions file ", conditionsFile)
with open(conditionsFile, "r") as cf:
for line in cf:
print("parser ", numOfParsers, " input line:")
print(line.strip())
r = csv.reader(line, quotechar='"', delimiter=',', quoting=csv.QUOTE_ALL, skipinitialspace=True)
print("output list elements:")
for cline in r:
print(cline)
numOfParsers = numOfParsers + 1
print("")
print("total number of parsers: ", numOfParsers)
Update: With the help of @Jean-FrançoisFabre, I haven't identified the root cause yet, but I have come up with a workaround - placing the csv elements into a list and then removing any blank elements.
for cline in r:
conditions.extend(cline)
conditions = filter(None, conditions)
print(conditions)