Automating delimiter selection when working with Csv cmdlets

Recently I walked passed a collegue with an error on his Powershell console which peaked my interest. He quickly noted that he had chosen the wrong delimiter for the Csv he imported which resulted in errors in the code, I then jokingly said ”Why don’t you just use the ’Get-CsvDelimiter’ cmdlet?” and we had a quick laugh.
30 minutes later the ”Get-CsvDelimiter” function was born, but first,

Let’s dive into how Import-Csv, ConvertFrom-Csv & delimiters really work

When used correctly Import-Csv & ConvertFrom-Csv creates an array object with each row as a PSCustomObject with named content based on headers.

Import-Csv has two constructors, one with Path as a required and the other with Path & Delimiter as required.
If you omit the delimiter parameter the Path constructor will be used, now this constructor has a required parameter (according to the documentation) that is auto filled. Instead of using Delimiter we can use ”UseCulture” which takes the current culture delimiter as input,

”To find the list separator for a culture, use the following command: (Get-Culture).TextInfo.ListSeparator.” –docs.microsoft

If we interpret the documentation this is a required parameter, but it is not really required, as we are able to pass only the -Path parameter and delimiter will autofill

Assume we’re using the following CSV as input ($File)

the command ”Import-Csv $File” will output a valid object. Now in my opinion it should fail. As my culture specifies semicolon (;) as the default listseparator, while forcing ”-UseCulture” can’t produce a correct csv object. This basically means that UseCulture is not a required parameter and the default unless specified is always a comma (,)

The exact same goes for ConvertFrom-Csv.

Presenting Get-CsvDelimiter

The below function will search and find the most probable delimiter used in a Csv file.

Let’s see how it works.

$File is a csv file with the below data.

If we run Import-Csv $File we’ll get a incorrect table object, every row will have a single property containing the row data.

If we instead specify a delimiter as shown below, we’ll calculate the most probable delimiter and use that to produce a correct CSV table object without having to inspect the csv culture or assume anything.

And to prove it works, the below code outputs only the Name column in the Csv

If we have a stringobject and need to convert it that is also possible.

With this method we can use virtually any delimiter we want, assuming that the Csv is formatted correctly. Again we’re using the same csv input but we change the delimiter to a ”greater than” (>) symbol and see how the function performs.

Now when we run the function standalone we’ll get a Greater than symbol as the return

And finally, when running the previous code we get the same output as when we were using a semicolon as the delimiter.

 

Get-CsvDelimiter

Disclaimer: All information on this blog is offered "as is" with no warranty. It is strongly recommended that you verify all information and validate all scripts in isolated test environments before using them in production environments.