[R] How to pre-process fwf or csv files to remove unexpected characters in R?

Lucas Ferreira Mation lucasmation at gmail.com
Sun Nov 6 14:36:46 CET 2016

I have some large .txt files about ~100GB containing a dataset in fixed
width file. This contains some errors:
- character characters in column that are supposed to be numeric,
- invalid characters
- rows with too many characters, possibly due to invalid characters or some
missing end of line character (so two rows in the original data become one
row in the .txt file).

The errors are not very frequent, but stop me from importing with readr

Is there some package, or workflow, in R to pre-process the files,
separating the valid from the not-valid rows into different files? This can
be done by ETL point-click tools, such as Pentaho PDI. Is there some
equivalent code in R to do this?

I googled it and could not find a solution. I also asked this in
StackOverflow and got no answer (here

Lucas Mation
IPEA - Brasil

	[[alternative HTML version deleted]]

More information about the R-help mailing list