Efficient Line-by-Line Text File Reading in C#

Efficiently reading text files line by line is a crucial skill for any C# developer. Whether you’re processing logs, parsing configuration files, or analyzing datasets, the method you choose can significantly impact performance and resource usage. This article explores three common approaches, highlighting their strengths and weaknesses to help you select the optimal technique for your specific needs.

Table of Contents

• Using File.ReadLines()
• Using File.ReadAllLines()
• Using StreamReader.ReadLine()
• Choosing the Right Method

Using File.ReadLines()

The File.ReadLines() method is generally the most efficient and convenient option, especially for large files. It returns an IEnumerable, meaning it processes the file line by line without loading the entire content into memory at once. This lazy loading prevents potential OutOfMemoryException errors when dealing with massive datasets.

using System;
using System.IO;
using System.Linq;

public class ReadLinesExample
{
    public static void Main(string[] args)
    {
        string filePath = "myFile.txt"; 

        try
        {
            foreach (string line in File.ReadLines(filePath))
            {
                Console.WriteLine(line); 
                //Process each line here
            }
        }
        catch (FileNotFoundException)
        {
            Console.WriteLine($"File not found: {filePath}");
        }
        catch (Exception ex)
        {
            Console.WriteLine($"An error occurred: {ex.Message}");
        }
    }
}

Using File.ReadAllLines()

File.ReadAllLines() offers a simpler syntax, reading all lines into a string array. However, this approach loads the entire file into memory, making it inefficient and potentially problematic for large files. It’s best suited for smaller files where memory consumption isn’t a major concern.

using System;
using System.IO;

public class ReadAllLinesExample
{
    public static void Main(string[] args)
    {
        string filePath = "myFile.txt";

        try
        {
            string[] lines = File.ReadAllLines(filePath);
            foreach (string line in lines)
            {
                Console.WriteLine(line); 
                //Process each line here
            }
        }
        catch (FileNotFoundException)
        {
            Console.WriteLine($"File not found: {filePath}");
        }
        catch (Exception ex)
        {
            Console.WriteLine($"An error occurred: {ex.Message}");
        }
    }
}

Using StreamReader.ReadLine()

StreamReader.ReadLine() provides the most granular control. You read one line at a time using a loop, allowing for custom processing of each line before reading the next. This method requires explicit resource management using a using statement to ensure proper disposal of the StreamReader.

using System;
using System.IO;

public class StreamReaderExample
{
    public static void Main(string[] args)
    {
        string filePath = "myFile.txt";

        try
        {
            using (StreamReader reader = new StreamReader(filePath))
            {
                string line;
                while ((line = reader.ReadLine()) != null)
                {
                    Console.WriteLine(line); 
                    //Process each line here
                }
            }
        }
        catch (FileNotFoundException)
        {
            Console.WriteLine($"File not found: {filePath}");
        }
        catch (Exception ex)
        {
            Console.WriteLine($"An error occurred: {ex.Message}");
        }
    }
}

Choosing the Right Method

For most scenarios, especially when dealing with large files, File.ReadLines() is the recommended approach due to its efficiency and ease of use. File.ReadAllLines() is suitable only for smaller files. StreamReader.ReadLine() provides more control but demands more careful resource management.

Remember to always include robust error handling to gracefully manage potential exceptions like file not found errors.