Efficiently Finding Files by Extension in Python

Finding files with specific extensions is a common task when working with files in Python. This article explores several efficient and versatile methods to accomplish this, catering to different needs and coding styles. We’ll cover various Python modules and techniques, allowing you to choose the optimal approach for your specific use case.

Table of Contents

1. Using the glob Module
2. Leveraging the os Module
3. Employing pathlib for Single Directory Searches
4. Recursive Searching with os.walk
5. Recursive Searching with pathlib.rglob

1. Using the glob Module

The glob module offers a concise way to locate files matching a defined pattern within a single directory. Its glob() function returns a list of paths that satisfy the pattern.

import glob

def find_files_with_extension_glob(directory, extension):
  """Finds files with a given extension using glob.glob()."""
  return glob.glob(f"{directory}/*{extension}")

# Example:
files = find_files_with_extension_glob("./my_directory", ".txt")
print(files)

2. Leveraging the os Module

The os module provides more granular control. os.listdir() lists all entries in a directory, allowing for custom filtering.

import os

def find_files_with_extension_os(directory, extension):
  """Finds files with a given extension using os.listdir()."""
  files = []
  for filename in os.listdir(directory):
    if filename.endswith(extension):
      files.append(os.path.join(directory, filename))
  return files

# Example:
files = find_files_with_extension_os("./my_directory", ".txt")
print(files)

3. Employing pathlib for Single Directory Searches

The pathlib module provides an object-oriented approach. Its glob() method offers a cleaner syntax than glob.glob().

from pathlib import Path

def find_files_with_extension_pathlib(directory, extension):
  """Finds files with a given extension using pathlib.glob()."""
  return list(Path(directory).glob(f"*{extension}"))

# Example:
files = find_files_with_extension_pathlib("./my_directory", ".txt")
print(files)

4. Recursive Searching with os.walk

For searching subdirectories, os.walk() traverses the directory tree, allowing you to check each file for the desired extension.

import os

def find_files_recursive_os(directory, extension):
    """Recursively finds files with a given extension using os.walk()."""
    files = []
    for root, _, filenames in os.walk(directory):
        for filename in filenames:
            if filename.endswith(extension):
                files.append(os.path.join(root, filename))
    return files

# Example:
files = find_files_recursive_os("./my_directory", ".txt")
print(files)

5. Recursive Searching with pathlib.rglob

pathlib‘s rglob() method offers the most elegant solution for recursive searches.

from pathlib import Path

def find_files_recursive_pathlib(directory, extension):
  """Recursively finds files with a given extension using pathlib.rglob()."""
  return list(Path(directory).rglob(f"*{extension}"))

# Example:
files = find_files_recursive_pathlib("./my_directory", ".txt")
print(files)

Remember to create a “my_directory” folder with some .txt files for the examples to work correctly. Select the method that best aligns with your coding style and project demands. For simple, single-directory searches, glob or pathlib.glob() are excellent choices. For recursive searches, pathlib.rglob() provides the most concise and readable solution.