The error “TypeError: Object of type ‘int64’ is not JSON serializable” frequently arises when working with libraries like Pandas and NumPy in Python. This occurs because JSON doesn’t inherently support the NumPy `int64` data type. This guide presents solutions to resolve this issue.
Table of Contents
- Converting ‘int64’ to Standard Python Types
- Handling Potentially Large Numbers
- Converting ‘int64’ to Strings
- Best Practices for Data Serialization
Converting ‘int64’ to Standard Python Types
The most common and often preferred solution is to convert `int64` values to standard Python integers (`int`) or floating-point numbers (`float`). This preserves the numerical data while ensuring JSON compatibility.
Pandas Example:
import pandas as pd
import json
data = {'col1': [1, 2, 3, 4, 5, 2**63]} # Includes a large number
df = pd.DataFrame(data)
# Convert to int (will cause OverflowError for large numbers)
try:
df['col1'] = df['col1'].astype(int)
json_data = json.dumps(df.to_dict('records'))
print(f"Integer conversion: {json_data}")
except OverflowError:
print("OverflowError occurred during integer conversion.")
# Convert to float (handles large numbers)
df['col1'] = df['col1'].astype(float)
json_data = json.dumps(df.to_dict('records'))
print(f"Float conversion: {json_data}")
NumPy Example:
import numpy as np
import json
arr = np.array([1, 2, 3, 4, 5, 2**63], dtype=np.int64)
# Convert to int (will cause OverflowError for large numbers)
try:
arr = arr.astype(int)
json_data = json.dumps(arr.tolist())
print(f"Integer conversion: {json_data}")
except OverflowError:
print("OverflowError occurred during integer conversion.")
# Convert to float (handles large numbers)
arr = arr.astype(float)
json_data = json.dumps(arr.tolist())
print(f"Float conversion: {json_data}")
Handling Potentially Large Numbers
If your `int64` values might exceed the maximum value representable by a standard Python integer, converting to `float` is essential to prevent `OverflowError`. Floats can handle a much wider range of numerical values.
Converting ‘int64’ to Strings
Converting to strings is a straightforward alternative if you don’t need to perform mathematical operations on the data after serialization. This is a robust approach that avoids potential overflow issues.
import pandas as pd
import json
data = {'col1': [1, 2, 3, 4, 5, 2**63]}
df = pd.DataFrame(data)
df['col1'] = df['col1'].astype(str)
json_data = json.dumps(df.to_dict('records'))
print(json_data)
Best Practices for Data Serialization
For cleaner code and easier maintenance, consistently handle data types before serialization. Consider using a dedicated serialization library if your data structures are complex. Always carefully select the appropriate conversion method (int, float, or string) based on your specific application’s requirements.