Repository navigation
Description
Accept pandas DataFrames in Python bindings
Goal
Allow the Python bindings to accept pandas DataFrames directly as input to cpplink, so users can pass their data from Python without having to write intermediate files.
The Python layer should convert the DataFrame into the native cpplink data structures, with the core C++ implementation remaining independent of pandas.
Tasks
-
Define the Python API for passing one or more pandas DataFrames to
cpplink. -
Implement conversion from pandas DataFrame columns to the corresponding C++ column types.
-
Support the main
cpplinkcolumn types, including:- String
- Integer
- Double
- Boolean
- Date/time, where supported
-
Correctly handle missing values (
NaN,pd.NA,NaT, etc.). -
Preserve column names and expose them to the
cpplinkschema. -
Avoid unnecessary copies where practical.
-
Define behaviour for unsupported pandas dtypes.
-
Provide useful Python exceptions for invalid or incompatible DataFrames.
-
Add unit tests covering:
- Different column types
- Missing values
- Empty DataFrames
- Large DataFrames
- Unsupported dtypes
- Multiple DataFrames
-
Add Python examples demonstrating the API.
-
Document the conversion rules between pandas and
cpplinktypes.
Design considerations
The binding should keep pandas-specific code at the Python/C++ interface. The cpplink core should operate on its existing native column/data structures and should not depend on pandas.
Investigate whether the conversion can use the pandas/NumPy buffer interfaces or other zero-copy mechanisms where appropriate, particularly for numerical columns, while keeping the API simple and robust.
Expected outcome
A Python user should be able to do something conceptually like:
import pandas as pd
import cpplink
df = pd.DataFrame({
"first_name": ["John", "Jane"],
"age": [35, 42],
"active": [True, False],
})
linker = cpplink.Linker(schema, df)
model, estimate = linker.estimate()
predictions_df = linker.predict(model)without first exporting the DataFrame to CSV, Parquet, or another intermediate format.
Activity