pdstools.adm.HealthCheckImport¶

Attributes¶

`HealthCheckSource`
`ConfiguredDtype`
`logger`
`MODEL_CACHE_FILENAME`
`PREDICTOR_CACHE_FILENAME`
`PREDICTION_CACHE_FILENAME`

Classes¶

`HealthCheckReadOptions`	Structured reader options for one Health Check input source.
`HealthCheckRowFilter`	Simple row predicate applied during Health Check source normalization.
`SourceNormalizationOptions`	Declarative repairs applied to one input before pdstools validation.
`SourceImportOptions`	Read and normalization options for one Health Check source.
`HealthCheckSourceMetadata`	Non-content metadata describing an imported source.
`HealthCheckImportResult`	Imported objects and constructor-ready normalized source data.

Functions¶

`normalize_health_check_data`(→ tuple[polars.LazyFrame, ...)	Apply safe, declarative repairs to one Health Check input.
`preview_health_check_columns`(→ tuple[str, Ellipsis])	Return column names read from a Health Check source.
`import_health_check_data`(→ HealthCheckImportResult)	Import model, optional predictor, and optional prediction data.
`resolve_health_check_output_dir`(→ pathlib.Path)	Resolve the `HC` output directory for processed parquet files.
`save_health_check_parquet`(→ dict[str, pathlib.Path])	Atomically persist canonical Health Check cache parquet files.

Module Contents¶

HealthCheckSource¶

ConfiguredDtype¶

logger¶

MODEL_CACHE_FILENAME = 'PR_DATA_DM_ADMMART_MDL_FACT.parquet'¶

PREDICTOR_CACHE_FILENAME = 'PR_DATA_DM_ADMMART_PRED.parquet'¶

PREDICTION_CACHE_FILENAME = 'PR_DATA_DM_SNAPSHOTS.parquet'¶

class HealthCheckReadOptions¶

Structured reader options for one Health Check input source.

Parameters:

delimiter (str, optional) – Field delimiter for CSV, TSV, or TXT input.
quote_char (str, optional) – Quote character for CSV, TSV, or TXT input. Set to None to disable quote handling.
encoding (str, default "utf8") – Text encoding forwarded to Polars.
has_header (bool, default True) – Whether the first input row contains column names.
skip_rows (int, default 0) – Number of rows to skip before reading delimited text.
infer_schema_length (int, optional) – Number of rows used for schema inference. Use zero to read all columns as strings before applying normalization overrides.
ignore_errors (bool, default False) – Continue parsing rows that contain values incompatible with the inferred delimited-text schema.
null_values (tuple[str, ...], optional) – Values interpreted as null for delimited text.
schema_overrides (Mapping[str, ConfiguredDtype]) – Reader-level column type overrides.
excel_sheet_name (str, optional) – Excel sheet name to read.
excel_sheet_id (int, optional) – One-based Excel sheet index to read.
excel_header_row (int, optional) – Zero-based Excel row containing column names.

delimiter: str | None = None¶

quote_char: str | None = '"'¶

encoding: str = 'utf8'¶

has_header: bool = True¶

skip_rows: int = 0¶

infer_schema_length: int | None = None¶

ignore_errors: bool = False¶

null_values: tuple[str, Ellipsis] | None = None¶

schema_overrides: collections.abc.Mapping[str, ConfiguredDtype]¶

excel_sheet_name: str | None = None¶

excel_sheet_id: int | None = None¶

excel_header_row: int | None = None¶

__post_init__() → None¶

Return type:: None

class HealthCheckRowFilter¶

Simple row predicate applied during Health Check source normalization.

Parameters:

column (str) – Column to evaluate, matched case-insensitively.
operator (str) – Predicate operator. Supported values are ==, !=, contains, not_contains, starts_with, not_starts_with, ends_with, not_ends_with, is_null, and is_not_null.
value (str, optional) – Predicate value. Required for all operators except is_null and is_not_null.

column: str¶

operator: Literal['==', '!=', 'contains', 'not_contains', 'starts_with', 'not_starts_with', 'ends_with', 'not_ends_with', 'is_null', 'is_not_null']¶

value: str | None = None¶

class SourceNormalizationOptions¶

Declarative repairs applied to one input before pdstools validation.

Parameters:

rename_columns (Mapping[str, str]) – Existing columns to rename, matched case-insensitively.
row_filters (tuple[HealthCheckRowFilter, ...]) – Simple row predicates to apply before column cleanup.
text_replacements (Mapping[str, Mapping[str, str]]) – Literal text replacements by column, matched case-insensitively.
fill_null_values (Mapping[str, object]) – Existing columns whose null values should be filled with a constant, matched case-insensitively.
derived_columns (Mapping[str, str]) – Columns to create or replace from a source column (target=source) or numeric subtraction (target=left-right), matched case-insensitively. String concatenation is supported as target=concat(column, "literal", other).
drop_columns (tuple[str, ...]) – Existing columns to remove, matched case-insensitively.
type_overrides (Mapping[str, ConfiguredDtype]) – Non-strict casts to apply, matched case-insensitively.
timestamp_column (str, optional) – Existing timestamp column to parse, or a new column to create when a fallback is configured.
timestamp_format (str, optional) – Explicit strptime format for a string timestamp column.
timestamp_fallback (datetime or date, optional) – Value used for missing or unparseable timestamps.
constant_columns (Mapping[str, object]) – New columns and their constant values.

rename_columns: collections.abc.Mapping[str, str]¶

row_filters: tuple[HealthCheckRowFilter, Ellipsis] = ()¶

text_replacements: collections.abc.Mapping[str, collections.abc.Mapping[str, str]]¶

fill_null_values: collections.abc.Mapping[str, object]¶

derived_columns: collections.abc.Mapping[str, str]¶

drop_columns: tuple[str, Ellipsis] = ()¶

type_overrides: collections.abc.Mapping[str, ConfiguredDtype]¶

timestamp_column: str | None = None¶

timestamp_format: str | None = None¶

timestamp_fallback: datetime.datetime | datetime.date | None = None¶

constant_columns: collections.abc.Mapping[str, object]¶

class SourceImportOptions¶

Read and normalization options for one Health Check source.

read: HealthCheckReadOptions¶

normalize: SourceNormalizationOptions¶

class HealthCheckSourceMetadata¶

Non-content metadata describing an imported source.

name: str¶

extension: str¶

class HealthCheckImportResult¶

Imported objects and constructor-ready normalized source data.

datamart: pdstools.adm.ADMDatamart.ADMDatamart¶

prediction: pdstools.prediction.Prediction.Prediction | None¶

model_data: polars.LazyFrame¶

predictor_data: polars.LazyFrame | None¶

prediction_data: polars.LazyFrame | None¶

sources: collections.abc.Mapping[str, HealthCheckSourceMetadata]¶

warnings: tuple[str, Ellipsis]¶

normalize_health_check_data(df: polars.LazyFrame, options: SourceNormalizationOptions | None = None) → tuple[polars.LazyFrame, tuple[str, Ellipsis]]¶

Apply safe, declarative repairs to one Health Check input.

Parameters:

df (pl.LazyFrame) – Raw input data.
options (SourceNormalizationOptions, optional) – Repairs to apply. Column names are matched case-insensitively.

Returns:

Normalized data and data-safe descriptions of applied repairs.

Return type:

tuple[pl.LazyFrame, tuple[str, …]]

Raises:

ValueError – If configured columns do not exist, settings conflict, or a configured data type is unsupported.

preview_health_check_columns(source: HealthCheckSource, options: HealthCheckReadOptions | None = None) → tuple[str, Ellipsis]¶

Return column names read from a Health Check source.

Parameters:

source (str, path-like, or BytesIO) – Input source to inspect.
options (HealthCheckReadOptions, optional) – Reader options to apply while inspecting the source.

Returns:

Column names found in the source.

Return type:

tuple[str, …]

import_health_check_data(model_source: HealthCheckSource, predictor_source: HealthCheckSource | None = None, prediction_source: HealthCheckSource | None = None, *, model_options: SourceImportOptions | None = None, predictor_options: SourceImportOptions | None = None, prediction_options: SourceImportOptions | None = None, extract_pyname_keys: bool = True, predictor_categorization: collections.abc.Mapping[str, str | list[str]] | None = None, predictor_categorization_uses_regex: bool = False) → HealthCheckImportResult¶

Import model, optional predictor, and optional prediction data.

The returned source frames are normalized but remain constructor-ready. This allows canonical parquet files to be loaded later by the normal ADMDatamart and Prediction validation paths.

Parameters:

model_source (HealthCheckSource) – Required model snapshot input.
predictor_source (HealthCheckSource, optional) – Predictor-binning snapshot input.
prediction_source (HealthCheckSource, optional) – Prediction snapshot input.
model_options (SourceImportOptions, optional) – Independent read and normalization options for each source.
predictor_options (SourceImportOptions, optional) – Independent read and normalization options for each source.
prediction_options (SourceImportOptions, optional) – Independent read and normalization options for each source.
extract_pyname_keys (bool, default True) – Whether to extract context keys embedded in the model Name field.
predictor_categorization (Mapping[str, str or list[str]], optional) – Category names mapped to PredictorName substring patterns.
predictor_categorization_uses_regex (bool, default False) – Interpret predictor categorization patterns as regular expressions.

Returns:

Validated analysis objects, constructor-ready source data, metadata, and data-safe descriptions of applied repairs.

Return type:

HealthCheckImportResult

resolve_health_check_output_dir(model_source: HealthCheckSource | None = None, predictor_source: HealthCheckSource | None = None, prediction_source: HealthCheckSource | None = None, *, output_parent: str | os.PathLike[str] | None = None) → pathlib.Path¶

Resolve the HC output directory for processed parquet files.

An explicit output parent takes precedence. Otherwise, the first filesystem source is selected in model, predictor, prediction order. In-memory-only imports fall back to the current working directory.

Parameters:

model_source (HealthCheckSource, optional) – Input sources used to select a filesystem parent.
predictor_source (HealthCheckSource, optional) – Input sources used to select a filesystem parent.
prediction_source (HealthCheckSource, optional) – Input sources used to select a filesystem parent.
output_parent (str or path-like, optional) – Explicit directory under which the HC directory is created.

Returns:

Resolved output directory ending in HC.

Return type:

pathlib.Path

save_health_check_parquet(result: HealthCheckImportResult, output_parent: str | os.PathLike[str]) → dict[str, pathlib.Path]¶

Atomically persist canonical Health Check cache parquet files.

All requested frames are written to temporary files before any canonical file is replaced. Optional canonical files from an earlier import are removed after a successful write when the corresponding source is absent.

Parameters:

result (HealthCheckImportResult) – Imported source frames and validated analysis objects.
output_parent (str or path-like) – Parent directory under which an HC directory is created.

Returns:

Canonical paths keyed by model, predictor, and prediction.

Return type:

dict[str, pathlib.Path]