mirror of
https://github.com/magnus919/agent-skills.git
synced 2026-09-15 05:26:28 +03:00
Magnus HedemarkandGitHub
dde5fa1020
Merge authorized after exact-head validation and paired evaluation checks passed. The follow-up Droid review run on head df31b25 stalled in the model step and ended with an automation error; its actionable findings from the prior review were fixed and independently verified.
2.6 KiB
2.6 KiB
Source notes and evidence map
Accessed 2026-08-17. The KDnuggets article is orientation; primary docs govern current behavior.
- KDnuggets: 5 Python Libraries: tool map and broad use cases; secondary source, so verify APIs.
- Wes McKinney, Data Cleaning: pandas mapping, replacement, missing data, strings, categoricals, and reshaping.
- pyjanitor docs: chainable pandas-style cleaning.
- Pandera schemas: schema, nullability, coercion, strictness, uniqueness, and backends.
- Great Expectations validation: expectations, batches, checkpoints, and evidence.
- ftfy docs: focused Unicode/mojibake repair.
- ydata-profiling: profiles and comparisons.
- Cerberus: nested dictionary validation and coercion.
- OpenRefine reconciliation: semi-automated matching with required human review.
- Frictionless validation: structured tabular errors and custom checks.
- dbt data tests: reusable failing-row assertions and generic tests.
- Deequ: Spark-scale metrics and constraints.
- Wickham, Tidy Data: variables in columns, observations in rows, values in cells.
- pandas missing data and pandas merging: nullable dtypes, sentinel behavior,
skipna, merge cardinality, and row multiplication. - Unicode UAX #15: normative normalization forms and the risk of erasing distinctions with compatibility normalization.
- dedupe documentation: labeled, blocked, reviewable probabilistic record linkage.
- csvkit and Miller: scriptable inspection and streaming tabular transformation.
Conclusion: profile to discover, transform explicitly, validate named contracts, and preserve evidence. The reusable artifact is a plan + decision log + contract + before/after report, not a magic cleaner.