data-import
Technical notes on web development, DevOps, and AI integration.
2 articles
- 04:28backend
Seed Migration and Status Mapping for Government Facility Data Import
Creating new categories through idempotent seed migrations, then importing facility data with status-based publish mapping for existing and planned facilities.
TL;DR: A missing Sarana Budaya category blocked importing eight cultural facilities, so the team added it via an idempotent seed migration using ON DUPLICATE KEY UPDATE. Existing facilities import as published, planned ones as drafts with admin notes. The importer ships with dry-run mode, a development-only guard, and slug-based deduplication for safe, repeatable runs.
#go#mysql#data-import - 04:23backend
Corrective Cultural Heritage Data: Pool Matching Without a Unique Key
The corrective import of PPKD cultural heritage data failed because the SK number wasn't unique. Candidate pools and overlap coefficient solve 1:1 matching.
TL;DR: Importing PPKD heritage data broke when SK numbers turned out to be shared across multiple objects, so direct key matching corrupted good records. The fix pairs each row with the best name match from an SK's candidate pool using the overlap coefficient, plus a used guard preventing double assignment. Column ownership rules keep verified coordinates and media untouched.
#go#mysql#data-import