Singapore's Duplicate Image Problem: The Key Decisions Ahead for Government Digital Archives

Singapore's public sector digitisation push has hit a friction point that administrators have quietly wrestled with for years: the proliferating problem of duplicate images embedded across government databases, land title records, and heritage archives. The immediate trigger is a directive issued in the first half of 2026 by the Smart Nation Group, which sits under the Ministry of Digital Development and Information, pressing agencies to standardise their data hygiene protocols before the next phase of the national AI strategy rolls out. Duplicate image files — the same scanned document or photograph stored multiple times under different filenames or record IDs — are one of the clearest sources of downstream error in automated systems.

The stakes are higher than they might appear. When AI-assisted services query databases that contain redundant image records, they surface inconsistent results. For residents applying for Housing and Development Board grants or checking Central Provident Fund statements, a duplicated identity photograph or a doubled property image can trigger a manual review flag, extending processing times. More critically, as agencies move toward automated approvals, a single duplicated file can split a record into two apparent entities — a data integrity failure that is expensive and time-consuming to untangle after the fact.

Where the Problem Lives

The issue is concentrated in three broad domains. The National Archives of Singapore, based at the National Library Building on Victoria Street, holds millions of digitised photographs, maps, and official documents accumulated since independence. The Singapore Land Authority manages cadastral survey images and title deed scans across the island's roughly 720 square kilometres. And the Immigration and Checkpoints Authority processes hundreds of thousands of biometric images annually across Woodlands Checkpoint, Tuas Checkpoint, and Changi Airport Terminal 4, among other entry points.

Each of these agencies developed its imaging infrastructure on different timelines and with different vendor systems, which is precisely why duplicates accumulated. A photograph scanned in 2009 under one naming convention could sit alongside a rescan from 2017 with no automated cross-reference linking them. The Government Technology Agency of Singapore, known as GovTech, has been piloting perceptual hashing — a technique that generates a short fingerprint from an image's visual content rather than its filename — to detect near-identical files at scale. GovTech has not published specifics of that pilot's scope, but the technology is well-established in commercial content moderation and is now being evaluated for public sector adaptation.

Singapore allocated S$1 billion toward AI and data infrastructure development in Budget 2024, a figure cited in official parliamentary materials, and that envelope funds the current round of database consolidation work. Agencies have until the end of the third quarter of 2026 to submit conformance reports to the Smart Nation Group on their image deduplication readiness.

The Decisions That Cannot Wait

Three choices now sit on the desks of agency heads and their IT leads. The first is whether to pursue automated deletion or archival quarantine for confirmed duplicates. Deletion is cleaner and reduces storage costs, but errors in deduplication algorithms can eliminate unique records — a risk that archivists at institutions like the National Museum of Singapore on Stamford Road have been vocal about in interagency discussions, though no public statement has been issued. Quarantine preserves the file but creates a secondary management burden.

The second decision concerns a common metadata standard. Without a shared schema dictating how image origin, capture date, and source agency are recorded, deduplication remains a one-time fix rather than a systemic solution. The Infocomm Media Development Authority has been working on cross-agency data standards since at least 2022, but harmonising image metadata specifically requires agencies to retrofit existing records — a labour-intensive exercise.

The third and most consequential question is audit accountability. When a deduplication process incorrectly merges two distinct records — say, two different residents whose identity photographs are visually similar — who carries responsibility for the downstream error? That question touches the Personal Data Protection Act framework administered by the Personal Data Protection Commission, and legal clarity on machine-driven data decisions is still catching up with operational reality.

Agencies that miss the Q3 deadline face a potential block on integration with the Singpass MyInfo platform's expanded data-sharing features, slated for rollout in late 2026. For most Singaporeans, that is the pressure point that will turn a technical governance problem into a visible public services one.