Singapore's Duplicate Image Problem: The Numbers Driving a Digital Clean-Up

Singapore's public and private sector databases collectively hold an estimated tens of millions of duplicate image files, a data-hygiene problem that digital archivists and IT procurement teams say is costing organisations here real money every quarter. While the figure is hard to pin down precisely across sectors, cloud storage audits conducted by several mid-sized firms along Changi Business Park Avenue 1 in 2025 found that duplicate or near-duplicate images accounted for between 28 and 34 percent of total stored visual assets — files that exist in two or more identical or near-identical copies, consuming paid-for capacity without adding informational value.

The issue is pressing because Singapore's push to become a regional AI and data hub has sharply increased the volume of images being ingested, processed, and archived. The Infocomm Media Development Authority's Digital Connectivity Blueprint, published in 2023, set targets for expanding national data infrastructure through 2030. That infrastructure is only as efficient as the data it holds. Redundant images inflate training datasets, skew machine-learning models, and drive up the cost of cloud compute — problems that compound as storage scales.

What the Storage Bills Actually Show

Amazon Web Services S3 standard storage in the Asia-Pacific Singapore region is priced at approximately USD 0.025 per gigabyte per month as of mid-2026. A single high-resolution image from a modern smartphone or professional camera can run between 5 and 25 megabytes. Scale that to a government agency managing millions of citizen-submitted documents — think MyInfo uploads processed through the Government Technology Agency of Singapore, or property inspection photographs filed through HDB's e-Service portal — and the arithmetic becomes uncomfortable quickly. One gigabyte of duplicates, replicated across three availability zones for redundancy, costs roughly USD 0.075 a month. Across tens of thousands of gigabytes, that is a six-figure annual drag before egress fees are factored in.

The National Library Board, which manages digitised collections across its branches including the flagship library at Victoria Street and the National Archives of Singapore facility at Canning Rise, flagged duplicate image management as a data-quality priority in its digital preservation framework. The NLB holds millions of digitised photographs, newspaper pages, and heritage images. Even a 10 percent duplication rate across a multi-terabyte archive translates into measurable wasted expenditure on storage infrastructure that the library funds from its public budget.

Private-sector exposure is sharper. PropertyGuru, which operates one of Singapore's largest real-estate image repositories, processes hundreds of thousands of new listing photographs weekly. Industry analysts who track Southeast Asian proptech note that platforms of this scale typically see duplication rates of 15 to 22 percent when sellers re-list the same unit across multiple listing cycles without purging prior uploads. Each duplicated image consumes bandwidth during indexing, slows visual-search algorithms, and marginally degrades recommendation accuracy — a technical drag that eventually surfaces in user experience metrics.

Detection Tools and What Comes Next

Deduplication is not a new problem, but the tooling has matured fast. Perceptual hashing algorithms — which generate a compact fingerprint for an image based on visual content rather than file metadata — can identify near-duplicate photographs even when they have been resized, recoloured, or lightly cropped. Open-source libraries such as ImageHash and commercial solutions integrated into platforms like Microsoft Azure AI Vision or Google Cloud Vision API can process thousands of images per minute at costs measured in fractions of a cent per image.

For Singapore organisations benchmarking a clean-up exercise, the practical calculation runs like this: a one-time deduplication audit of a 50-terabyte image repository, using cloud-based perceptual hashing at roughly USD 0.001 per image, typically costs less than the six-month storage bill for the duplicates it removes. The Smart Nation and Digital Government Office has encouraged agencies to run periodic data-quality audits under the Digital Government Blueprint refresh, though mandatory deduplication standards for image assets have not yet been formalised.

For businesses and agencies assessing their own exposure, the first step is an inventory. Firms without a centralised digital asset management system — still common among small and medium enterprises clustered in areas like Buona Vista's one-north tech precinct — should treat a baseline duplication audit as a line item in the next IT budget cycle, not a discretionary task. The storage savings alone, in most cases, cover the audit cost within a quarter.