Generative AI for Data Management

Multi-modal datasets such as image collections are full of rich information, but that richness is often locked away. The raw items may be visually expressive, yet the textual descriptions that would let a system understand them are usually sparse, incomplete, or entirely missing. As soon as a user asks a complex natural-language query, something that depends on nuanced semantics rather than a short keyword label, standard retrieval pipelines begin to fail.

The core difficulty is deeper than missing metadata. In traditional nearest-neighbor search, both the data and the query live in the same metric space, so comparison is direct. Here, however, the query is expressed in language while the tuples are multimodal objects, meaning the two sides start in fundamentally different spaces. Prior work tries to bridge that gap through jointly trained text-image representations, but those approaches often struggle once the query becomes compositionally rich or semantically demanding.

This paper takes a different path. Instead of forcing the query into a single brittle vector representation, it introduces a generative-AI-powered Monte Carlo method that uses foundation models to create synthetic samples capturing the intent and complexity of the natural-language request. Those generated samples are then represented in the same metric space as the multi-modal data, making retrieval possible in a way that better respects the structure of the original query.

Building on that idea, the paper presents NEEDLE, a database system for image retrieval that is driven by synthetic data generation rather than contrastive learning or metadata search. The result is not just a research idea, but an open-source, deployment-ready system designed for practical adoption by researchers and developers. Across benchmark datasets, NEEDLE significantly outperforms state-of-the-art text-to-image retrieval methods, while remaining flexible enough to benefit from future progress in foundation models and embedding technologies.

Citation: Mahdi Erfanian, Mohsen Dehghankar, and Abolfazl Asudeh. 2024. Needle: A Generative-AI Powered Monte Carlo Method for Answering Complex Natural Language Queries on Multi-modal Data. arXiv:2412.00639.

Paper

Machine learning systems are increasingly scrutinized for being unfair to minorities and other marginalized groups, and one recurring reason is simple but serious: those groups are often under-represented in the training data. In the ideal case, the dataset could be repaired by collecting more real examples from external sources. In practice, that option is often unavailable, too expensive, or too slow.

That gap creates a natural question for modern generative AI: if real data cannot be collected, can high-quality synthetic data be used to repair the dataset instead? This paper explores that question through Chameleon, a system that uses foundation models to generate synthetic multimodal tuples for fairness-aware data augmentation.

The appeal of this idea is immediate, because the dataset itself is rarely the final objective. It is usually an input to something else, such as training an ML model. If a small amount of carefully generated synthetic data can reduce under-representation and improve downstream fairness, then augmentation may be a practical substitute when real-world repair is impossible.

But making that work requires more than just generating extra examples. The system must identify the minimal set of synthetic tuples needed to address representation bias, preserve semantic integrity so the generated tuples fit the context and distribution of the original dataset, maintain visual and semantic quality that looks realistic to human evaluators, and remain cost-effective despite the monetary cost of querying foundation models. To address these challenges, the work introduces several technical ideas, including rejection tests and multi-armed bandit methods.

Comprehensive experiments show that augmentation with synthetic tuples generated by advanced foundation models can come remarkably close to the effect of adding real tuples when the goal is to reduce downstream unfairness. In that sense, Chameleon demonstrates that generative AI can play a concrete role in responsible data management, not merely by analyzing biased datasets, but by actively helping repair them.

Citation: Mahdi Erfanian, H. Jagadish, and Abolfazl Asudeh. 2024. Chameleon: Foundation Models for Fairness-aware Multi-modal Data Augmentation to Enhance Coverage of Minorities. In Proceedings of the VLDB Endowment, 17(11): 3470-3483.

Paper Slides Video

Retrieving images with natural language sounds straightforward until the query becomes genuinely descriptive. Existing systems often depend on contrastive-learning embeddings such as CLIP, which work reasonably well for short or simple prompts but begin to degrade when users ask for something nuanced, compositional, or visually specific. That gap is exactly where a practical retrieval system begins to matter.

NeedleDB is presented as a deployment-ready database system built to answer those harder queries over image collections. Its central idea is to use generative AI not just as an auxiliary tool, but as part of the retrieval mechanism itself. Rather than forcing a difficult text query into a single embedding and hoping it aligns well with the image space, the system synthesizes guide images that visually represent the query, effectively turning a fragile text-to-image search problem into a more tractable image-to-image retrieval task.

The system then strengthens retrieval quality by aggregating nearest-neighbor results across multiple vision embedders through a weighted rank-fusion strategy grounded in a Monte Carlo estimator with provable error bounds. That algorithmic core is paired with a complete systems stack: a command-line interface through needlectl, a browser-based web interface, and a modular microservice architecture backed by PostgreSQL and Milvus.

The result is a system that is not only accurate, but operational. On challenging benchmarks, NeedleDB improves Mean Average Precision by up to 93% over the strongest baseline while maintaining sub-second query latency. In demonstration settings, users can interact with the system through realistic scenarios that highlight retrieval quality, ingestion workflows, and the configurability of the end-to-end pipeline.

Citation: Mahdi Erfanian and Abolfazl Asudeh. 2026. NeedleDB: A Generative-AI Based System for Accurate and Efficient Image Retrieval using Complex Natural Language Queries. CoRR abs/2603.27464.

Paper Interactive Demo GitHub