Needle Banner

We have found the Needle in haystack! 🪡🔍

Needle is an open-source image retrieval database with high accuracy that can easily handle complex queries in natural language. It is Fast, Efficient, and Precise, outperforming state-of-the-art methods. Born from high-end research, Needle is designed to be accessible to everyone while delivering top-notch performance. Whether you’re a researcher, developer, or an enthusiast, Needle opens up innovative ways to explore your image datasets. ✨

See Needle in Action

*Watch as Needle transforms natural language queries into precise image retrieval results in real time.*

Comparison to State-of-the-Art Methods

Curious how Needle measures up against other cutting-edge approaches? Here, you’ll soon find performance plots that compare Needle with OPEN-AI CLIP image retrieval method for LVIS, Caltech256 and BDD100k.

Mean Average Precision Charts

Mean Average Precision Across Datasets (All Queries)

Mean Average Precision Across Datasets (Hard Queries)

User Study Preferences

Get Started Today!

Ready to revolutionize your image retrieval process? 🚀
Install and test Needle now to experience the future of multimodal search!

Tip: For detailed installation instructions, check out the Getting Started section.

Cite us

For a deep dive into Needle’s theoretical guarantees and performance insights, please refer to our research paper.

If you find Needle beneficial for your work, we kindly ask that you cite our work to help support continued innovation.

@article{erfanian2024needle,
  title={Needle: A Generative-AI Powered Monte Carlo Method for Answering Complex Natural Language Queries on Multi-modal Data},
  author={Erfanian, Mahdi and Dehghankar, Mohsen and Asudeh, Abolfazl},
  journal={arXiv preprint arXiv:2412.00639},
  year={2024}
}

Getting Started

Prerequisites

Before installing Needle, ensure that you have the following prerequisites installed:

  • Docker: Needle relies on Docker to containerize its services.
    Install Docker

  • Docker Compose: This tool is required to orchestrate the multi-container setup.
    Install Docker Compose

Warning: Make sure your user account is added to the Docker group so you can run Docker commands (e.g., docker ps) without needing root privileges.

Note: Currently, Needle is supported only on Linux.

Installation

To install Needle, run the following one-liner in your terminal:

curl -fsSL https://raw.githubusercontent.com/UIC-InDeXLab/Needle/main/scripts/install.sh -o install.sh && bash install.sh && rm install.sh

During the installation process, you will be prompted to choose the database mode. The available options are:

  • Fast: Offers quick responses and indexing with lower accuracy.
  • Balanced: Provides a compromise between speed and accuracy.
  • Accurate: Prioritizes high accuracy, which may come at the cost of slower performance.

Warning: Once the database mode is set, it cannot be changed without uninstalling and reinstalling Needle, which will result in data loss.

Note:“ Needle automatically checks for GPU accessibility and will use the GPU if available to optimize performance.

After the installation completes, please close and reopen your terminal session (or source your shell configuration file) to ensure that all environment variables are correctly set.

Starting the Needle Service

Once installed, start the Needle service by running:

needlectl service start

This command will download the required model weights and initiate all the necessary services.

To verify that everything is running as expected, you can check the service status with:

needlectl service status

And confirm the installed version using:

needlectl --version

About needlectl

The needlectl command-line tool is the primary interface for interacting with Needle. It will be discussed in detail in the subsequent sections, where you’ll learn how to leverage its full capabilities.

needlectl Command Line Tool

needlectl is the primary command-line interface that connects to the Needle backend service and performs a variety of tasks to manage and interact with Needle. It provides both general options and component-specific commands.

General Options

The following options are available with needlectl:

  • --api-url: Specifies the backend URL to connect to.
    Default: 127.0.0.1:8000

  • --output: Determines the format of the output.
    Supported formats: human-readable, JSON, YAML.
    Default: human-readable.

  • --version: Prints the version information for both the Needle backend and the needlectl tool.

  • --install-completion: Installs auto-completion for your current shell.

Note: --api-url and --output are accessible globally and in all commands, for example, in order to get the outputs of a query in JSON format you can use following command:

needlectl --output json query run "a wolf howling"

Components Overview

needlectl is organized into several components, each dedicated to specific functions within the Needle system. For more detailed information, please refer to the corresponding pages for each component.

Service

Manages Needle’s core service operations, including:

  • Starting the service
  • Restarting the service
  • Stopping the service
  • Viewing logs
  • Upgrading the service

Directory

Handles image directory management tasks, such as:

  • Adding directories
  • Removing directories
  • Indexing images

Query

Executes natural language queries against the image database, providing flexible and powerful search capabilities.

Generator

Manages image generation operations, allowing you to adjust parameters and generate images as needed.

Service Component

The service component of needlectl manages the core operations of Needle. It connects to the Needle backend service and uses Docker Compose to start, stop, restart, check status, view logs, and configure the service environment.

Note: Most commands rely on the --api-url option (default: 127.0.0.1:8000) to connect to the backend API. You can set this globally if needed.

Commands Overview

start

Starts the Needle services by launching the required Docker containers and waiting for the backend API to become available.

  • What it does:

    • Uses the Docker Compose Manager to start containers.
    • Calls the backend API and waits until it is accessible.

  • Usage Example:

    needlectl service start

  • Output: Displays messages indicating that the services are starting, and if the backend API becomes available, it prints “Services started.” If not, an error is shown.

stop

Stops the Needle services by shutting down the Docker containers.

  • What it does:

    • Uses the Docker Compose Manager to stop all running containers.

  • Usage Example:

     needlectl service stop

  • Output: A confirmation message “Services stopped.” is printed once the containers are shut down.

restart

Restarts the Needle services by first stopping and then starting the Docker containers again. It also waits for the backend API to be available after the restart.

  • Usage Example: 
    needlectl service restart

stop

Stops the Needle services by shutting down the Docker containers.

  • What it does:

    • Uses the Docker Compose Manager to restart containers.
    • Waits for the backend API to respond after restarting.

  • Output: Displays messages for restarting and confirms when the services have been restarted successfully.

  • Usage Example:

    needlectl service stop

status

Retrieves and displays the current status of Needle services using the backend API.

  • Usage Example: 
    needlectl service status

log

Displays the logs from the Needle backend service.

  • Usage Example: 
    needlectl service log

config

Manages the configuration for the Needle service environment.

  • Usage Example: 
    needlectl service config

Warning: Changes made via this command can affect service operations. Ensure you understand the impact before making modifications.

Directory Component

The directory component of needlectl is responsible for managing image directories in the Needle system. It allows you to add directories for image indexing, remove directories, list existing directories, modify directory configurations, and view detailed information about a directory.

Commands Overview

add

This command adds a new image directory to Needle’s indexing system.

  • Options:

    • path (string): The path to the directory to be added.
    • --show-progress (boolean, optional): If enabled, displays a progress bar showing the indexing progress.

  • What It Does:

    1. Connects to the backend API and adds the directory.
    2. If successful, returns a directory ID.
    3. If the --show-progress flag is set, a progress bar (powered by the tqdm library) is displayed. This progress bar periodically updates based on the indexing ratio reported by the backend and shows an estimated time for completion.

  • Usage Examples:

    # Add a directory without displaying indexing progress
needlectl directory add /path/to/my/images

# Add a directory and display the indexing progress
needlectl directory add /path/to/my/images --show-progress

remove

This command removes an image directory from Needle.

  • Options:

    • path (string): The path to the directory to be removed.

  • Usage Examples:

    needlectl directory remove /path/to/my/images

list

This command lists all the directories currently registered in Needle.

  • Usage Examples: 
    needlectl directory list

modify

This command allows you to modify the configuration for your added directories, you can enable/disable them for searching. Needle will not search in disabled directories.

  • Usage Examples: 
    needlectl directory modify

describe

This command provides detailed information about a specific directory by its directory ID.

  • Options:

    • did (integer): The unique identifier of the directory.

  • Usage Examples:

    needlectl directory describe 1

config

This command manages the environmental configuration for the directory component.

  • Usage Examples: 
    needlectl directory config

Query Component

The Query Component of needlectl allows you to interact with Needle’s image retrieval capabilities. It lets you execute natural language queries, view query logs, and configure query-related settings.

Commands Overview

run

This command executes a search query against the Needle image retrieval database.

  • Options:

    • prompt (string): The search prompt in natural language.
    • -n or --n (integer, optional): Specifies the number of images to retrieve.
    • -m or --m (integer, optional): Specifies the number of images per engine.
    • -k or --k (integer, optional): Specifies the number of engines to use.
    • --image-size (integer, optional): Sets the desired image size for image generation.
    • --include-base-images (boolean, optional): Indicates whether to include base images in the results preview.
    • --use-fallback (boolean, optional): Indicates whether to use fallback mode if the primary engines set fails.

  • What It Does:

    1. Retrieves the enabled and activated generator configurations using the Generator Config Manager.
    2. If no generator configuration is found, it displays an error message suggesting you use needlectl generator config to configure a generator.
    3. Executes the search query with the provided parameters via the backend API.
    4. Displays the search results in the specified output format (human-readable, JSON, or YAML).

  • Usage Examples:

    # Run a basic query with a prompt
needlectl query run "red cars"

# Run a query with additional options
needlectl query run "red cars" --n 5 --m 2 --k 3 --image-size 512 --include-base-images true --use-fallback false

log

This command retrieves and displays the logs of search queries executed by Needle.

  • Usage Examples: 
    needlectl query log

config

This command manages the environmental configuration for the query component.

  • Usage Examples: 
    needlectl query config

Generator Component

The Generator Component of needlectl is used to manage image generators for Needle. It allows you to list all available generators from the backend and configure generator settings that are used during image queries.

Commands Overview

list

This command lists all available image generators as retrieved from the backend.

  • What It Does:

    1. Connects to the backend API.
    2. Retrieves the list of available generators.
    3. Displays the result in the specified output format (human-readable, JSON, or YAML).

  • Usage Example:

    needlectl generator list

config

This command manages the configuration for image generators used by Needle.

  • Usage Example: 
    needlectl generator config

Uninstallation

If you decide to remove Needle from your system, you can do so quickly by running the following one-liner command in your terminal:

curl -fsSL https://raw.githubusercontent.com/UIC-InDeXLab/Needle/main/scripts/uninstall.sh | bash

Note: This uninstallation command will remove Needle’s services, data and configurations. However, it will not remove any cached Docker images. If you need to free up additional disk space, please consider manually removing those images.