Skip to content
Robotics-STAR-LabPublic

About

An aerial object navigation system that locates open-set objects in outdoor environments from natural-language instructions.

Resources

Stars

28 stars

Watchers

1 watching

Forks

Repository files navigation

AirHunt: Bridging VLM Semantics and Continuous Planning for Efficient Aerial Object Navigation

IEEE Robotics and Automation Letters (RA-L), 2026

Xuecheng Chen1, Zongzhuo Liu2, Jianfa Ma3, Bang Du4, Tiantian Zhang1, Xueqian Wang1,†, Boyu Zhou2,5,†

1 Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
2 Department of Mechanical and Energy Engineering, Southern University of Science and Technology, Shenzhen, China
3 Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China
4 Department of Physics, Southern University of Science and Technology, Shenzhen, China
5 Differential Robotics

† Corresponding Authors

RA-L Paper Bilibili

Introduction

AirHunt is an aerial object navigation system that locates open-set objects in outdoor environments from natural-language instructions. It addresses the frequency mismatch between VLM inference and real-time planning, limited 3D scene understanding, and the challenge of balancing semantic guidance with motion efficiency. A dual-pathway asynchronous architecture connects active visual reasoning with continuous planning through a persistent 3D value map, enabling efficient search and continuous flight with zero-shot generalization.

AirHunt system overview and pipeline comparison

Installation

1. Prerequisites

The current runtime has been tested on Ubuntu 24.04, Python 3.11, ROS Noetic through RoboStack, and PyTorch 2.7.1 with CUDA 12.8. An NVIDIA GPU with a compatible driver and Vulkan support is required to render the simulator and run the perception models.

Install micromamba, then clone the repository and create the environment:

git clone https://github.com/Robotics-STAR-Lab/AirHunt.git
cd AirHunt
micromamba create -p "$PWD/.runtime" -f environment.yml

2. Runtime Configuration

Create the local configuration file:

test -f .env || cp .env.example .env

Edit .env and use absolute paths:

Variable Description
AIRHUNT_RUNTIME Environment created above, e.g. /absolute/path/to/AirHunt/.runtime
MICROMAMBA micromamba on PATH, or its absolute executable path
AIRHUNT_ENV_EXECUTABLE Full path to CabinLake-Linux-Shipping inside the extracted scene
AIRHUNT_CHECKPOINTS Directory containing the two model checkpoints listed below
HF_HOME Hugging Face cache directory for CLIP and BERT
DASHSCOPE_API_KEY API key for the Qwen vision-language model
DASHSCOPE_BASE_URL OpenAI-compatible endpoint associated with the API key
DASHSCOPE_MODEL Vision-language model name; the example uses qwen3-vl-plus

Keep .env local. The launcher reads it automatically.

3. Python Dependencies

Install the CUDA 12.8 PyTorch build and the remaining dependencies:

./scripts/env.sh python -m pip install torch==2.7.1 torchvision==0.22.1 \
  --index-url https://download.pytorch.org/whl/cu128
./scripts/env.sh python -m pip install -r requirements.txt
./scripts/env.sh python -m pip install --no-deps airsim==1.8.1 msgpack-rpc-python==0.4.1
./scripts/env.sh python scripts/patch-rpc.py

The environment manifest includes the ROS and C++ build dependencies. Each command uses the environment configured in .env.

Simulation Environment and Model Weights

AirSim Environment

Download CabinLake.zip from the UAV-ON training environments and extract it locally. Retain the entire scene directory, including its packaged content and binaries. Set AIRHUNT_ENV_EXECUTABLE in .env to the absolute path of CabinLake/Binaries/Linux/CabinLake-Linux-Shipping inside the extracted archive. Scene assets are external to this repository.

The launcher supplies the AirSim settings and starts the packaged environment automatically.

Model Weights

Download the following checkpoints from their official projects and place both files in AIRHUNT_CHECKPOINTS:

Model Checkpoint Download
SAM 2.1 Large sam2.1_hiera_large.pt SAM 2
Grounding DINO Swin-T groundingdino_swint_ogc.pth Grounding DINO

CLIP (ViT-B-32, laion2b_s34b_b79k) and BERT (bert-base-uncased) are downloaded to the Hugging Face cache on first use. Ensure that the Qwen endpoint supports image inputs and that the configured model is available to your API key.

Usage

From the repository root, build and run the CabinLake umbrella navigation example:

./scripts/build.sh
./scripts/run-example.sh --timeout 180

Videos, trajectories and logs are saved to outputs/navigation-<timestamp>/.

Code Structure

configs/                      Camera, perception and navigation settings
examples/cabinlake_umbrella/   Example episode and launcher
perception/airhunt/            Perception, object memory, VLM reasoning and recording
ros_ws/src/                   Mapping, SOP planning, trajectories and AirSim bridge
scripts/                      Environment, build and run commands
third_party/                  Model inference code and SOP solver

Perception parameters are in configs/perception.yaml. The ROS entry point is configs/navigation.launch, and the SOP threshold rho=0.15 is configured in airsim_algorithm.xml.

Acknowledgment

We thank the authors and maintainers of the following open-source projects:

Source versions are recorded in UPSTREAM.json.

Citation

If you find AirHunt useful in your research, please cite:

@article{chen2026airhunt,
  author  = {Chen, Xuecheng and Liu, Zongzhuo and Ma, Jianfa and Du, Bang and Zhang, Tiantian and Wang, Xueqian and Zhou, Boyu},
  title   = {{AirHunt}: Bridging {VLM} Semantics and Continuous Planning for Efficient Aerial Object Navigation},
  journal = {IEEE Robotics and Automation Letters},
  year    = {2026},
  volume  = {11},
  number  = {11},
  pages   = {12495--12502},
  doi     = {10.1109/LRA.2026.3730212}
}

About

An aerial object navigation system that locates open-set objects in outdoor environments from natural-language instructions.

Resources

Stars

28 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages