This repository contains the code for the paper Fast and Efficient Transformer-based Method for Bird’s Eye View Instance Prediction by:
Miguel Antunes
Luis M. Bergasa
Santiago Montiel-Marín
Rafael Barea
Fabio Sánchez-García
Angel Llamazares
Change the nuscenes path in the makefile. If WANDB KEY is not set, the code will ask for it. If you don't want to create or use your account, yo can log anonymously.
Download the NuScenes dataset from the official website and extract the files in a folder with the following structure:
nuscenes/ └──── trainval/ ├──── maps/ ├──── samples/ ├──── sweeps/ ├──── v1.0-trainval/ └──── v1.0-mini/
Configure the path to the NuScenes dataset in the Makefile:
NUSCENES_PATH = /path/to/nuscenes
% Table with the models and their performance
| Model | VPQ short | IoU short | Params (M) | Latency (ms) | Checkpoint |
|---|---|---|---|---|---|
| Full | 53.7 | 59.1 | 13.46 | 63 | 'Full model ckpt' |
| Tiny | 52.3 | 57.5 | 7.42 | 60 | 'Tiny model ckpt' |
Build the Docker image with the following command:
make build
You can configure the following parameters of the image in the Makefile:
IMAGE_NAME: Name of the generated Docker image.
TAG_NAME: Tag of the generated Docker image.
USER_NAME: Name of the user inside the Docker container.
Once the image is built, you can run the container with the following command:
make run
This command will run a bash inside the container and mount the current directory and dataset inside the container.
Inside the container, you can:
Evaluate the model:
python val.py --checkpoint 'Path/to/model.ckpt' --dataset_root 'Path/to/nuscenes'
Visualize the predictions:
python predict.py --checkpoint 'Path/to/model.ckpt' --dataset_root 'Path/to/nuscenes' --save_path 'Path/to/save/predictions'
Train the model:
First, you can configure some training parameters in the prediction/configs/baseline.py file. We also provide the configuration files for our models in the same folder.
If you want to use your Wandb account, you can set a WANDB_API_KEY environment variable with your API key.
The model can be trained with the following command:
python train.py --config 'config_name'
Where config_name is the name of the configuration file without the .py extension in prediction/configs. New custom configurations can also be added to this folder.
If you specify a checkpoint path in baseline.py you can:
Load the weights of a pre-trained model if LOAD_WEIGHTS is set to True.
Continue training from a checkpoint if CONTINUE_TRAINING is also set to True, which mantains the optimizer and scheduler states.
If you have any questions, feel free to contact me at [email protected].
