Viewers in front of an augmented reality window display created by Future Colossal

BMWi: Window into the Near Future

Our Role
Development, Experiential Design Consultant


Advertising agency KBS enlisted Future Colossal to create a jaw-dropping augmented reality experience that would bring BMWi Concept Vehicles to New York City. With "Window Into the Near Future" we used our expertise in computer vision and real-time graphic processing to bring KBS' conceptual vision to life.

Displayed on a 6th Avenue storefront along Bryant Park, this project transformed traffic into BMWi Concept Vehicles, replicating weather conditions, time of day, and leaving iconic New York taxis unchanged. On foot or in a car, people were surprised as the future of the streets was reflected back at them.


BMWi: Window into the Near Future utilized custom tracking and display software, two computers, and three cameras. Each row of traffic needed to be tracked independently from a low angle camera. An integral component to the activation's functionality, we developed the tracking software to account for constant changes in lighting conditions, fast moving shadows from nearby trees and buildings, dense pedestrian and rush hour traffic, and disparate weather patterns.

The video above shows reflected cars on the glass storefront and the corresponding virtual BMWi vehicles.

Combined multi-threaded C++ and GLSL (GPU based Shader language for fast graphics processing) algorithms detected cars, while ignoring all other movement and changes in the environment. It then cataloged every car, determining whether it was a taxi or a normal car.

The car tracking data was sent to a second computer that rendered the augmented mirror display. It determined what was foreground and what was background, then used eight layers of image processing per car to match environmental lighting, add reflections, and animate the virtual BMW i8 and i3 cars. The corresponding image was rendered in an impressive 1/30th of a second, creating a perfect mirror of the street traffic.

Evening street view of AR window display created by Future Colossal with statistics caption


The software generated real time metrics for all cars and pedestrians that passed by the display. The car metrics were then tied into the BMWi series campaign, calculating potential savings in CO2 emissions, fuel consumption and cost. These savings were displayed live on the screen and could be seen increasing with each passing car. In addition, an automated backend system allowed for the display of pre-screened tweets.

Cars in NYC driving past Augmented Reality window being digitally transformed into BMWs