FDT5-Model
Last updated on 8 months ago
- Github_link:
Overview:
The FDT5_Model is a system capable of generating engaging questions from specified locations, represented by a vehicle’s GPS coordinates and four side street-view images captured by on-car cameras. We utilize data from the Google Street View dataset and craft prompts based on the address obtained through reverse geocoding of the GPS coordinates, complemented by captions from street-view images generated by an advanced image captioning model. This repository demonstrates street views and coordinates from various locations, including USA_Pittsburgh, USA_Orlando, and USA_NewYork, to create engaging questions.
Demo Video:
Four side images USA_NewYork:
coordinate 40.73055,-74.001715
5 generated Engaging Questions:
- “How do you think the streets around us compare to other cities you’ve visited?”
- “Can you spot any local businesses or cafes nearby? How do you think these restaurants contribute to the overall atmosphere and energy of New York City?”
- “As we drive along this busy city street, can you spot any architectural styles or designs that stand out to you?”
- “Have you noticed the diverse range of shops and restaurants in this area? Did you know that New York City has over 500 hotels and restaurants?”
- “Have you ever noticed the history of this neighborhood in New York City? How do you think the traffic flow in your hometown compared to other cities you’ve visited?”
Four side images USA_Orlando:
coordinate 28.541323,-81.380703
5 generated Engaging Questions:
- “How do you think the streets around us compare to other cities you’ve visited?”
- “Can you spot any local businesses or restaurants that you’re excited to visit during your stay in Orlando? Do you find it appealing?”
- “As we drive along this bustling city street, can you spot any unique architectural features or architecture that stand out to you?”
- “Have you ever taken a ride in a neighborhood like Orlando? The city is known for its lush landscaping and cultural diversity.”
- “Are you familiar with the history of Orlando’s landmarks and buildings in the area? Orlando is known for its vibrant atmosphere, a vibrant, and many museums and restaurants.”
Four side images USA_Pittsburgh:
coordinate 40.440309,-80.0
5 generated Engaging Questions:
- “How do you think the streets around us compare to other cities you’ve visited?”
- “Can you spot any local businesses or cafes nearby? How do you think these restaurants contribute to the overall atmosphere and energy of Pittsburgh?”
- “As we drive along this bustling city street, can you spot any unique architectural features or architecture that stand out to you?”
- “Have you noticed the diverse range of shops and restaurants in this area? The busy city street to our south has a diverse cuisine. Can you guess what kind of cuisine is occupying?”
- “Have you ever taken a ride in Pittsburgh before? We’re currently in the heart of the city.”