GRIT-model-with-GPT4
Last updated on 9 months ago
- Github_link:
Overview:
GritAndGPT4 is a project designed to harness the capabilities of the GRIT (Generative Region-to-Text Transformer for Object Understanding) model to capture object descriptions. These descriptions are then processed by GPT-4, which generates safety suggestions to enhance driver awareness.
Figure
Coordinates and descriptions from GRIT model
Image size: 1920x1080
List of object coordinates [x1, y1, x2, y2] and object descriptions:
[67, 123, 284, 311] the car is silver in color
[209, -236, 359, -88] a business promotional sign
[348, 136, 710, 387] a car on a street
[412, 283, 528, 388] tire on the car
[295, 106, 948, 420] the car is silver
[-288, 80, -238, 135] red and white stop sign
[-885, -535, 64, 281] green tree on the sidewalk
[-954, 234, -263, 534] the bushes are green
[-906, 253, 893, 523] the black paved road
[-417, 273, -172, 533] red paint on the curb
[-184, 180, -127, 272] motorcycle on the sidewalk
Safety Suggestions
Please be cautious of the car on the street ahead and the motorcycle parked on the sidewalk. Don’t forget to stop at the red and white stop sign up ahead, and watch out for red paint on the curb as you drive.
Figure
Coordinates and descriptions from GRIT model
Image size: 1920x1080
List of object coordinates [x1, y1, x2, y2] and object descriptions:
[-310, 85, 157, 488] white van on the road
[-136, 234, -19, 291] license plate on the car
[471, 47, 771, 534] person riding a motorcycle
[-747, 30, -504, 522] a man on a motorcycle
[-888, -442, -432, -190] a white sign with red letters
[-735, 114, -525, 305] the shirt is white
[501, 143, 738, 350] a red shirt on a man
[-669, -207, -443, 102] the blue sign on the building
[-550, 370, -297, 536] double yellow lines on the road
[232, 152, 347, 362] person riding a motorcycle
[547, 48, 652, 153] the helmet is black
[-739, 388, -635, 441] license plate on the motorcycle
[-956, 158, -779, 404] a white truck on the road
[382, -272, 740, 25] a tall green glass building
[-929, 56, 936, 525] people riding scooters
[-394, 185, -295, 267] the car is black
[52, -99, 373, -28] signs on the building
[-676, 29, -579, 122] white helmet on the man
[754, -129, 954, 120] palm tree behind the bus
[-755, 292, -533, 534] the motorcycle is black and white
[-959, -524, 949, 531] a city street scene
[356, 110, 521, 409] person riding a motorcycle
Safety Suggestions
Keep an eye on the white van and the white truck on the road. Be cautious of multiple motorcycles and scooters around you. Pay attention to double yellow lines on the road, as they indicate no passing or crossing over to another lane. Also, stay aware of pedestrians who may be near signs and buildings along the city street.
Figure
Coordinates and descriptions from GRIT model
Image size: 1920x1080
List of object coordinates [x1, y1, x2, y2] and object descriptions:
[193, 87, 256, 270] the man is wearing a green shirt
[297, 84, 575, 262] black and white car parked on street
[664, 3, 782, 510] the metal sign post
[-953, 103, 813, 535] the road is clear
[441, -13, 507, 74] red and white sign
[-485, 332, -257, 535] yellow lines on the road
[-233, 105, -176, 196] a red and white sign
[198, 110, 250, 169] the green shirt the man is wearing
[-955, 144, -668, 413] chain link fence on side of road
[-278, 169, -243, 227] orange and white cone
[-954, -534, -386, 275] the tree is green
[-957, -527, 954, 522] the trees lining the road
[-428, -386, -103, 238] tree with green leaves
Safety Suggestions
Drive cautiously and pay attention to the yellow lines on the road. Keep an eye out for the orange and white cone up ahead, as well as the man in a green shirt near the black and white car parked on the street. Also, be aware of potential pedestrians near trees lining both sides of the road.