The map said there was nowhere to sit in Kolkata, so I asked Gemma to look

This is a submission for the Hacktoberfest Open-Source AI Challenge: Touch Grass.

What I Built

My father has knee pain. He likes walking, but he stops going out when he can't be sure there is somewhere to sit before his knee gives up. So the question for every walk is not "how far" but "where can I rest".

Bench to Bench plans a short walk around wherever you are, where a place to sit is never more than 150 m away, with no steps, gravel or cobblestones. It asks for your location, you pick a length and tap Start. Then the screen goes black, the phone goes in your pocket, and it talks: "Place to sit in 70 metres." "In 30 metres, turn left." "Halfway. The way back is just as gentle."

The screen is on for about ten seconds. The rest of the walk happens outside.

I tested it in two places, and the difference between them is the story:

• Mainz-Neustadt, Germany. OpenStreetMap knows 431 benches here. The app plans loops of 590 m, 987 m and 1,294 m straight from the map, and the longest stretch without a seat is 149 m.
• Kalighat-Hazra, Kolkata. OpenStreetMap knows zero places to sit. Map only, the app says "No safe walk possible here." Anyone who has walked in Kolkata knows that's wrong. There are tea-stall benches, shop steps, raised platforms and low walls everywhere. They just aren't on any map.

That gap is what Gemma fills.

Demo

The app: https://koushik890.github.io/bench-to-bench/web/ (allow location, then tap "Practice walk" to hear the cues without going anywhere).

Code

OpenStreetMap streets ──> walking graph (no

How I Built It

Gemma looks, plain code decides. That rule shaped everything.

1. Streets. OpenStreetMap data from the Overpass API becomes a walking graph. Code removes steps, gravel, cobblestones, private paths and slopes over 6%. In Mainz that drops 1.4 km of steps and 3.4 km of rough ground.
2. Photos. I pulled 165 street photos of Kalighat-Hazra from Mapillary.
3. Gemma 4 E4B, running locally through Ollama on my RTX 5060 laptop, looks at each photo and answers in a fixed JSON format: is there a place to sit, what kind, steps, broken footpath, shade, plus one sentence on why. Code checks every answer. If it doesn't match the format, the photo counts as a failure and adds nothing to the map.
4. Routes. A seeded search hops from seat to seat and only accepts loops where every stretch is 150 m or less. Gemma never sees this step.
5. The walk. A static web page. It downloads the streets and benches within 900 m of you and plans the walks in the browser, with the same rules. GPS snaps you to the route and Chrome's built-in voice reads the cues.

Result in Kolkata: Gemma found 7 places to sit in 165 photos (mostly low ledges and platforms in front of shops, plus chairs outside one shop). With those, the app plans a 254 m loop where you are never more than 69 m from a seat. The map alone could plan nothing.

It's a small walk, and I want to be honest about why. All 157 recent photos in that area come from one person, on about 40 points along a few streets. I checked the eight surrounding areas and none has a recent photo. Gemma can only find seats where someone took a picture. 600 m and 1 km walks were not possible.

How good is Gemma at this? I checked by hand

I labelled 40 Mainz street photos myself before looking at Gemma's answers, then compared.

What I take from this:

• For seats, Gemma beat the map on the same photos. A mapped bench 15 m away is often behind the camera. Gemma only names what it can see.
• For steps and shade, Gemma lost to a rule that always answers "no". E4B said "steps" on 16 photos that had none and "shade" on 26 that had none. So the route planner uses only the seat answer. This is exactly why the model reads and code decides.
• Only 3 of my 40 photos had a seat, so these numbers rest on very few examples. I labelled Mainz, not Kolkata, so the Kolkata seats are not checked by hand yet.
• Speed: about 0.9 s per photo for E4B and 0.6 s for E2B, all 165 photos in under 3 minutes.

What broke

• The model server crashed twice loading E4B for the first time (0xc0000409). The model only half fit in free GPU memory, so the first photo took 77 s. After that it was about 1 s. The script now retries failed photos.
• The app talked too much. In Kolkata the seats are about 20 m apart, so my first practice walk spoke 23 sentences in 254 m. Now it only announces the next seat and says "here" at most every 40 m: 12 sentences.
• The black walk screen sat under the map. The map library's layers had a higher z-index. I only caught it by pulling frames out of the test video.
• ElevenLabs was blocked. My free account got flagged for "unusual activity" on the first request, so the voice is the phone's own. It works offline, which turned out to fit better anyway.
• Nobody has walked a route yet. I built and tested everything at home with the simulated walk. I'd rather say so than make up a walk.

Why open AI matters here

• Street photos of where someone's father walks never leave the laptop. Gemma runs on my GPU through Ollama.
• It costs nothing per photo, so reading every photo of a neighbourhood is fine.
• Swapping models is one word, which made the E2B vs E4B test cheap. It showed the bigger model finds more seats and the smaller one invents fewer.
• I could see where it fails and build the rules around that, instead of trusting it.

Prize Categories

• Gemma: Gemma 4 E4B (and E2B for the comparison), running locally with Ollama, is the only way the app knows where to sit in Kolkata.

Map data © OpenStreetMap contributors (ODbL). Street photos from Mapillary (CC BY-SA 4.0). Built with help from Claude Code.