How does a blind athlete swim in a lap pool? This is a problem my buddy David and I faced when he decided that he wanted to start racing triathlons. We had already been running and biking together, but swimming presented new problems: jamming your fingers in a lane line HURTS, and banging your head into the wall at the end of a lap hurts even more!  David is a visually impaired athlete, and as he does with everything, he jumped right in. We swam many laps side by side, with a short tether attached to our thighs, and I would tap his shoulder when we got a couple yards from the wall. It worked, but wasn’t great for many reasons.

We tried waterproof headphones connected to a mobile phone, but water and bluetooth signals don’t mix well. For one swim we used a tapping pole like they use in the paralympics, which worked, but also had a number of problems (good for when approaching the wall, not great for correcting when zig-zagging mid lap).

I did some googling, and came upon an exciting prospect: a device that a swimmer could wear that provides feedback to the swimmer in realtime.  I reached out to Annika Muehlbradt, the lead researcher on the Goby project, and she was super supportive when I told her of my interest in creating of a new version.

Software

The initial lane sensing algorithms were created by Yash Chitambar, who at the time was a high school student in my son’s Scout troop. Yash and his dad had approached me about internship possibilities in the computer programming field (difficult to land for a 17 year old), but he jumped at the opportunity to leverage skills he had learned in the robotics club at school.  During his free time, Yash put together a number of Python prototypes utilizing Canny Edge Detection, and then analyzing the found lines to determine position and direction within the lane.

The Canny approach worked well with the “clean” images we were using for testing, but struggled with real world scenarios like faded lines or the variety of end-of-lane markers in different pools. So the project pivoted to using a Machine Learning algorithm based on a Convolutional Neural Network. By using an AI model, we are able to handle a much broader set of environments. As of March 2025, the algorithm has an 87% accuracy rate. Our collection of training images is limited, so the accuracy should improve significantly as we test and use the device (it stores all the images it captures). Yash is also working on a Generative Adversarial Network (GAN) to help auto-generate training images.

I owe a huge debt of gratitude to my coworker Mark Holt, the super smart Data Scientist who developed our company’s algorithms that detect when a patient’s health is at risk due to their breathing patterns. Mark provided hours of guidance about selecting model types, tuning the model generation, and training data techniques. And Mark was also always there with an encouraging word when I ran into roadblocks…and there have been LOTS of roadblocks!

Hardware

The Goby II prototype is contained within a sturdy waterproof bag, attached to the swimmers belly with a belt.  The device’s camera is oriented so that it is pointing straight down when the athlete is swimming, so that it has a clear view of the lane’s centerline painted on the bottom of the pool.

The device itself has a number of components:

The core of the device is a Raspberry Pi 5.  I had originally hoped that a much smaller Raspberry Pi Zero 2 W would be sufficient, but it routinely took 9+ seconds to analyze a captured image, which is way too slow for our real-time needs. The current image classification scheme takes around .3 seconds per cycle. For now the board just lives in a standard Raspberry Pi case that also serves as a heatsink.

Images are captured using a standard Raspberry Pi Camera Module.  After trying many different image parameters, the system uses 640×480 color images.

To assure that images are not captured when the camera is pointed to the side, such as when the swimmer is taking a breath, a three dimensional accelerometer is used.  The accelerometer is also used to avoid processing images while the swimmer is standing at the wall.

I’m too embarassed to show the other side of the existing circuit board – my soldering skills are, well, it’s an overstatement to even use the word “skill”. I’m grateful for the offer from Gábor Kismárton to develop a proper circuit board design that can be sent to a pcb fabricator.

To communicate with the swimmer, GII uses two haptic (vibration) motors. If the swimmer is too far to the right of the lane, the motor on the right side of the device buzzes.  When the device senses an upcoming wall due to the painted cross on the bottom, it buzzes in a sequence. A flexible 3-D printed harness is used to position the haptic motors within the waterproof bag.

A custom made 3-D printed box houses the camera and a circuit board for the accelerometer and the transistor that drives the haptic motors.  The box is a little bigger than it needs to be, since it originally also housed the Pi Zero.

I’ve had challenges feeling the haptic motors through the waterproof bag, so I’m currently exploring using wired waterproof earbuds.  I held off on this, since I was reluctant to poke a hole through the waterproof bag. But I experimented with Flex Glue, and it adhered very well to the bag, and remains flexible after curing.

I thought power was going to be the easy part – a small phone charger power bank supplies plenty of juice for a RPi 5.  What I didn’t anticipate was that the current draw from the Pi so low that the power bank kept turning itself off after 30 seconds of what it thought was inactivity.  I switched to an Anker Power Bank that has a trickle-charge mode, and that solved that problem.

The waterproof bag I used has a web belt, but I found that it wasn’t very stable.  Tucking the bottom of the bag into the pocket of a running belt helped keep everything in place.

Current status

The project is still very much in the prototyping phase. As mentioned above, the classification accuracy is pretty good, but is not yet at a level that would be reliably safe for a vision impaired swimmer. And the hardware components are truly at the “proof of concept” level. I haven’t been too constrained by size, as the bag I’m using was the smallest I could find that would do what I wanted – but everything could much smaller.

We still need to do some testing on use of the headphones, and whether the audio is loud enough with all the splashing while swimming. If it’s too quiet, an amplifier may need to be added.

Link to the Github repository: https://github.com/mikehedman/goby