Osmo Systems: AI for efficient aquaculture

A novel AI-powered IoT water quality sensor for efficient aquaculture

Osmo Systems was a seed-stage climate tech company aiming to make aquaculture more sustainable when they hired us on as their software engineering team.

---

The context

Food is responsible for a huge percentage of our global carbon footprint. The reasons are myriad: tractors and other farm equipment guzzle fossil fuels, cows belch methane, forests are cut down for grazing and cropland. As always, there are many many pieces to this puzzle, and today we’re isolating one specific sliver.

In Should We Eat Meat?, Valclav Smil dissects the titular question with his customary exhaustive detail, considering the human body’s need for high-quality protein alongside the deleterious environmental consequences. And his conclusion is similarly complex—he recommends “‘rational meat eating’, where environmental and health impacts are reduced, animals are treated more humanely, and alternative sources of protein make a higher contribution.”

One such rational choice is shifting some meat consumption away from high-carbon-cost cattle in favor of lower-carbon seafood (both wild-caught and farm-raised).

(If you’re not sure of this claim, check out this insanely detailed and comprehensive article from Our World in Data on Fish and Overfishing.)

The problem

Aquaculture is crazy inefficient.

There are a variety of metrics that relate to the overall crop yield, but by far the biggest, baddest wolf is dissolved oxygen. Just like humans, aquatic life needs to breathe oxygen, but unlike us they get it straight from the water in the form of dissolved oxygen. Oxygen in water is produced in a diurnal cycle, peaking during the day when algae are actively photosynthesizing, and dropping overnight when there’s no sun to power this process.

These graphs quantify fish yield relative to primary water quality metrics: dissolved oxygen (DO), turbidity, pH. Note the near-zero yield when DO drops too far below optimal levels. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10238929/

This all feels a little academic, but I hope it hits home when I tell you that shrimp farmers in Thailand frequently (and I mean frequently) have their entire crop die off overnight, because their shrimp consumed all of the available oxygen and then suffocated. This isn’t just inefficient; this is catastrophic for their livelihood.

During in-field user research with Osmo Systems, we spoke at length with this Thai shrimp farmer. He lived just a few hours away from his hometown and family, but hadn’t been home in years. He couldn’t be gone for even a single night, he explained, because if the mechanical aerators failed overnight, his whole crop would die. He needed to sleep where he could hear the generators running.

Diesel-powered mechanical aerators. These are the sledgehammer solution to the water quality problem; they’re definitely going to keep the pond aerated, but they’re also wasteful and only actually necessary when dissolved oxygen is critically low.

The insight

Facing age-old problems, modern solutions demand better data powering more targeted insights. Aquaculture is an industry ripe for better data—if these farmers had real-time water quality data with alerts when any metric fell below critical levels, they would quite literally sleep better at night.

Unfortunately, existing water quality sensors are just too expensive, and so most onshore fish and shrimp farms don’t have adequate water quality monitoring.

The solution

At Osmo Systems, we were working to develop cheaper, smarter, lower-maintenance water quality sensors, packaged up into IoT devices to power a cloud-hosted dashboard with real-time water quality monitoring, alerting, analysis, and insights.

The Osmo Systems product, more or less.

In detail, the components of the product prototype. A single board computer to power on-device computation and communication, a Hologram chip to provide internet connection, the standard MQTT messaging protocol, all connecting back to standard cloud-hosted services on AWS.

From a software standpoint, managing a fleet of IoT devices and building a web dashboard is not entirely trivial, but certainly well within the world of known software solutions. Building this out is why Osmo Systems initially hired us on.

The hard part is the sensor design.

Osmo Systems was developing their sensor technology around small chemical patches that fluoresced proportional to the absence of dissolved oxygen (DO), i.e. the fluorescence was less bright when more DO was present. These patches are traditionally read using an expensive piece of equipment called a phase fluorometer. Osmo Systems bet on being able to measure those changes accurately enough using simpler technology, first with simple photodiodes, and later with off-the-shelf cameras.

A timelapse of images collected from one of our legion of sensor prototypes. Our challenge was: can we tell how much that dissolved oxygen (DO) patch is fluorescing. (Can you?) Also: how many times do we have to type out “fluorescence” before we spell it correctly on the first try?

The software

As it turned out, developing the sensors became a major R&D effort, and we, as the software team, pivoted to apply crucial acceleration to that process.

We paused work on the product prototype, and shifted into streamlining a R&D testing apparatus to accelerate the team’s ability to rapidly test new sensor designs. Our scientists and mechanical engineers built a water bath with precisely controllable temperature, pressure, and dissolved oxygen levels. We set up a raspberry pi and python script to run tests, setting the environmental conditions and collecting and storing the sensor data.

The components of our R&D testing, with data flowing into Amazon S3 for later analysis in Jupyter.

We taught our science team enough python to be able to analyze the data in Jupyter notebooks.

After reviewing the relevant scientific literature, we designed calibration algorithms to characterize the sensor’s response to dissolved oxygen and temperature, and to assess the stability of the fit over time to determine readiness of the sensor.

This model shows a high-quality fit of our data to the two-site model of the Stern-Volmer equation, which relates oxygen concentration to the measurable luminescence intensity of a given fluorophore. In our case, we were attempting to use our captured images to stand in for direct measures of luminescence intensity.

Despite some initially promising high-quality fits of the data to the latest models, we were eventually forced to conclude that simple image processing and standard ML regression models weren’t going to be sufficient to convert our sensor images into the data points we needed.

However, the software acceleration of this work meant it took weeks instead of months or years to identify that we needed to move on.

So, we pivoted again, this time to apply the most cutting-edge software to the problem: AI.

Specifically, we worked on training a convolutional neural network (a deep learning model subtype optimized for image processing) to predict dissolved oxygen concentration given an image of a fluorescing patch.

Convolutional neural networks are most often used as classifiers - e.g. identifying if an image is a “cat” or a “dog” or a “robot”. But we trained ours to produce a single numeric output, representing the concentration of dissolved oxygen.

We built out a custom machine, jam-packed with GPUs¹. We built our own models in TensorFlow, and used Weights & Biases to meticulously track and compare the performance of our various training runs.

We repurposed the R&D testbed to collect first hundreds and then thousands of images, captured alongside precise and accurate measurements of actual water quality metrics. We fed this as training data into our ravenous deep learning models.

And it wasn’t enough.

Our models failed to accurately predict dissolved oxygen across even a subset of the range of relevant environmental conditions. We were forced to conclude that there simply wasn’t enough information encoded in our images to extract the data we needed.

In conclusion..

In this case, the software team was able to vastly accelerate the company’s learning process, so that it took only months to conclusively decide that the big bet on cheap sensors simply wasn’t going to pay off.

As a result of this fast failure, Osmo was able to repurpose its remaining funds to pivot into a completely different climate solution vertical. Much of the project management DNA and software strategy that we contributed continues in the company’s current reincarnation as Carbon Collective, a climate fintech company developing stock & bond portfolios purpose-built for solving climate change.

1. This was 2019 and we were a little ahead of the AI gold rush, so these same GPUs later sold used for more than we bought them new.