edit: I've released a working implementation of the ideas presented below here.

Throughout the years there have been discussions around utilizing a microphone and a computer to record audio from an ongoing roast and then alerting the user (or adjusting the roaster) when the sounds of first crack has been detected.

Prior threads, papers, and claims of patents pending all appear to be approaching the problem from a traditional signals analysis methodology, where one attempts to strictly quantify the defining spectral and temporal characteristics of first and second crack using FFTs and similar tools. I observe an absence of actual implementations following this approach which suggests that it may be a bit harder to generalize and to implement into live production roasting conditions.

Fortunately, time moves on and new technologies make themselves available. In December of 2015 the pyAudioAnalysis toolkit was released allowing the application of modern machine-learning approaches to signals analysis of audio recordings. This approach has been used in variety of use cases from ornithologists identifying bird calls to detecting unauthorized people in your pool by identifying the sound of splashes. That last use case is interesting to me as it was a one-semester undergrad project by students who don't appear to have a signals analysis background. This suggests that this problem is approachable without a graduate degree in digital signals analysis utilizing commonly-available hardware and libraries. A paper outlining the general use of this library was published earlier this year which is good reading and provides a solid overview of the capabilities and requirements.

Machine learning requires a sample set to learn from. The initial paper linked above used all sorts of high-end recording gear and studio space which is great for isolating the particular sound you're hunting for, but doesn't do a great job of representing an actual roasting environment. Whether you're a large scale commercial operation or a home roaster like me standing next to an open window, one thing is fairly certain - you're not roasting in an anechoic recording chamber. Machine noises, barking dogs, and rounds of applause from your adoring customers are all going to be present in any sort of real-world application.

To that end, I've gone and purchased a dirt-cheap USB microphone that will work under basically any OS (Windows for my development and current roasting setup, Linux on Raspberry Pi as the desired development target). I've added an alarm to Artisan to launch a recording when the Charge event is triggered, stop it at Drop, and timestamp the file so I can line it up against Artisan's own logs later. This then will let me segment the recordings later into "crack noises" and "normal machine/environmental noises" for the training process.

The resulting recordings will then be used to train a classification model, which can later be run in realtime to detect the desired signals. Performance testing against a Raspberry Pi 2 (shown here as Ratio 3) suggests this can be done in realtime, but might leave little room for running Artisan at the same time.

As I only roast for home use, my initial efforts are going to be limited by the amount of data I can collect, which itself is limited by the amount of coffee my family can consume I expect it to be a few months before I have a usable corpus to train against.

If anyone else has any interest I'd love to hear about it. This is an old idea that (to my knowledge) has never been successfully deployed in a general-purpose way, but which should now be approachable with the application of newer analysis tools. Further, I suspect that it can be developed by neophyte users like myself and others here on HB.

edit: a working implementation of the ideas presented above is now running! The system is fast and accurate and usable in realtime. Still needs some polish to make it portable for other users, but the core concept is proven out and working.

Replying to my own thread here to pose a question for the audience: how do you, personally, define first crack? In my experience, there will be individual cracks a minute or more before things really get rolling. I use a general rule of "3 cracks in a second" (ish) before I mark the event in Artisan. It has never really occurred to me to quantify the thing in detail and I'm interested to hear what others have come up.

Great read. Thanks. My marker for first crack is determined the same as you.

I can also see this being useful for automating the blower in a fluid bed roaster. If we can't detect bean circulation noise then increase blower power.

What sort of roaster are you going to test this with?

Interesting idea and i like the sensor aspect.

Sound seems a bit remote as an indicator. If you are roasting to a plan, it's based on reaching a bean temp by a certain proportion of time of the whole roast -- 75-80% with Rao I think. So you need to have planned the roast in advance.

While there are still some roasting to 1c then starting to count after that as "development time", i think that approach has been undermined by Rao's argument that all roasting is development time. I believe Rao argues that the whole profile counts... The shape of the curve, length of roast, and drop temp.

With high charge temps, then power off to turning point, then power on and extended drying up to peak temp then declining ROR, I can certainly roast up to 1c and produce a sweet, rich, balanced cup without going beyond, or eben necessarily hearing anything. The only thing eluding me is consistent florals.

i would certainly be interested in humidity changes and weight changes. A system that automatically records input weight, output weight... It's in my plans. Sound, i'm not so sure.

luma wrote:Replying to my own thread here to pose a question for the audience: how do you, personally, define first crack? In my experience, there will be individual cracks a minute or more before things really get rolling.

Outliers. Often immature or seriously undersized beans. I ignore them.

I use a general rule of "3 cracks in a second" (ish) before I mark the event in Artisan. It has never really occurred to me to quantify the thing in detail and I'm interested to hear what others have come up.

Similar to mine. I mark at a series of cracks, 3-5 in rapid window of 2 seconds.

If you need more data, i would be happy to also record samples for you when i roast?

greencardigan wrote:What sort of roaster are you going to test this with?

This will initially be developed on a HotTop 2k+. I have a Behmor here that I might fire up for some data collection at a later point.

happycat wrote: Sound seems a bit remote as an indicator. If you are roasting to a plan, it's based on reaching a bean temp by a certain proportion of time of the whole roast -- 75-80% with Rao I think. So you need to have planned the roast in advance.

While there are still some roasting to 1c then starting to count after that as "development time", i think that approach has been undermined by Rao's argument that all roasting is development time. I believe Rao argues that the whole profile counts... The shape of the curve, length of roast, and drop temp.

I actually agree with this and it's how I run my own roasts. I'm using Artisan connected to a 2k+ which allows almost complete programmed automation of my roast based on temperature. I record FC manually just for review later, but with a given batch of greens it's always within a couple degrees based on the BT probe so my Artisan alarms are triggered by temp. This exercise is mostly a vehicle for me to learn some ML tools and to attempt a feat which, to my knowledge, has never successfully been accomplished. If it works, great, I'll write it up and share the results. If it doesn't work, also kinda great because I'll still likely have learned a bunch in the process.

happycat wrote:i would certainly be interested in humidity changes and weight changes. A system that automatically records input weight, output weight... It's in my plans. Sound, i'm not so sure.

You've already talked me into tackling the weight thing next I went and ordered an HX711/load cell kit to do some experimentation with the ESP8266 and intend to present it the via HTTP as I did with the DHT22 project. Expect another thread when the slow boat arrives.

frachlitz wrote:If you need more data, i would be happy to also record samples for you when i roast?

I might make such a request at a later date, but I want to confirm that this has any chance of working at all before I get anyone too spun up on collecting data for me. If it works, expect a PM

luma wrote:I actually agree with this and it's how I run my own roasts. I'm using Artisan connected to a 2k+ which allows almost complete programmed automation of my roast based on temperature. I record FC manually just for review later, but with a given batch of greens it's always within a couple degrees based on the BT probe so my Artisan alarms are triggered by temp. This exercise is mostly a vehicle for me to learn some ML tools and to attempt a feat which, to my knowledge, has never successfully been accomplished. If it works, great, I'll write it up and share the results. If it doesn't work, also kinda great because I'll still likely have learned a bunch in the process.

In that case, I say more power to you.

At some point I plan to share how I've integrated my Gaggia Classic as an Internet-controlled device with automation, settings adjustment, and control via iOS devices. Not to mention the true insanity of the planned flow sensor, ultrasonic rangefinder for the water tank (pressure sensor wasn't reliable), multicolour LED to illuminate the tank based on mode, and a voice module and realtime clock so it will have awareness of the time and date and a little personality.

Looking forward to more.

I kind of think we should have an "automation/controls" forum.

My cheap-o USB microphone arrived and true to the Amazon listing, it required no drivers to install on a Windows 10 system and immediately was setup as the default recording device.

I'm using SoX to record audio. I've created a couple batch files to start and stop the recording process and saved them into a "scripts" folder underneath the Artisan install folder.

Here's the start_recording.cmd script: Here's the stop_recording.cmd script: An alarm was created to start recording at the start of roast logging like this:


...and another to stop recording 30 seconds after the drop so I also catch the sound of the beans falling into the cooling tray along with some agitation noise:


I use a pair of gooseneck LEDs on my roasting cart, so I've ziptied the microphone to one of the lights to kinda point at the roasting chamber and ran a USB extension from it to my laptop.


The resulting WAV file was around 77MB for 14:33 worth of single-channel recording at 44khz/16bit. I made sure to make some extra environmental noise, got my dogs to bark, made some clapping sounds, let the air conditioning run, turned up my window fan, and some jagoff on a Harley did me a favor and gunned his engine in front of my place (surely a sign of being young and virile despite all outward appearances to the contrary). A quick review of the recording sounds good, the cracks are clear and so is the rest of the noise.

So far, so good. Now to get a working Python environment put together to run the pyAudioAnalysis scripts and test against my (limited) training set.