Chatting with the Inverter

 Hey Electrified Miata team, here's the latest update in the never ending quest to go faster, have fun, and be safe! For those of you following along, last week, we left our story with a disappointing conclusion of not being able to communicate with the inverter yet. I was reminded of when I had failed at a previous programming job, shame and self-doubt. Please join us on our journey this week as we work through our roadblocks and discover more challenges.   I think my Mom may have joined the team this week, because I could hear her say "If a first you don't succeed, try try again".

This is a detailed discussion as an illustration of 'what it takes' when things don't go right. Details are what count with perseverance and team work. I'm tired just reviewing our weeks progress but it also brings a smile to my face remembering the many small victories along the way.

In preparation for car night, I started puzzling on what is preventing us from chatting with the inverter. Joe sent me five links containing information about CAN bus protocol and I started reading.  I'll save y'all the myriad technical details and pages of scrolling: both ends of the CAN bus need termination and all CAN nodes need to share a common ground.  On the software side, I discovered the transmit queue has a default size of 10.   That means in less time than it takes to sneeze, I'll have filled the queue.  I can change the queue size and I set it to 1000.  I rushed out to the garage to try all these improvements and fixes. My software no longer dies with "no space left" but still, no communication with the inverter.

My wife commented I seemed obsessed and despondent, and I have to admit that's true. Despite my obsession, I needed rest. A bit of sleep can work wonders. I sprang out of bed Monday morning with new ideas. Two facts stood out for me.  One, Joe captured a message that can only be from the inverter. His laptop and logic analyzer was able to receive an inverter message. Two, my Raspberry Pi's can talk back and forth, but not with the inverter.  With all the new data on CAN, thank you Joe, I calculated that 1 CAN message should take about 200 microseconds to send at the 500,000 baud rate the inverter uses. I got my scope connected, fumbled through the setup, and finally conclude that it looks like my message is taking about 414 microseconds to send..  This is a clue!    Early on I read something about the clock data sent to the chip driver needing to be two times the oscillator rate.   I remember thinking that seemed  counter intuitive to give the driver parameters that don't match any obvious markings on the hardware.   Guess what?I'm off by a factor of two and setting that particular value higher would cause the system to operate at half the intended rate.    I reconfigured the drivers on both Pi's and they still can talk with each other.   However, the scope tells me the message now takes 204 microseconds, much closer to my estimate.   I've also changed the software so the power controls are separate from the CAN messaging. I set up the CAN monitor, power up the inverter and see thousands of glorious messages from the inverter!   Challenge one overcome!  I did the happy dance all the way into Teresa's office to celebrate. What challenge is next? Oh, that's right, let's see if the software can interpret anything the inverter was saying.

I started the CAN message software and Whoop!  We have two temperature readings, the inverter reports 48F degrees  and the motor says 52F.   I zipped inside to get Teresa's keys so I can see what her car says the temp is in the garage. Her car displays 55 degrees, so everything is in the same ball park!   Of course the voltage looks all messed up and inverter reports a fault after 10 seconds just like described in this link. I'm now in possession of one, possibly 3 very heavy temperature sensors that require a 12 volt battery to run.  I'm not convinced these Leaf motors were a very good bargain yet.

I cobbled together some batteries in series and connect that 16.7 volt pack to the
inverter.  After a bit of debugging and fixing, I got the inverter to tell me the voltage it sees on the battery bus.  Progress!  I own a heavy voltmeter, good to about 500 volts that can read in 2 volt increments. I'm still not convinced this is a good bargain, but it is getting better.

Now I use my knowledge of the message format to decode the motor shaft RPM's. I try this, and guess what? I can read both the direction and how fast the motor is turning when I turn it by hand.   Wow, the bargain got just a little bit better.  I started to wonder about how I can test if the RPM's reported by motor is accurate.  An idea popped in my head that with a little math, moving the shaft a certain distance over a measured time, I could find out if the RPM's reported by the motor are accurate.   I found a string that I could experiment with and wound it around the motor and pulled to unwind it.   The results puzzled me because there was voltage on the inverter bus with no battery connected.   At first I thought it was a software bug, but it was just too easy to put the voltmeter on the battery bus and I discovered I had a generator in my shop too! I was jazzed about this.

So now I have a really heavy thermometer, a low resolution voltmeter, an RPM counter and a generator, capacity unknown.   Still not sure if this was a bargain, but it continues to get better and better. I never did get around to measuring to see if the RPM's are accurate.   I'll save that problem for when the motor is spinning.

While I was playing with the inverter, the rest of the guys were busy too.  Joe, working on the BMS,  discovered a problem in his wiring, fixed that and got proper voltage measurements for a simulation of a battery cell!  He also deciphered more the code and documentation which lead him to discoveries on why the temperature sensors seems to be giving strange results. Joe will be ordering a few test boards soon to see how everything works as a package.   Bruce picked up the go-kart and enlisted his son Eric to help dis-assemble and re-furbish it for use as our test mule.

It got rusty sitting out in the shed
for quite a few years.  We need a new chain and actually had one from when the kart was first constructed.  Ed got a good mold of the Leaf motor shaft and supplied all the parts to get our test power supply up and running during car night.  Ed, thanks for the extra trip to get the better variac.

Guess who I emailed and responded? Tim Burress! He was the team lead of the Department of Energy's Benchmarking EV and HEV Technologies at Oak Ridge National Laboratory [ click here ] They studied the leaf charger. I was blown away that he actually replied back to my email requesting a little more information on how they 'took over' the charger controls.  He verified the team did that on a hardware level.  That is the likely route that we will pursue also. How cool was that? Ask and ye shall receive.

When we got together for car night, we decided to try and make the motor spin.    Ed supplied a power supply that we had used to launch fireworks (out in the country)!

​We rapidly converted that to power our inverter.  As a quick and dirty test, we used a variac (variable tap transformer) to supply voltage to the Inverter battery bus, avoiding the need for a pre-charge circuit.  These variacs have been around in our family ever since  I can remember.   I couldn't find a build date on them but the are likely older than I am. The first one protested with sparks when we tried to ramp up the voltage.  We discovered the brush was worn down to the nub and couldn't make contact anymore at the higher power levels.   Ed went and got another one that was able to power the inverter at the higher voltage settings. He also brought this little gem for measuring RPMs.

 After all that, the inverter still went into the fault state despite reaching 150 volts (this document states 140 volts).   Stay tuned to see how we deal with this faulted inverter challenge.   Overall, we had a great week with lots of progress on all fronts. I'm feeling more hopeful we can make the charger work and get the Leaf motors turning soon as I share this.