June 20, 2004 notes

 

Good intentions, bad results

 

If anyone missed the mid-week update I did about the perfect flight of the small vehicle, go read that first.  We had problems trying to hover the big vehicle this weekend.

 

A minor improvement I have made to the laptop software is to display the dilution of precision (PDOP) number from the GPS on the status light, because previously it would have been green even if it was running with very poor coverage.  I need to spend some time and collect more GPS diagnostic information, like satellite S/N ratios, but I’m not sure if I want to add a lot more traffic to the serial port during actual run times.

 

The first problem we ran into was installing the batteries in the new battery boxes on the big vehicle.  We have started using upset thread lock nuts for a lot of things now, but when rapidly tightening them down on one of the battery clamp studs, it galled onto the stud.  We use stainless for everything, because a little mist of peroxide will corrode steel bolts pretty badly, but galling is a common issue for us.  Using anti-seize prevents this, but we should probably use nylon insert lock nuts in areas that aren’t going to be heat affected, and leave the all-metal locking nuts just for areas near the engine.  We had to cut off the studs and weld new ones in place, all inside the vehicle, because the battery boxes are permanently bonded in.

 

When we rolled the vehicle outside, we noticed something off when I ran an extra check on the jet vane controllers: vane one was moving about half speed one direction, but full speed the other direction.  We puzzled over this a bit, finding that swapping controllers around always had that actuator displaying the move-slow-one-way behavior, so we were confident it was the actuator.  This was the new one that we had replaced after melting one of them last test.  We were thinking that we could still test with a slow drive, because it was still moving fast enough to work, but while exercising it with normal flight movement, Phil looked over our way and let out a cry of “Smoke!”  A pretty good quantity of smoke was coming out of the cabin where the electronics were mounted.  One of the motor drives was fried.  Fortunately, we have plenty of spares of the driver board, so we prepped up another one and replaced it.  To make us even more confused, when we tested the problem actuator directly with a manual switch box, it was full speed both ways, and when we pulled the valve off and tested it with a different channel on the burned board, it also ran fine both directions.  The new driver board still showed the slow-one-way problem, so we stopped quickly to avoid smoking it.  We finally thought to check the resistance from the motor drive to the vehicle frame, and found that there was a low resistance path from one of the drive leads to the motor shaft, which makes its way to the vanes, conductive graphite bearings, up through the engine, and back to ground through the spark plug return line.  None of the other valves were like this.

 

http://media.armadilloaerospace.com/2004_06_19/burnedDriver.jpg

 

When we pulled the actuator apart, we found out what was wrong: We make a few modifications to the actuators, primarily removing the thermal cutout and soldering on any internal quick connect terminals, but we also usually solder the drive wires directly to the board instead of attaching them through the little bare wire push down connectors that they are normally connected with.  We found that one of the wires had been pushed so far through the hole when it was soldered that it brushed the metal case below it.  That made perfect sense.  The actuator was perfectly normal when bench tested, because the motor connector didn’t have a path to ground.  When we drove it with a manual switch and battery, it could draw enough current to still move full speed even though many amps were shorting directly to ground.  When the motor drive was pushing it, it would suck as much power as the transistors would give it, burning them up.  We trimmed the wire down, put the valve back on, and everything worked properly.

 

We set everything up, and started loading propellant.  Because we were collapsing the vacuum hose last time, we rearranged things so that the venturi vacuum pump was plugged directly into the vehicle, with the hose going to it carrying 70 psi air.  It took us a little while to get the adjustment setting where it wasn’t wasting huge amounts of nitrogen, but the general arrangement works well.  It is unfortunate that we need a ladder to connect to the vehicle top port, we will probably try and get all connections at the bottom in the future, using a fountain tube to get the inlet above the liquid level.

 

Things looked good when we started pressurizing, but when we got over 200 psi, it started clearly leaking into the engine.  After a little while, the engine popped and started crackling from internal combustion, which we know is bad for its lifespan, so we hustled to get things disconnected and ready to go.

 

Just as I was about to start, I lost the flight computer.  It did that same thing last test on the big vehicle, but it never happened once on the small one, so there is definitely a problem someplace.  The computer came back again by itself, and we cautiously tried to burn off the propellant.

 

We had covered the engine facing sides of the 2’ foam stand cubes with adhesive aluminum insulation, hoping that it would keep the foam from turning into flying molten plastic, but as soon as the engine was throttled up a bit, the aluminum on all blocks just blew right off.  The vehicle then showed the same problem as last time, even though we were at a higher tank pressure: as soon as it lightened up, the blocks blew out, but the slow valve and low thrust to weight had it falling down and bouncing on the tether before it gained positive thrust.  Just as well, because it turns out than vane zero wasn’t moving at all.

 

I gave it enough throttle to slacken the tether a couple times, but it obviously wasn’t able to control itself with one dead vane.  The leaking propellant valves kept the engine heating up continuously, rapidly bringing the entire nozzle to full red heat.  Just as I finally got all the propellant burned off at a fairly low throttle, three little fires lit up down by the nozzle.  We got them extinguished quickly, and took stock of the situation.

 

http://media.armadilloaerospace.com/2004_06_19/hotEngine.mpg

 

The aluminum covering on the foam blocks was a horrible mistake.  Melted plastic last time was bad enough, but the adhesive on the back of the aluminum was a tar like substance, and it now covers everything, making about the worst mess you can imagine.  Foam worked so well on the lander vehicle, but for the big one it looks like we really need to make the blow-out blocks from metal, and make them rigid, both to avoid melting them and to avoid the premature blow-out problem.

 

http://media.armadilloaerospace.com/2004_06_19/mess.jpg

 

The little fires were from the RTV we used to seal underneath the inner bearings.  I expected RTV to ablate instead of burn, but cooking them long enough did set them on fire.  We need to switch to a Cotronics ceramic sealant.  All of the actuators and wiring seemed to do just fine, even after the extended high heat soak.

 

Vane zero was stuck in the 45 degree position the entire time, getting a full heat soak, then getting hammered by liftoff level thrust.  This vane was bent after the test.  All the others were fine, because the computer never moves them more than 20 degrees from vertical, and they usually stay within a few degrees of vertical.  We are going to hammer this one straight and continue using it, but we will probably move to a better alloy, as well as thickening it, on our next upscale.

 

http://media.armadilloaerospace.com/2004_06_19/bentVane.jpg

 

When I did a testValves, vane zero did a complete 360 cycle, then went back to normal 90 degree range, indicating that it had been hung up on one of the limit switches.  We have seen this a couple times before, and I think I know what may have happened.  We had had been doing testValves to check the speed issue, which cycles the valves all the way each direction and reports stats.  We probably left it in that position before the test, and when we pressurized the tank up and the engine started going by itself, the gas flow may have exerted enough force to nudge it past the limit switch.  I should have the computer always try and leave the vanes at zero degrees when possible.  Even though the valve is back in range, going back and forth properly, the pot feedback range has shifted.  We have had this happen to several valves over the years, and we don’t know exactly why.  I will recalibrate, but it is a concern.

 

We also didn’t get thermocouple data on this test, so I spent some time on Sunday finding out why.  It turns out that when I rebuilt the electronics after the lander crash, I changed the thermocouple pinout slightly, and I forgot to duplicate the change in the big vehicle after making it in the new subscale vehicle.  A mistake I made correcting it did give me a useful data point:  I had made a connector to just bridge +12v onto the thermocouple signal wire to give me a definitive max scale A/D reading.  After I found out that I had the wrong signal wire, I changed it to a different wire, but I accidentally put it on yet another wrong one.  When I plugged my bridge connector in, the laptop lost communication with the flight computer.  After unplugging it, I could reconnect with it, and found that the flight computer hadn’t rebooted, it had just lost communication.  I had accidentally connected the signal to a ground, and when I bridged +12v to it, the DC/DC converter couldn’t supply enough power to the wireless radio, which shares +12v with the sensors.  The computer is on a separate +5v DC/DC converter, so it was unaffected.  This is good to know.  If we lose contact, but come back to find the flight computer not rebooted, it is probably a voltage problem with sensors.  If I had been running the sensors on unregulated power, a short like this would have probably generated more smoke and brought everything down.  I finally got the thermocouple on the right wire, and it works fine now.  I need to check the vane actuators and see if rotating them all the way around ever causes the pot to sit in a zero resistance range, which would draw bring down the telemetry.  That might be what happened today.

 

We really need to get a non-leaking ball valve before testing again.

 

I am going to write some more automatic checkout code for the flight computer, so it won’t even start if any of the vanes aren’t moving, or are out of the calibrated pot range.  An override would be required to operate a malfunctioning vehicle (to vent propellant, for instance).  I am hesitant to do the same thing with the throttle, although I could, in theory, alternately open and close the master cutoff and the throttle for full checkout without venting anything.

 

There was another anomaly during the flight that we haven’t noticed before: the main throttle valve was hunting close to zero even when I wasn’t doing anything.  I think this is a result of recent software changes, and it is benign, but I should fix it to reduce the stress on the drivers and actuators.

 

We are still waiting for big shock absorbers to replace the wire rope isolators.  We aren’t going to do a ground liftoff until we have them installed.

 

In other work, we are setting up improvements to our test stand plumbing, and Phil made a fiberglass nose for the small vehicle, which we may use as a GPS-transparent ejectable cap to allow us to have an emergency parachute on the vehicle, if AST will allow it on our waiver.

 

http://media.armadilloaerospace.com/2004_06_19/fiberglassNose.jpg

 

 

Good luck to the Scaled team on the flight tomorrow morning!  I’m a late riser, so by the time I check news, there should be plenty of writeups on the entire thing…