September 22, 2001 Meeting Notes

 

In attendance:

 

John Carmack

Phil Eaton

Russ Blink

Bob Norwood

Neil Milburn

 

We were hoping to lift off with a person today, but we had a really bad crash in our full-up test flight. Nobody was hurt, but we broke a lot of things.

 

Bob brought over the finished support clamps for the joystick and twist throttle, and we finished up everything on the frame. It was completely done.

 

Russ finished the new custom motor driver board, and we started testing it. We had a bunch of strange problems with it, including smoking another transistor. When it seemed to be basically working, we got it on the vehicle and started testing it with everything else connected. When the motor drive went on, the gyro rate on one axis started drifting fairly quickly, similarly to what happened before we had isolated the power sources. Eventually, we found that if the potentiometer feedback was disconnected, the motor could run without effecting anything. We opened up the motor connector that houses both the three wires for the potentiometer and the two wires for the motor drive, and found that it had been twisted up internally, pulling one of the wires partly out of its mating pin. We have seen this on another connector before, due to someone attempting to disconnect the connector by twisting off the strain relief housing instead of the release collar. The strain relief is clamped to the wires, so twisting that really hard tends to twist or break the wires and pins. I rebuilt the connector, and the motor drive worked perfectly. One of the motor power leads was probably shorting to the potentiometer ground, which would cause the ground noise, and potentially draw huge current, which was almost certainly the death of the previous motor boards.

 

After resolving that, we weighed our pilot in all the safety gear, and made exactly equivalent ballast for the test flight. That required two water jugs, plus the 70 pound heavy bag, which had to be put forward and leaned back over the water jugs to securely strap down.

 

I warmed the attitude engines, then gave it a little throttle to lift off. It took off perfectly straight, but faster than expected. I backed the throttle way down, but it continued to accelerate up. I had no choice but to kill the main engine, or it would have completely flown away.

 

After releasing the joystick trigger, my mind quickly imagined two possibilities – it might fall straight back down and bounce on its landing pads, maybe just bending some tubing, or, it might crash down, tear off all the hose ends, and spray peroxide all over the place.

 

It turned out to be somewhere in between. When the engines are killed, the attitude engines stop immediately, but it takes up to 0.8 seconds to close the central ball valve, so that means that with an offset center of gravity, the closing thrust from the center engine will always wind up rotating the vehicle towards the offset. This made it hit the ground almost upside down.

 

http://media.armadilloaerospace.com/misc/BigCrash.mpg

 

The Damage

 

The damage was extensive, and it was leaking a bit, but not spraying anything.

 

Bent a stainless steel half inch fitting coming out of the tank manifold.

Broke the manifold pressure gauge. We can move one from the small lander, but I have ordered more spares.

Broke off an aluminum fitting coming out of the manifold. Russ is probably going to make a new, flat sided manifold instead of repairing this one.

Bent the motorized ball valve body, but it looks repairable.

Bashed two of the attitude solenoids really hard, but they still seem to work.

Broke one of the attitude motor top plates. Russ is making a new one.

Wrecked the serial joystick, which is an out-of-production model, so I am going to have to configure the flight computer linux install for a modern USB joystick.

The frame is bent in a few places, and the auxiliary mounting plates are trashed. Bob is going to repair or replace it.

Pulled all the cables off the electronics box and broke the mounting brackets as it flung it away. I have most of this fixed already.

In the electronics box, two of the FOG’s came loose from their mounting brackets, just like when the little lander had its flip-crash. Bob is going to make a better mounting plate for these. We haven’t checked to see if all the FOGs still work.

The batteries were secured to the box with epoxied-on angle brackets. The epoxy broke lose, but the brackets still caged the batteries so they didn’t thrash around too much.

The one thing the loose batteries did seem to do is mash the accelerometer cable badly enough to short it out, but I have repaired that.

 

The PC104 stack stripped out the nylon standoffs near the bottom and popped the upper boards loose, making a mess of the bus pins on that board. We had used some combinations of metal standoffs on some of the boards, but we didn’t have exactly the right size, and the PCMCIA board was giving me problems until I went back to the nylon standoffs there. I have ordered some proper metal standoffs from VersaLogic today. Surprisingly, after straightening all the pins and putting it all back together, everything still seems to work.

 

On first surveying everything, it looked like we were going to be out of business for quite some time, but after sorting everything out, it looks like it won’t take all that long to rebuild. The biggest question is the FOGs, which are a six week lead time item.

 

What Happened

 

We have been flying the big lander with the central engine jetted to produce a constant amount of thrust to just “lighten” the vehicle, which allowed it to be flown exactly like the small lander, with the attitude engines providing throttle control as well as attitude control. This was also necessary because the original ball valve motor drive board we were using didn’t have as precise of control for dynamic throttling as I would have liked.

 

The last ballasted flight test we did required nearly full throttle to lift off, and we knew that the full up weight was going to be 15 or 20 pounds heavier that that test, so we increased the size of the main engine metering jet. We tested it on the test stand, and the increase in thrust was what we expected, but we don’t know exactly how much it makes in the vehicle, because the test stand plumbing is more restrictive than the straight shot from the tank that the vehicle has.

 

There was a small effect due to the CG being offset farther from the centerline, which forces the attitude engines to stay on a larger portion of the time, relieving the central engine of some weight.

 

The biggest change we made, that we really should have thought about, was that we were into our new supply of peroxide, which is straight 90%, as opposed to the 98% diluted mix that we were previously making that was only about 85%. That gave us significantly more thrust than we were expecting.

 

So, even with the throttle all the way down, the two attitude engines on the offset CG side staying on a significant amount of the time to keep the platform balanced, in combination with the jetting on the main engine was enough to accelerate the entire 335 pound mess.

 

Just to make me feel bad, after the flight we thought that I might have been able to save the vehicle, or at least keep it upright, by restarting the flight control on the way down. That works out well in the simulator when you just have the attitude engines, but I’m not sure it would have been possible to “manually pulse” the ball valve well enough to get any reasonable lift out of it. It might still have been possible to let the attitude engines keep it upright, but every release of control would have resulted in the same pitch-towards offset CG that the first release had, so it probably would have required only doing that immediately before hitting the ground. Starting the engine again when it is pitched over may have also caused it to accelerate that way (towards us), so maybe it is better I didn’t try it.

 

However, the next day I realized that if I had left the throttle at zero, but continued to hold the trigger, the main ball valve would have closed, but the attitude engines would have continued to keep it upright. It would have landed just as hard, but it would have done it on the foam pads. The attitude engines should continue to keep it level even if it rebounded very high, assuming the initial hard landing didn’t kill the computer. This would have unquestionably been a better action, but when the big, heavy, and expensive vehicle was about to fly above rooftop level, all I was thinking about was “STOP EVERYTHING IMMEDIATELY”. Continuing to hold the trigger for 0.8 seconds while it is flying away would have taken a lot of nerve, because if the ball valve wasn’t closing for some reason, it would have put another ten or twenty feet of drop distance on the vehicle. What probably would have been possible is to release the controls, wait for it to clearly begin coming down, then engage the controls with the throttle at zero, which should level the vehicle without opening the main ball valve at all.

 

An even better approach would have been to start the flight by waiting a full second with the throttle at 1%, just enough to activate the main engine and let it fully open. If it lifted off at all, it would have been easy to immediately drop it. I am used to rapidly throttling up, because most of our configurations need over 50% throttle to get off the ground, and with total hover times of only six seconds or so at this propellant load, it has made sense to get it off the ground fast.

 

We were probably overconfident about our experience with the motor valve configuration. We had a half dozen flights with the big lander (and at least a dozen with the small lander) with just solenoid configurations, but we had only had two flights and a tiny hop with the motor valve configuration before today, and we had changed both the motor drive board (although that wasn’t part of the problem) and the jet size before this flight.

 

Some blame falls towards being stingy with our peroxide. We knew we were at the bottom of our first drum, and we could only take 15 gallons at a time from Rinchem, so we were somewhat reluctant to make multiple back to back runs with many gallons of peroxide at a time. We now have 2000 pounds of peroxide, and we are going to start keeping a drum at our test site, so if we need to go through 200 pounds of peroxide in a test day, we will.

 

We are going to move to the throttle controlling the main motor valve, leaving the attitude engines to do nothing but correct attitude. We were planning on doing this for the next vehicle, but we are bumping the priority. Now that we have our own motor drive board that can respond faster, I will have more confidence in doing it this way. We are going to map the thrust over different valve fractions on the test stand on Tuesday. I still have some concern that the range of interest will only be a small fraction of the ball valve travel, which may make precise control very difficult.

 

When we are ready to consider putting a person on it, we will have an ambulance on site, just like at racing events, and I actually think it will be better to not wear a seatbelt, so jumping off can be done quicker.