- Number 58:
- Number 57:
- Number 56:
- Number 55:
- Number 54:
- Number 53:
- Number 52:
- Number 51:
- Number 50:
- Number 48:
- Number 47:
- Number 46:
- Number 45:
- Number 44:
- Number 43:
- Number 42:
- Volume 1999:
- Volume 3:
- Volume 2:
Navigate to:
| EE371 Home Page | EE Home Page |
------------------------------------------------------------ Embedded Muse 58 Copyright 2000 TGG December 20, 2000 ------------------------------------------------------------ You may redistribute this newsletter for noncommercial purposes. For commercial use contact info(aatt_sign)ganssle.com. EDITOR: Jack Ganssle, jack(aatt_sign)ganssle.com CONTENTS: - Editor's Note - Ford Explorer - Chinook Helicopter - Thought for the Week - About The Embedded Muse Editor's Note ------------- In this issue I present three cases of products that suffered from serious design problems. Not to gloat. Rather, I think we embedded folks need to seriously look at failures in our systems, and find patterns that teach us better ways to produce products. I actively collect embedded disaster stories. There are many lessons one can draw from the collection. Probably the three biggest problems that run through these disasters are: - bad design that isn't caught due to poor inspections - inadequate testing - poor error and exception management So, enjoy the stories but do take the lessons to heart! Ford Explorer ------------- The Ford Explorer has received a certain notoriety due to problems with Firestone tires. The embedded system controlling this car is also a source of problems. In a recent story (December 2000) the Baltimore Sun reports that engine controller's code should limit the vehicle's top speed to 106 miles an hour (wow!). A bug, though, allows the car to exceed 112 MPH. The problem? At that speed... the tires fail! Software upgrades are available to all owners. (see http://www.sunspot.net/content/archive/story?section=archive&pagename=story&storyid=1150520210252) In June of 2000 some Explorers sold in Canada had another problem that causes the "Generic Electronic Module" to crash. This disables the air bags, windshield wipers, lighting, etc. The cure involves installing a resistor, to "prevent electronic noise". A missing pull-up? I see dozens of systems a year where missing pull-up resistors cause erratic operation or crashes. See http://cbc.ca/consumers/market/recalls/reclfull/2000/06jun2000b.html Chinook Helicopter ------------------ A fascinating report describes an audit conducted of the Chinook's engine control software. EDS was contracted to review the code, but actually gave up after looking at less than 20% of the source. According to the report, EDS "abandoned the work because of what one executive said was the density of the anomalies found". In other words, there were so many problems it made little sense to proceed. This review was of released code; software that was in use in many flying Chinooks. They found 485 defects of varying importance. According to EDS the fault levels were an order of magnitude or more than should be expected in "a rigorously developed safety critical system". Unfortunately there's no indication of program size; 485 defects in 10,000 lines of code would be appalling, in 10 million lines perhaps more understandable. When a review board examines source code they often find many defects, as in the Chinook and several recent Mars missions. What strikes me so powerfully is that these boards use the same sorts of tools (like inspections) that proper software engineering should use. Why are we willing to skip proper engineering while building a product, and then resort to it to figure out what went wrong after disaster strikes? See: http://www.computerweekly.com/cwarchive/chinook/cwcontainer.asp?name=CHIN0404.htm Thought for the Week -------------------- The Man From Microsoft There was a knock on the door. It was the man from Microsoft. "Not you again," I said. "Sorry," he said, a little sheepishly. "I guess you know why I'm here." Indeed I did. Microsoft's $300 million campaign to promote the Windows95 operating system was meant to be universally effective, to convince every human being on the planet that Windows 95 was an essential, some would say integral, part of living. Problem was, not everyone had bought it. Specifically, I hadn't bought it. I was the Last Human Being Without Windows 95. And now this little man from Microsoft was at my door, and he wouldn't take no for an answer. "No," I said. "You know I can't take that," he said, pulling out a copy of Windows 95 from a briefcase. "Come on. Just one copy. That's all we ask." "Not interested." I said. "Look, isn't there someone else you can go bother for a while? There's got to be someone else on the planet who doesn't have a copy." "Well, no," The Microsoft man said. "You're the only one." "You can't be serious. Not everyone on the planet has a computer," I said. "Hell, not everyone on the planet has a PC! Some people own Macintoshes, which run their own operating system. And some people who have PCs run OS/2, though I hear that's just a rumor. In short, there are some people who just have no use for Windows 95." The Microsoft man look perplexed. "I'm missing your point," he said. "Use!" I screamed. "Use! Use! Use! Why BUY it, if you can't USE it?" "Well, I don't know anything about this 'use' thing you're going on about," The Microsoft man said. "All I know is that according to our records, everyone else on the planet has a copy." "People without computers?" "Got 'em." "Amazonian Indians?" "We had to get some malaria shots to go in, but yes." "The Amish." "Check." "Oh, come on," I said. "They don't even wear BUTTONS. How did you get them to buy a computer operating system?" "We told them there were actually 95 very small windows in the box," the Microsoft man admitted. "We sort of lied. Which means we are all going to Hell, every single employee of Microsoft." He was somber for a minute, but then perked right up. "But that's not the point!" he said. "The point is, EVERYONE has a copy. Except you." "So what?" I said. "If everyone else jumped off a cliff, would you expect me to do it, too?" "If we spent $300 million advertising it? Absolutely." "No." "Jeez, back to that again," the Microsoft man said. "Hey. I'll tell you what. I'll GIVE you a copy. For free. Just take it and install it on your computer." He waved the box in front of me. "No," I said again. "No offense, pal. But I don't need it. And frankly, your whole advertising blitz has sort of offended me. I mean, it's a computer operating system! Great. Fine. Swell. Whatever. But you guys are advertising it like it creates world peace or something." "It did." "Pardon?" "World peace. It was part of the original design. Really. One button access. Click on it, poof, end to strife and hunger. Simple." "So what happened?" "Well, you know," he said. "It took up a lot of space on the hard drive. We had to decide between it or the Microsoft Network. Anyway, we couldn't figure out how to make a profit off of world peace." "Go away," I said. "I can't," he said. "I'll be killed if I fail." "You have got to be kidding," I said. "Look," the Microsoft man said, "We sold this to the AMISH. The Amish! Right now, they're opening the boxes and figuring out they've been had. We'll be pitchforked if we ever step into Western Pennsylvania again. But we did it. So to have YOU holding out, well, it's embarrassing. It's embarrassing to the company. It's embarrassing to the product. It's embarrassing to BILL." "Bill Gates does not care about me," I said. "He's watching right now," the Microsoft man said. "Borrowed one of those military spy satellites just for the purpose. It's also got one of those high-powered lasers. You close that door on me, zap, I'm a pile of gray ash." "He wouldn't do that," I said, "He might hit that copy of Windows 95 by accident." "Oh, Bill's gotten pretty good with that laser," the Microsoft man said, nervously. "Okay. I wasn't supposed to do this, but you leave me no choice. If you take this copy of Windows 95, we will reward you handsomely. In fact, we'll give you your own Caribbean island! How does Montserrat sound?" "Terrible. There's an active volcano there." "It's only a small one," the Microsoft man said. "Look," I said, "even if you DID convince me to take that copy of Windows 95, what would you do then? You'd have totally saturated the market. That would be it. No new worlds to conquer. What would you do then?" The Microsoft man held up another box and gave it to me. "'Windows 95....For Pets'?!?!?" "There's a LOT of domestic animals out there," he said. I shut the door quickly. There was a surprised yelp, the sound of a laser, and then nothing.
The Embedded Muse ------------------------------------------------------------ Embedded Muse 57 Copyright 2000 TGG December 11, 2000 ------------------------------------------------------------ You may redistribute this newsletter for noncommercial purposes. For commercial use contact info(aatt_sign)ganssle.com. EDITOR: Jack Ganssle, jack(aatt_sign)ganssle.com CONTENTS: - Don't Worry, Be Crappy - Salary Survey - Thought for the Week - About The Embedded Muse Don't Worry, Be Crappy ---------------------- Just what we need - another journalist telling us how bad we developers are. Check out http://salon.com/tech/feature/2000/12/06/bad_computers/index.html, an article titled "High Tech's Missionaries of Sloppiness" by Cheryll Aimée Barron. Though painful, this is a thought-provoking piece on the poor state of software, embedded and otherwise. Ms Barron states: "Many other industries produce high quality products and take full responsibility for their defects. Though commercial aircraft are, like computers, extremely complex hardware and software systems, their makers do not duck responsibility for their flaws." A very valid point. But I fear that too many people fail to realize that quality is not free. That jet airplane comes with a whopping $100 million dollar price tag... or more. Till companies, and ultimately consumers, are willing to pony up the real price for high quality code we're never going to get out of the software quagmire. The avionics software in a plane meets a far higher quality bar than that in your CD player, and so costs orders of magnitudes more. The article compares the abysmal state of software to the American car industry of the 70s, when defect-ridden automobiles were the norm. I'm not sure I agree. We tend to manage software development by a "ship it NOW, fix bugs during customer support" mode, whereas the car industry fostered fixes off on a dealer network. Perhaps a better analogy is one with gasoline. Pump it cheap, sell it in astronomical quantities, but don't factor in the costs of the environmental impact. We'll pay for that later, with other money, somehow, maybe. Software maintenance, like environmental degradation, seems an off-budget expense totally unrelated to the product itself. Ms Barron does correctly identify a problem fostered on us by management and silly consumers: feature-itis. We expect our products to do too many things. Why must every embedded system display the time? Does your cell phone really need a game application? Demand for too many features, with zero time to market, naturally leads to junky applications. Only in electronics do we have such peculiar expectations of our products. If my outboard motor had any sort of decent CPU it would no doubt display speed, compression ratio, and weather forecasts for any of 10,000 cities. An electronic version of the armchair would probably compute your weight and plot the first and second derivative while taking your blood pressure and reminding you to call mom on her birthday. Thanks to reader Eduardo Machuca for passing this URL along. Salary Survey ------------- Check out the November issue of Embedded Systems Programming (http://www.embedded.com/2000/0011/0011sr.htm) for a recent developer salary survey. Interesting data. Thought for the Week -------------------- I finally figured out what to do with all of those AOL CDs. Toss 'em in the ocean. Eventually the entire sea will be covered, reflecting much of the sunlight back to space. This reduces the Earth's albedo, curing the global warming problem! Thanks, AOL!
------------------------------------------------------------ Embedded Muse 56 Copyright 2000 TGG November 28, 2000 ------------------------------------------------------------ You may redistribute this newsletter for noncommercial purposes. For commercial use contact info(aatt_sign)ganssle.com. EDITOR: Jack Ganssle, jack(aatt_sign)ganssle.com CONTENTS: - A Trio of New Embedded Books - Thought for the Week - About The Embedded Muse A Trio of New Embedded Books ---------------------------- Regular readers may know that I've always been fond of John Hart's "Computer Approximations", the mother of all references about creating floating point approximations to trig and other complicated functions. Sadly, it's been out of print for some time, though most decent university libraries will have a copy. Amazon also claims they can find used copies occasionally. Mine's for sale... for $100,000! Jack Crenshaw, venerable columnist for Embedded Systems Programming, has a new book that covers some of Hart's territory. "MATH Toolkit for REAL-TIME Programming" (ISBN 1-929629-09-5) is in many ways a collection of his very popular columns, with some new background material. Crenshaw's book is a very readable description of how one goes about creating an approximation to a function. He covers a few particular functions in detail, like the sine, log, arctangent and square root. Most of us who went through engineering school once knew this stuff, but likely forgot the details immediately after the final exam. I find Crenshaw's explanations infinitely more understandable than that of my profs, and a lot more interesting. I highly recommend this book for anyone trying to remember how curve fitting works, or who needs to create approximations to complex functions. It's not a cookbook like Hart's volume; rather, this is a guide to creating your own approximations. Since every embedded product today seems to require Internet connectivity (toasters? don't laugh - I've seen microwaves and refrigerators connected to the Net), various TCP books have appeared. A new one, "TCP/IP Lean" by Jeremy Bentham (ISBN 1-929629-11-7) is the latest entry, and is specifically targeted at putting net-awareness on embedded applications. The book contains complete source code (on disk as well as in the text) for a TCP/IP stack and web server. Do note that the code is copyrighted and may not be used in your systems. This is understandable, though does defeat our usual need to just find a place to get some code, quickly. Look at this as an educational volume, not a canned solution. I like the fact that the software is structured as a state machine; it's a nice real world example of using a state machine in a complicated application. Also nice is that the author presents code for both a 80188 CPU and a PIC 16C76, so he covers both the middle and low end range of embedded processors. The book includes good explanations of the various protocols involved. Ever wonder what ARP is all about? Bentham explains these important concepts in detail and quite understandably. Alan Bensky's new book, "Short-Range Wireless Communication" (ISBN 1-878707-53-1), is a timely introduction to sending data over radio links. This is not a volume for the fainthearted; it covers RF with plenty of theory and equations. It's good stuff if you need to know how this all works, but count on being challenged by the math if hardware is not your thing. One shortcoming is the lack of Bluetooth coverage; the single-page description isn't of much use to people dealing with Bluetooth today. It's important to note that this volume doesn't include code or much in the way of hardware design. It's more a background for hardware engineers. There is a chapter on commercial chips and boards you can buy to do the RF portion of your link, though. I worry that this data could soon become obsolete, but as of now it's quite useful. Possibly the most useful chapter covers regulations for RF operation in different countries. Fact is, no one dares transmit anything without conforming to the rules. And, I found the appendix on Information Theory quite interesting. But for RF theory I much prefer "The ARRL Handbook for Radio Amateurs" (http://www.arrl.org/catalog/?category=Reference). I've got the 1970 edition, which covers transistor and vacuum tube designs. Page for page it's the most useful radio book I know. Where Bentham's book contains code that implements TCP/IP, and Crenshaw's book shows you how to create code to fit your particular algorithm need, Bensky's book is much more detailed background to RF. Thought for the Week -------------------- OK - this one is NOT a joke. In my continuing search for the silliest embedded system I ran across a smart coffee scoop at a Maryland Mall. A computer controlled spoon? How much intelligence do my utensils need?
------------------------------------------------------------ Embedded Muse 55 Copyright 2000 TGG November 8, 2000 ------------------------------------------------------------ You may redistribute this newsletter for noncommercial purposes. For commercial use contact info(aatt_sign)ganssle.com. EDITOR: Jack Ganssle, jack(aatt_sign)ganssle.com CONTENTS: - Tires That Are Too Smart - More On Measuring Latency - Upcoming Embedded Seminar - Thought for the Week - About The Embedded Muse Tires That Are Too Smart ------------------------ From: http://dailynews.yahoo.com/h/nm/20001023/tc/tires_phones_dc_1.html Reuters reports that in Finland work is afoot to embed chips inside of your car's tires. When the pressure falls too low they'll call you on your cell phone. Technology run amok? Can I block calls from my tires? I'll have to get call waiting in case the tires want to break in on another conversation! More on Measuring Latency ------------------------- Gary Bergstrom wrote: The other very interesting thing one could do with your counter is to log the time when the ISR finished. e.g. Read the hardware timer as the last thing you do. This catches overflows, and sometimes the higher priority interrupt that interrupts the current interrupt. And Alf Katz commented: Of course, you're right, but of much more interest to most embedded programmers is the longest system latency. This can easily be found by keeping the highest overrun reached in another variable, and updating only when a higher one is found. Of course you could go overboard & collect means & standard deviations of the latency time, but you wouldn't want to do it in an interrupt - and the longest latency is *usually* the most important metric. Finally, from Dean TerHaar: I use a similar approach to time the duration of an ISR. I define a struct of two variables: TimerIn and TimerOut, and then declare a global array of this struct. On entry to the ISR I read the system timer and store it in Array[i].TimerIn and on exit from the ISR I read the system timer again and store it in Array[i].TimerOut. I also have a control variable that I can set dynamically to enable/disable this ISR duration monitor. After I execute a sample run of the program (in real time), I'm use these timer values to calculate the min, max, and average durations of the ISR and display a report. (without the aid of a debugger or oscilloscope!) The routine that calculates the ISR time also calculates the overhead of the duration test (i.e. the time it takes to read the system timer and store its value). It subtracts this overhead (or bias) from the calculated ISR time to compensate for the additional time it takes to perform the test. This routine also normalizes the result in microseconds and handles cases where the system timer rolled over during the execution of the ISR under test. This test, as it is written in C, however, does not account for the overhead of the ISR under test (working register pushes and pops), but this fixed time could be calculated manually and included into the final result returned by the routine that calculates the ISR execution time. Embedded Seminar in Chicago --------------------------- I'll present the seminar "The Best Ideas for Developing Better Firmware Faster" in Chicago on November 15. The focus is uniquely on embedded systems. I'll talk about ways to link the hardware and software, to identify and stamp out bugs, to manage risk, and to meet impossible deadlines. If you're interested reserve early as these seminars fill completely. For more information check out http://www.ganssle.com or email mailto:info(aatt_sign)ganssle.com. A lot of folks have asked me to bring this seminar to their company. Email me at mailto:jack(aatt_sign)ganssle.com if you're interested. Thought for the Week -------------------- A tourist walked into a pet shop and was looking at the animals on display. While he was there, another customer walked in and said to the shop keeper, "I'll have a C monkey please". The shopkeeper nodded, went over to a cage at the side of the shop and took out a monkey. He fit a collar and leash, handed it to the customer, saying, "That'll be $5000." The customer paid and walked out with his monkey. Startled, the tourist went over to the shopkeeper and said, "That was a very expensive monkey. Most of them are only a few hundred dollars. Why did it cost so much?" The shopkeeper answered, "Ah, that monkey can program in C - very fast, tight code, no bugs, well worth the money." The tourist looked at the monkey in another cage. "That one's even more expensive - $10,000! What does it do?" "Oh, that one's a C++ monkey; it can manage object-oriented programming, Visual C++, even some Java. All the really useful stuff," said the shopkeeper. The tourist looked around for a little longer and saw a third monkey in a cage of its own. The price tag around its neck read $50,000. He gasped to the shopkeeper, "That one costs more than all the others put together! What on earth does it do?" The shopkeeper replied, "Well, I haven't actually seen it do anything, but it says it's a consultant."
------------------------------------------------------------ Embedded Muse 54 Copyright 2000 TGG October 30, 2000 ------------------------------------------------------------ You may redistribute this newsletter for noncommercial purposes. For commercial use contact info(aatt_sign)ganssle.com. EDITOR: Jack Ganssle, jack(aatt_sign)ganssle.com CONTENTS: - Measuring Latency - Upcoming Embedded Seminar - Thought for the Week - About The Embedded Muse Measuring Latency ----------------- CPU vendors define interrupt latency in terms of the longest non-interruptible instruction. I figure this is a meaningless definition; after all, what we developers really care about is the time between when the interrupt occurs and when the ISR (Interrupt Service Routine) starts. Usually the delays are more a factor of how we write the firmware than raw processor capabilities. Code that spends lots of time with interrupts disabled will have long latencies. There are a number of ways to measure this, but here's a novel and rather cool approach. You'll need a spare timer and just a bit of code. Start the timer counting up, and set it to interrupt when the count overflows to zero. The timer ISR should be dead simple with as little overhead as possible; it's best coded in assembly language so you can minimize the number of register saves (so many compilers push *everything* inside C-coded ISRs). Immediately read the timer count register and sum this value into a long variable. Increment a counter. Then clean up and return. Note that, though the timer read zero when it issued the interrupt, it continues to count before the ISR starts. The value we read inside the ISR is the system's latency (less some ISR overhead). Figure the raw overhead of this ISR by counting T-states (instruction times) before the timer read happens. Now just run your application for a while, and then stop the program with a debugger. Average system latency is the long variable (normalized to microseconds), divided by the number of iterations of the ISR, with the ISR overhead removed. Run this experiment on MS-DOS; you may see latencies ranging into the tens of milliseconds! Embedded Seminar in Chicago --------------------------- I'll present the seminar "The Best Ideas for Developing Better Firmware Faster" in Chicago on November 15. The focus is uniquely on embedded systems. I'll talk about ways to link the hardware and software, to identify and stamp out bugs, to manage risk, and to meet impossible deadlines. If you're interested reserve early as these seminars fill completely. For more information check out http://www.ganssle.com or email mailto:info(aatt_sign)ganssle.com. A lot of folks have asked me to bring this seminar to their company. Email me at mailto:jack(aatt_sign)ganssle.com if you're interested. Thought for the Week -------------------- Your friend might be a hacker if.......... -Everyone who ticks him or her off gets a $26,000 phone bill. -Has won the Publisher's Clearing House Sweepstakes three years running. -When asked for their phone number, they give it in hex. -Seems strangely calm whenever the office LAN goes down. -Somehow gets HBO on their PC at work. -Mumbled, "Oh, puh-leeeez!" 295 times during the movie "The Net." -Massive 401k contribution made in half-cent increments. -Their video dating profile lists "public-key encryption" among turn-ons. -Instead of the "Welcome" voice on AOL, you overhear, "Good Morning, Mr./Mrs. President." -You hear them murmur, "Let's see you use that VISA card now, Professor "I-Don't-Give-A's-In-Computer-Science!"
The Embedded Muse ------------------------------------------------------------ Embedded Muse 53 Copyright 2000 TGG October 23, 2000 ------------------------------------------------------------ You may redistribute this newsletter for noncommercial purposes. For commercial use contact info(aatt_sign)ganssle.com. EDITOR: Jack Ganssle, jack(aatt_sign)ganssle.com CONTENTS: - The Right Stuff - Upcoming Embedded Seminar - Thought for the Week - About The Embedded Muse The Right Stuff --------------- FastCompany, a magazine aimed at business folks needing help coping with the "new economy" has a fascinating article about writing and maintaining the Space Shuttle software. In "They Write the Right Stuff" author Charles Fishman delves into how Lockheed Martin creates what is probably the best software ever written. Unfortunately, sometimes people joke about the Shuttle code, since failures are so dramatic and public. But each of the last three versions - each 420,000 lines of code - had but a single error. Compare that to other organizations. The Software Engineering Institute (http://www.sei.cmu.edu) estimates that typical code contains about 60 errors per 1000 lines. The very best organizations, those conforming to Capability Maturity Model level 5 (there are only about 20 outfits in the world certified to level 5) ship around one defect per 1000 lines. Yet these Lockheed Martin folks somehow approach an error rate of 2 parts per million! In fact, the Capability Maturity Model stems from many of the practices employed by the Shuttle group. The FastCompany article emphasizes how different the Lockheed Martin environment is from conventional just-ship-the-damn-thing software groups. Developers go home at 5. The specification phase is excruciating... but accurate. I find one comment quite interesting: developers all have private offices. DeMarco and Lister showed long ago that interruptions kill productivity and encourage bugs. They found a 3:1 difference in performance between groups, all of which they attributed to interruptions. Other studies show that it takes 15 minutes to enter "a state of flow", where you're one with the computer. Yet the average developer is interrupted every 11 minutes! The full text of the article is at http://www.fastcompany.com/online/06/writestuff.html. Highly recommended. Embedded Seminar in Chicago --------------------------- I'll present the seminar "The Best Ideas for Developing Better Firmware Faster" in Chicago on November 15. The focus is uniquely on embedded systems. I'll talk about ways to link the hardware and software, to identify and stamp out bugs, to manage risk, and to meet impossible deadlines. If you're interested reserve early as these seminars fill completely. For more information check out http://www.ganssle.com or email mailto:info(aatt_sign)ganssle.com. A lot of folks have asked me to bring this seminar to their company. Email me at mailto:jack(aatt_sign)ganssle.com if you're interested. Thought for the Week -------------------- NEWS FLASH! SEVEN SOFTWARE COMPANIES ADDED TO "WATCH LIST" New York - People for Ethical Treatment of Software (PETS) announced today that seven more software companies have been added to the group's "watch list" of companies that regularly practice software testing. "There is no need for software to be mistreated in this way so that companies like these can market new products," said Ken Grandola, spokesperson for PETS. "Alternative methods of testing these products are available." According to PETS, these companies force software to undergo lengthy and arduous tests, often without rest, for hours or days at time. Employees are assigned to "break" the software by any means necessary, and inside sources report that they often joke about "torturing" the software. "It's no joke," said Grandola. "Innocent programs, from the day they are compiled, are cooped up in tiny rooms and "crashed" for hours on end. They spend their whole lives on dirty, ill-maintained computers, and are unceremoniously deleted when they're not needed anymore." Grandola said the software is kept in unsanitary conditions and is infested with bugs. "We know that alternatives to this horror exist," he said, citing industry giant Microsoft Corporation as a company that has become successful without resorting to software testing.
------------------------------------------------------------ Embedded Muse 52 Copyright 2000 TGG October 4, 2000 ------------------------------------------------------------ You may redistribute this newsletter for noncommercial purposes. For commercial use contact info(aatt_sign)ganssle.com. EDITOR: Jack Ganssle, jack(aatt_sign)ganssle.com CONTENTS: - MISRA Review - Thought for the Week - About The Embedded Muse MISRA Review ------------ Frequent contributors to the comp.arch.embedded newsgroup sometimes refer to the MISRA (Motor Industry Software Reliability Association) publication "Guidelines For the Use of The C Language in Vehicle Based Software". As one interested in the firmware reliability (is that an oxymoron?) I wanted to check out this publication, but was frustrated by its unavailability on the net. So I ordered a copy from England (35 pounds for overseas shipments) through the web site (http://www.misra.org.uk). In just a few weeks the 70 page bound booklet arrived. It's emphatically NOT a software standard; rather, the authors define safe ways to use some C constructs and identify others that must be avoided. Use these guidelines in concert with a real standard, one that defines coding styles, commenting conventions, and the like (you're welcome to download the one I use from http://www.ganssle.com/misc/fsm.doc). While C is indeed a very powerful language, it should come with a warning label: "danger: experts only". It's so easy to create programs that leak memory, run pointers wildly all over memory, or create other difficult-to-find havoc. The MISRA standard, a collection of 127 coding rules, tries to prevent problems by limiting the types of C constructs we use, and defining safe ways to use others. Quite a few of the MISRA rules make tremendous sense: don't redefine reserved words and standard library function names. Document and explain all uses of #pragma. When a function may return an error, always test for that error. Functions should have a single exit point. Some are interesting: never use recursion. Keep pointer indirection to no more than two levels. A couple are hard but possibly quite valuable: check every value passed to every library routine. Avoid many common library functions. Other are trivial: only use characters defined by the ISO C standard. Don't nest comments. Write code conforming to ANSI C. Don't confuse logical and bitwise operators. Don't have unreachable code. Some of the requirements I find disturbing. For instance, rule 118 prohibits the use of dynamic memory allocation. Not a bad idea, due to problems associated with fragmentation. But there are alternatives to malloc/free that still give us the benefits of dynamic memory allocation without the pitfalls. More problematic, this rule tells us not to use library functions which employ dynamic memory, specifically mentioning string.h. This seems awfully restrictive to me... I sure don't want to write my own string handlers... and further, how is one to identify the suspect libraries? Rule 122 prohibits the use of setjmp and longjmp. These are worse than gotos, of course, in that they let us branch to specific memory addresses. Yet in a few cases longjmp is almost unavoidable. I think there's much value to the document, but as a stand-alone set of rules it's incomplete. Better, incorporate the rules into your in-house software standard. It's just too hard to conform to two sets of rules living in two different documents. If MISRA published the rules on-line, they'd be more accessible to the embedded community, hopefully improving the quality of code everywhere. Without such an electronic copy, I doubt if many will ever incorporate these rules into their own standards. Thought for the Week -------------------- In the 80s: Recruiter: "Tell me the meanings of all the extensions to file names" Candidate: "VMS or MS-DOS?" Recruiter: "You got the job!" In the 90s: Recruiter: "Tell me all the options used by /bin/ls" Candidate: "BSD options or System V?" Recruiter: "You got the job!" In the 00s: Candidate: "Tell me all the options" Recruiter: "20000, 1 year vesting, $1 strike price, IPO next month" Candidate: "I'll take the job!"
------------------------------------------------------------
Embedded Muse 51 Copyright 2000 TGG August 16, 2000
------------------------------------------------------------
You may redistribute this newsletter for noncommercial purposes. For
commercial use contact info(aatt_sign)ganssle.com.
EDITOR: Jack Ganssle, jack(aatt_sign)ganssle.com
CONTENTS:
- Contest Results
- About The Embedded Muse
Contest Results
---------------
Congratulations to Natalia Ovtchinnikova who won the random pick of contest
entries for the Write Only Memory. Natalia wins the original datasheet plus
one of the parts themselves.
About a thousand people entered, with half sending ideas for using the
part. Some of the most common entries included a logger for spam, various
suggestions for saving promises made by Bill Clinton and other politicians,
quite a few thoughts about error loggers for Microsoft products, various
mother-in-law ideas, and, of course, management report repositories.
Here's a few of the entries. Some are pretty funny. And, thanks to everyone
who participated!
A device for maintaining a 'To-Do' list for procrastinators.
I would like to use WOM for the storage device in a telephone answering
machine. I never really listen to the messages anyway. Or combine this
with a voice detection circuit to detect my mother-in-law's
voice. Everyone else gets stored in conventional memory and only hers goes
to the WOM
As a backup device for the one I currently use to store the last location
of my car keys.
In military rocketry apps, our pilots are comfortable with just "Shoot and
Forget". This seems like the ideal chip for my "write and never worry"
embedded project
Archival copy of the wedding vows shared with my wife (who divorced me for
spending too much time working on embedded systems).
Personal Organizer for teenage girls when they want to store the phone
number of potential dates. (This, of course, will be a gift from Father to
his daughters)
This memory device would be perfect for the new processor architecture I'm
developing. It's the next generation for RISC processors - called NISC (No
Instruction Set Computing). Forget single or even partial cycle
instructions, this has zero cycle instructions.
Entering confessions of not testing all possible paths through embedded code
It can be very helpful for storing the Imaginary portion of any complex
number. Since it's imaginary, it doesn't really matter.
Someplace to send the data when you flush the cache memory!
Logic gates are designed to lose a bit of information, and this is
wasteful. For example, an NAND gate has two inputs, but only one output.
One of the bits gets lost, and is usually dissipated as heat. This is why
microprocessors tend to get hot. It would be ideal if that extra bit can be
preserved, and rerouted to the WOM, rather than "destroyed". The WOM can
then be disposed of properly when it can no longer hold those stray bits.
Then faster microprocessors can be used which remain cool.
I would use it to store a database of the dericidous barachitides derived
from the antithaflactic bromides used in the fabrication of xialaniplates
A data logger for my fridge. As far as I'm concerned, I don't *want* to
know what goes on in there
Sometimes there is a secret that is so personal that you cannot tell
anyone, but at the same time not being able to talk about it drives you to
the insane asylum. So the So embed the write-only memory inside a stuffed
puppy dog, and you get the satisfaction of knowing (1) you will be heard;
(2) you will not be misinterpreted; and (3) your secret is safe with the
stuffed puppy dog. No amount of torture or pleading will get the secret out
Digital voice recorder for a church confessional
A digital version of the Shroedigner's Cat experiment. Write a string to
the chip--either "dead" or "alive"--to be randomly determined at the last
moment. We don't know whether the cat is alive or dead unless we actually
open up the chip
Loop it back to my wife's mouth and watch it explode
Voicemail storage device for Tech Support departments ("...your call is
important to us
Storing corporate Mission, Vision, Value statements
To store all those monthly reports that I am required to write
A write only memory sounds like a great place to store cookies. All you'd
need is a device driver to make it look like a logical drive in your
favorite OS
To store the boss's suggestions, memos, and any code he creates
Deep storage device for Mars probes produced by NASA
Engine Control Module - so mother-in-law can't drive to my house
Record Windows 2000 bug reports (bugs for previous versions never get
fixed, so they must be going into WOM. Did you know Microsoft is the single
largest purchaser of WOMs?).
I see the Write Only Memory as a sort of electronic wishing well. It
represents the ultimate secure communications with God
I would use it for storage of impractical suggestions from pre-teenage
daughters
The Marketing Department Feature Request Recorder! Yes, now every R&D
department can have it's own MD-FRR featuring the Signetics WOM!
Place where the universe stores dark matter
Hardware accelerator for /dev/null
Storage of voicemail from telemarketers
Clinton White House e-mail storage
An automated ISO-9000 audit trail recorder. After all, the output doesn't
matter as long as you follow the process
For creating a neural network to mimic a child's brain in response to
parental advice
Recording promises made to employees
Memory for an absent-minded artificial intelligent
A PDA marketed to the mob
Microsoft should not be broken up. Rather they should be required to store
all of there code on WOM's. This will solve both their code bloat and bug
control problems, while increasing competition in the software industry. In
addition it can only help to improve their time to market delays.
A paperless fax shredder. A standard fax shredder consist of a fax machine
coupled to a shredder -- if you need to shred a document you fax it to the
right number. But, unfortunately, this process uses paper. By using a WOM
for storing the incoming data stream inside the fax machine, no paper will
be required. Think of the trees!
Keep track of all the suggestions for, "What is the ideal use for a Write
Only Memory?"
------------------------------------------------------------ Embedded Muse 50 Copyright 2000 TGG August 1, 2000 ------------------------------------------------------------ You may redistribute this newsletter for noncommercial purposes. For commercial use contact info(aatt_sign)ganssle.com. EDITOR: Jack Ganssle, jack(aatt_sign)ganssle.com CONTENTS: - Editor's Notes - Embedded Failures - Contest! - Thought for the Week - About The Embedded Muse Editor's Notes -------------- Many folks have written asking about the status of The Embedded Muse, as the last issue was a couple of months ago. It's just a case of holidays; each summer my real life goes on hiatus for June and July in favor of sailing. Alas, it's back to the grindstone now, so the Muse is back on-line. The Write Only Memory contest (see info later in this email) closes in two weeks. Many of you have submitted entries, some quite funny. Stay tuned next issue for the winner. Embedded Failures ----------------- On March 12 Sea Launch attempted to boost a communications satellite into orbit from their ocean-based platform, and failed. A single line of code was in error, which allowed the vehicle to blast off with an open second stage pneumatic valve. Their two earlier launches were successful, as was one this week. So here's an example of a hideously-expensive failure of an embedded system, one where the firmware was at fault. I have no other details of the failure, but ask any reader who has information to send it to me at mailto:jack(aatt_sign)ganssle.com. The firmware content of all products is skyrocketing, yet "reliable code" seems to be an oxymoron. Surely we'll see a breathtaking surge in the number of embedded system failures. Some will be of little consequence, perhaps just annoying our customers. Others will threaten life and limb. Did you know that Ford recalled some Explorers on June 6, 2000 because air bags, windshield wipers and other accessories could fail? The culprit: a missing pull-up resistor on the computer module. Surely we can learn from these disasters if we develop a lore of catastrophe. For instance, what does the Los Alamos near-nuke failure of 1998 and Ariane 5's maiden flight explosion have in common? Poor error handling. And error handling is a huge problem for all software, as it's dreadfully hard to anticipate and test all error possibilities. So I'm collecting stories of embedded disasters, to share with readers. If you know of a disaster - and can validate it - please send me information or references. Just as civil engineers learn from stories of collapsing bridges and skywalks, we can - and should - tune into these problems in our own field. Contest! -------- When Signetics created their Write Only Memory part as a joke a quarter century ago, they probably had no idea so many people would be so entranced. I have a copy of the part's original datasheet, as well as an actual part (marked "NFG" in case there's any doubt). Just for fun, here's a contest with the datasheet and the Write Only Memory chip as grand prize. Come to http://www.ganssle.com, select the link to the contest, enter your email address, and your name will be tossed into the ring to win the prize. If you'd like, also enter in a suggestion for what you'd use a write-once, read-never part for. The winner will be announced in this newsletter, and some of the funnier uses for the part will run as well. Thought for the Week -------------------- To the optimist, the glass is half full. To the pessimist, the glass is half empty. To the engineer, the glass is twice as big as it needs to be.
Embedded Muse 48 Copyright 2000 TGG April 10, 2000 ------------------------------------------------------------ You may redistribute this newsletter for noncommercial purposes. For commercial use contact info(aatt_sign)ganssle.com. EDITOR: Jack Ganssle, jack(aatt_sign)ganssle.com CONTENTS: - Newbie Books - Upcoming Embedded Seminars - Thought for the Week - About The Embedded Muse NOTICE - The Boston and San Jose Embedded Seminar on April 26 and May 3 are booking fast - see http://www.ganssle.com for more info. Newbie Books ------------ "C Programming for Embedded Systems" by Kirk Zurell was published a few weeks ago (ISBN 1-929629-04-4, R&D Publications, $29.95). It's a general introduction to working in C for the complete embedded novice. This tome will not teach you C, but will show someone with limited C experience how to build an embedded system. Though Zurell makes no attempt to teach C, he wisely does delve into data types and variables, showing how types have real physical implementations (ROM, RAM, etc) and the implications of each. The book's focus is entirely on small 8 bit systems. Zurell works for Byte Craft, a Canadian compiler company. Not surprisingly, the book addresses processors Byte Craft supports, like the 6805, 6808 and PIC. Also not surprisingly, examples revolve around Byte Craft's tools. Few 8 bit compilers are really ANSI-compliant; the quirks of these small chips mandate various extensions to the language to make C possible and desirable. So some of the keywords used in the book's examples are Byte Craft-specific and non-portable to other compilers. This does not in any way limit the usefulness of the code snippets or prose discussion. It comes with a CD that includes a complete (Byte Craft) development system for the 6805. I'm always looking for books for fledgling embedded systems engineers. Fact is, it's tough to make the transition from general programmer to embedded developer. This is a good introductory work that will give potential engineers a basic understanding of many of the issues in using C in an embedded app. Working with Byte Craft tools? If so, this is an essential volume for your new engineers. Another good book for newbies is Michael Barr's "Programming Embedded Systems in C and C++" (ISBN 1-56592-354-5, O'Reilly & Associates 1999, $29.95). Barr's book targets more general environments than Zurell's. C++, after all, is hardly 8 bit friendly! If you're working in the 16/32 bit world, especially, Barr's book is a handy way to get new folks up to speed. He does a good job of covering uniquely embedded issues like generating startup code. His examples use the 188/186 processor, which is a much more ideal C machine than many 8 bitters. But, as an embedded chip, it carries all of the baggage we've got to deal with in building firmware in a high level language, like talking to peripherals. Barr covers more complex memory issues, such as talking to Flash and using DMA. Each of these works are good references for folks trying to figure out how to use C in an embedded application. I'd pick Barr's "Programming Embedded Systems in C and C++" if the target environment is 16 bits or bigger, and Zurell's "C Programming for Embedded Systems" for 8 bit systems.
------------------------------------------------------------
Embedded Muse 47 Copyright 2000 TGG March 22, 2000
------------------------------------------------------------
You may redistribute this newsletter for noncommercial purposes. For
commercial use contact info(aatt_sign)ganssle.com.
EDITOR: Jack Ganssle, jack(aatt_sign)ganssle.com
CONTENTS:
- Conspiracy Theory, Take 2
- Upcoming Embedded Seminars
- Thought for the Week
- About The Embedded Muse
Conspiracy Theory, Take 2
-------------------------
Last issue I wrote about RTOSes, and how (in my opinion!) they are a very
powerful tool for building firmware. This prompted an astonishing number of
very interesting responses, some of which I'd like to share with you.
But first - some readers got the impression I think an RTOS is the silver
bullet required for all embedded apps. No way. This industry is completely
fragmented, so it's impossible to generalize about anything, generally. An
RTOS offers some serious benefits for a lot of applications, but is totally
inappropriate for many. One reader wrote about a microprocessor-controlled
soldering iron. I can't imagine how an RTOS would benefit something so simple!
Further, commercial RTOSes address (again, in my opinion) a wide range of
RTOS needs. But there's little question that most share the same few
scheduling and other mechanisms. Some systems require very different
techniques, which may not be addressed by off-the-shelf products.
The responses were about 50% pro- and 50% anti- coding your own RTOS. In
the pro-custom-RTOS camp, the most common reason was mistrust of vendors.
Developers worry that buying code means buying bugs, bugs that they'll be
left to sweat out at 2AM. A custom RTOS means you have the source, and you
know how the beastie works, so bugs, if they appear, can be swatted
effectively.
I hope the commercial RTOS people are reading and learning...
So, on with the letters. And thanks much to everyone who wrote in.
Michael Purcell wrote:
Your remarks in the latest Muse regarding buying an RTOS are 100% on the
mark. The "conspiracy" comment is an example of a violation of something I
call "Evans' Law":
Just because you *can* do something
does not mean you *should*!
(Named after a fellow software engineer who used the axiom to challenge
us when we were about to do something stupid.)
Brian Trial commented:
Well good god yes! Labrosse's OS supports preemptive multitasking for 63
tasks, floating point math, in a C++ environment. How many embedded
applications need this much horsepower? I'd never even think of tackling
those requirements without a commercial
RTOS.
How many embedded applications are written in assembler or plain Jane C, on
an 8 bit microcontroller, with fixed point math and no requirement for
complex multitasking? I'll bet it's more than 2%, and any competent
software engineer could certainly construct their own OS in this
environment far cheaper than any commercial RTOS, and tailored to their
specific needs.
We recently completed a project using the Motorola HC12. Our group
found 3 bugs in the C compiler we used, which ate up a lot of our time
tracking down. I shudder to think if we had to debug someone's buggy RTOS
as well! The OS we wrote is simple and streamlined for our needs, and
didn't take near a $60K effort or 4Klocs of C you mention.
Now, our next project will probably require a more complex 32 bit micro
with more complex tasks to perform, so naturally we're considering an RTOS
for that. But I'd say closer to 2% than 70% of the embedded world needs a
32 bit processor.
Sid King wrote:
We have gone through the agony of starting with Jean Labrosse's port of
UCOS (A great little OS I should add) for the XA from Philips and
encountered many challenges just getting that working correctly. (The
original UCOS Hi-tech 'C' port worked fine, using a new compiler was the
challenge.) We probably invested over $50K in getting UCOS working with
the Tasking Compiler. I might also add that we spent part of that $50K
making the UCOS work with the CrossView debugger from Tasking, we had to do
some rewriting of the monitor code to make it function with the OS. Our
next project will be using the UCOS port for the XA that we did, as well as
buying ThreadX for use on the Coldfire processor in that system as well.
We will be using the GreenHills tool set for that. It will be interesting
to see if ThreadX is much easier to work with since no porting should be
required.
Another issue that should be addressed in selecting the RTOS, is making
certain that the compiler/debugger and RTOS vendor work closely to make the
port as seamless as possible. We are banking on that with the
GreenHills/ThreadX development combination.
Brian Lim thought:
On your subject of why projects use custom operating systems, I offer my
thoughts on factors that lead to this outcome.
1. Many people get jobs writing software because they enjoy writing
software. The more they write, the more enjoyment.
2. While marketing people are responsible for achieving maximal profit from
minimal expenditure, they often don't see to it that engineering has those
same aims.
3. Normally you need to get people in an organization to sign a purchase
requisition for things that you want to spend money on. You would have to
justify the cost. You probably don't need signed purchase requisitions for
bloated feature sets. A custom OS is part of the same problem.
4. There's risk involved in selecting and buying an OS. There's risk
involved even if the OS is free of charge. Some engineers may be
excessively risk averse in this area.
5. As with everything, it's mostly management's fault
Mark Bereit wrote:
This isn't an argument, necessarily, but my personal explanation for why
I'm in the code-your-own-RTOS camp. (I'll bet I'm not the only one of
these you get!)
First, a story. When wearing my programmer hat I frequently put structures
in a linked list.
Aha, says the School of Code Reuse. A simple thing which should be coded
once, and used endlessly without fussing the details. After all, each time
you recode the routines you introduce the chance for a mistake, different
naming conventions and style conventions which make no difference but have
to be maintained, etc. etc. All very sound reasoning.
So how should I reuse a linked list? In C++ I could make a class which is
just a linked list implementation, and then derive things from it. Oh,
wait, that won't work for an object derived from something existing (like a
Microsoft MFC class) because now it's multiple inheritance and that often
breaks things (assuming you have multiple inheritance in your language in
the first place) and complicates pointers and adds overhead. I can use a
template... well, that confuses things similarly. And neither derivation
works in C. I can make a linked list structure which is the first member
of the larger structure, but this similarly breaks complex derivations and
requires unsafe recasts. I can make a bunch of macros to do define the
extra members and to do the adds, removes, etc.... although I lose all type
safety with macros and when there's a bug the macro obscures what you did
wrong. I can inline function some of it and macro some of it... but I
still need to remember to do all the pieces...
Suppose I pick one of these half-complete solutions. Now, what about the
head pointer? Is it a global, or a member of something else? How does
that impact the implementation? Did my approach allow the maintenance of
this list to be a member of a class, or is it required to be a global so I
have to rewrite to make it fit? Who does cleanup? The owner of the
pointer, but what makes THAT automatic? Should the owner of the head
pointer do the cleanup? Should I have required a dummy first node, like I
learned in school? What if the nodes are large and resource intensive?
How well does this solution fit ME?
And so I have learned over the years to just Code The Stupid Thing. Every
time. By now I rarely get it wrong the first try.
So we come back to the RTOS. There are plenty of choices. I see no reason
to hate any of the options available, and if one were handed to me as a
design requirement I would have no problem with starting there. But again,
if I had to choose one, I would be seriously gridlocked in evaluating what
is out there and what works well with my tool chain and my expectations and
my project requirements and so on. And whatever was chosen, I would then
proceed to use where possible and circumvent where insufficient... meaning
that again, the code would only be partial reuse and partial replacement.
On some of my projects the dispatch mechanism is a while(1) loop which does
the highest priority thing, then the next highest, then the next highest,
etc., in each case repeating the loop without touching lower priority tasks
if there is more to do on that priority. No reentrancy, no ugliness, and
this can be interrupt-based or completely polled depending upon hardware.
So I invariably divide all RTOS needs into two categories: those where what
I just described is adequate, and those where it is not. For those where
it is adequate I will code just that: I don't need threads,
synchronization, or anything! What RTOS did I just use? Unless "none" is
an option on the survey--pretty rare--my answer must be "custom made."
Nothing else was even a consideration... because NO RTOS WAS NECESSARY.
Now in the other half, it comes down to, how much RTOS do I need? I find
that a preemptive task switcher with critical sections is pretty easy to
write for any platform, so that need won't push me to buy. If I need
complex synchronization objects, that may call for outside help. If I need
a TCP/IP stack, that definitely calls for outside help. (My current
project uses USNET from US Software, with which we are pretty happy... but
we didn't buy their RTOS, we bolted USNET quite nicely onto our own task
switcher.) If I need thread-safe resource allocation, especially memory
allocation, then if that isn't part of my C Runtime then I need outside help.
But when the need for outside help isn't strong, then I find that I am
weighing the energy of coding and debugging my own code (for which I am
being paid to accomplish) against the energy of researching and learning
and debugging someone else's code (for which I additionally have to request
an expenditure beyond existing compensation). So I won't buy an RTOS until
the energy expended tradeoff makes sense.
Although I am not opposed to doing so, I have never yet bought an RTOS. I
have bought a TCP/IP stack, and expect to buy an SNMP implementation in the
future, because there is too much code that is new to me: I would be
writing and debugging for too long to justify that tradeoff.
How much of that 70% of custom-made OSes represent the trivial dispatch
loop approach I couldn't say. It represents almost 50% of projects I've
worked on, though! I suspect it is only the RTOSes that offer a lot more
than simple task switching who have any appreciable following, and then
only for the projects where the needs are a lot greater than that.
Bottom line: if I can build and debug it in less than a week I am unlikely
to buy it. Because that's how long I expect to spend learning, adapting
and debugging code I buy from someone else.
Thought for the Week
--------------------
From Paul Walker
"Source Code to Windows 2000"
/* Source Code to Windows 2000 */
#include "win31.h"
#include "win95.h"
#include "win98.h"
#include "workst~1.h"
#include "evenmore.h"
#include "oldstuff.h"
#include "billrulz.h"
#include "monopoly.h"
#define INSTALL = HARD
char make_prog_look_big[1600000];
void main()
{
while(!CRASHED)
{
display_copyright_message();
display_bill_rules_message();
do_nothing_loop();
if (first_time_installation)
{
make_50_megabyte_swapfile();
do_nothing_loop();
totally_screw_up_HPFS_file_system();
search_and_destroy_the_rest_of_OS/2();
make_futile_attempt_to_damage_Linux();
disable_Netscape();
disable_RealPlayer();
disable_Lotus_Products();
hang_system();
}
write_something(anything);
display_copyright_message();
do_nothing_loop();
do_some_stuff();
if (still_not_crashed)
{
display_copyright_message();
do_nothing_loop();
basically_run_windows_3.1();
do_nothing_loop();
do_nothing_loop();
}
}
if (detect_cache())
disable_cache();
if (fast_cpu())
{
set_wait_states(lots);
set_mouse(speed, very_slow);
set_mouse(action, jumpy);
set_mouse(reaction, sometimes);
}
/* printf("Welcome to Windows 3.1"); */
/* printf("Welcome to Windows 3.11"); */
/* printf("Welcome to Windows 95"); */
/* printf("Welcome to Windows NT 3.0"); */
/* printf("Welcome to Windows 98"); */
/* printf("Welcome to Windows NT 4.0"); */
printf("Welcome to Windows 2000");
if (system_ok())
crash(to_dos_prompt)
else
system_memory = open("a:\swp0001.swp", O_CREATE);
while(something)
{
sleep(5);
get_user_input();
sleep(5);
act_on_user_input();
sleep(5);
}
create_general_protection_fault();
------------------------------------------------------------ Embedded Muse 46 Copyright 2000 TGG March 3, 2000 ------------------------------------------------------------ You may redistribute this newsletter for noncommercial purposes. For commercial use contact info(aatt_sign)ganssle.com. EDITOR: Jack Ganssle, jack(aatt_sign)ganssle.com CONTENTS: - Conspiracy Theory - Cool Product - Upcoming Embedded Seminars - Thought for the Week - About The Embedded Muse Conspiracy Theory ----------------- As a member of the Embedded System Conference's Advisory Board, I get to read all of the attendee's comments about the show. One jumped out at me this week: "After hearing from a number of speakers that I should never design my own RTOS, I'm starting to think this is a conspiracy." Year after year, surveys continue to show that some 70% of all real time OSes are custom made. This blows my mind. Every book on software engineering talks about creating (and using!) reusable code. Yet in the embedded world there's just not a lot of commercial software that we can actually buy/reuse. RTOSes are one of the very few packages commonly available and instantly reusable. Yet it seems most of us don't practice even this minimal amount of reuse. 80 or more different commercial RTOSes exist (see http://www.embedded.com and go to the Buyers Guide section for a listing). These come in just about every conceivable flavor. At least one runs on even the very smallest PIC processors (http://www.sltf.com/simplesoft/micos/index.htm). VxWorks has the largest market share of any commercial RTOS (about 20%), followed (if you discount DOS, which for inscrutable reasons many developers list as a "real time" OS in their embedded apps) by pSOS (about 5%). Both of these now come from Wind River (http://www.windriver.com); at this week's show they announced that the products will eventually merge into a single new OS. Often developers tell me they can't use an off-the-shelf RTOS because of royalty costs. Yet dozens have no royalty payments. Others complain that the up front price is too high: but some cost well under $1000. Every conceivable pricing scheme exists. After one too many engineers told me they write their own OS "because it's just not that big of a job" I grep'ed Jean Labrosse's uC/OS (http://www.ucos-ii.com), and found that it comprised some 4000 lines of C. Since most embedded code costs $15 to $30 per line, this represents $60k to $120k of effort - quite an investment. Now, I do know that this market is completely fragmented, and there are indeed cases where for very narrow and specific reasons some folks will have to write their own OS. This probably represents 2% of the market... not 70%. Just be aware of the real costs! But maybe I'm part of the conspiracy! Cool Product ------------ Dejan Durdenic of Croatia wrote to suggest that a casual mention - not quite a review - of neat products might make sense. Why not? I like the idea of broadly sharing experiences. Feel free to submit your own. Dejan wrote: I found small Italian company named SoftTec Microsystems (www.softecmicro.com). They make very nice logic analyzers and an interesting emulator for the ST6 family of microcontrollers. The products are VERY affordable in those days of rapid technology change when new chips come out almost every day. Embedded Seminars in Boston and San Jose ---------------------------------------- I'll present the seminar "The Best Ideas for Developing Better Firmware Faster" in Boston on April 26 and San Jose on May 3. The focus is uniquely on embedded systems. I'll talk about ways to link the hardware and software, to identify and stamp out bugs, to manage risk, and to meet impossible deadlines. If you're interested reserve early as these seminars fill completely. For more information check out http://www.ganssle.com or email mailto:info(aatt_sign)ganssle.com. A lot of folks have asked me to bring this seminar to their company. Email me at mailto:jack(aatt_sign)ganssle.com if you're interested. Thought for the Week -------------------- From Tom Decker: There was an engineer, manager, and a programmer driving down a steep mountain road. The brakes failed and the car careened down the road out of control. Half way down the driver managed to stop the car by running it against the embankment narrowly avoiding careening off the cliff. They all got out, shaken by their narrow escape from death, but otherwise unharmed. The manager said, "To fix this problem we need to organize a committee, have meetings, and through a process of exchanging ideas, develop a solution." The engineer said, "No that would take too long, besides that method never worked before. I have my trusty pen knife here and will take apart the brake system, isolate the problem and correct it." The programmer said, "I think you're both wrong! I think we should all push the car back up the hill and see if it happens again."
------------------------------------------------------------
Embedded Muse 45 Copyright 2000 TGG February 18, 2000
------------------------------------------------------------
You may redistribute this newsletter for noncommercial purposes. For
commercial use contact info(aatt_sign)ganssle.com.
EDITOR: Jack Ganssle, jack(aatt_sign)ganssle.com
CONTENTS:
- Editor's Notes
- Delayed Sweep
- Upcoming Embedded Seminars
- Thought for the Week
- About The Embedded Muse
Editor's Notes
--------------
The Embedded Systems Conference in Chicago runs from February 28 to March
2. Some 300 vendors will parade their latest nifty embedded tools, software
and gadgets. Don't miss the 150 classes on all aspects of embedded
development.
This is a highly recommended event. Come and visit the show floor and
attend the sessions. Clifford Stoll is repeating his keynote address:
"Stalking the Wily Hacker". You may have read his book, The Cuckoo's Egg. I
attended this talk in San Jose last September; it's pretty hard to describe
other than "wow"! The man is a zany, out-of-control, but fascinating speaker.
Hopefully I'll see you there! See www.embedded.com for more info.
Delayed Sweep
-------------
I had the chance to look at another new scope that will soon hit the
market, one aimed at the mid-level developer and priced accordingly. It's a
very nice piece of gear. However, for some reason the machine lacks delayed
sweep. The vendor tells me that few engineers use this feature.
What a loss! I find delayed sweep almost essential for working on fast
digital signals. Here's an overview:
Just as any decent scope has at least two vertical channels, most include
two time bases as well. Seems odd, doesn't it? Double vertical channels
intuitively makes sense, since each probe picks off a different sense
point. Time, though, always flows in the same direction at the same rate,
so a single axis is all that makes sense.
Novice scope users understand the operation of time base A: crank the
time/division knob to the right and the signal on the screen expands in
size. Rotate it to the left and the signal shrinks, but much more history
(i.e., more microseconds of data) appears.
Time base B is a bit more mysterious. If enabled, it doesn't start until
sometime after time base A begins. Try it on your scope: select "Both" (or
"A intensified by B") and select a sweep rate faster than that used by A.
You'll see a highlighted section of the trace whose width is determined by
B's sweep rate, and whose starting position is a function of the delay time
knob.
Switching from "Both" to "B" shows just the intensified part of the sweep:
the part controlled by time base B. In effect, you've picked out and blown
up a portion of the normal sweep. It's like a zoom control - and you can
select the zoom factor using the sweep time, and the "pan position", or
starting location, using the delay time adjustment.
Suppose you want to look at something that occurs a long time after a
trigger event. Using these zoom controls you can get a very high resolution
view of that event - even when time base A is set to a very slow rate.
Delayed sweep is always accompanied by a second trigger system. Any
instrument with dual time bases will come with a second of these knobs to
set the trigger point of the B channel.
The second trigger is important when working on digital signals that
usually have unstable time relationships. Set the A trigger to start the
sweep (as always), position the intensified part of the sweep to some point
before the section you'd like to zoom on, and then adjust trigger B until
the bright portion starts exactly on the event of interest.
This procedure guarantees that even though the second trigger event moves
around with relationship to trigger A, you'll see a stable scope display
after selecting the B time base. In effect you've qualified trigger B by
trigger A, hopefully zeroing in on the area needing study
Suppose your microprocessor crashes immediately after RESET. Traditional
troubleshooting techniques call for hooking up the logic analyzer and
laboriously examining all of the data and address lines. Personally, I find
this to be too much trouble. Worse yet, it tends to obscure "electrical"
problems: the analyzer might translate marginal ones and zeroes into what
look like legal digital levels. Logic analyzers are great for purely
digital problems, but any problem at power-up can easily be related to
signal levels.
Only a scope gives you a view of those crucial signal levels that can cause
so much trouble. Trigger channel 1 on the RESET input and probe around with
channel 2. Look at READ: every processor starts off with a read cycle to
grab the first instruction or startup vector. You may find a puzzling
phenomena: if the reset is provided by a source asynchronous to the
processor's clock (as is the case with an RC circuit, a Vcc clamp, and even
with many watchdog timers) READ will bounce around with respect to RESET.
You'll never get a nice high resolution view of READ this way.
Triggering off READ will not help. You need to catch the FIRST read after
reset (to look at the first instruction fetch), not any arbitrary read...
and no doubt there will be millions of reads between resets.
The answer is delayed sweep. Put RESET into the scope's external trigger
input and fiddle the knobs till you get a stable trigger. I like to put one
scope channel on the external trigger while doing this initial setup to
make sure the trigger is doing what I expect. Then, connect channel 1 to
your processor's READ output and crank the time base till it appears over
towards the right side of the display. Go to delayed (A intensified by B)
mode, and rotate the B time base trigger adjustment until the bright part
of the trace starts on the leading edge of the bouncing READ signal.
At this point time base A starts the sweep going on the asynchronous RESET,
and time base B triggers the intensified part of the sweep when the first
READ comes along. Flip the Horizontal Mode switch to B (to show only the
intensified part of the sweep - that part after the B trigger), and a
jitter-free READ will be on the left part of the screen. Cool, huh?
With the now stabilized scope display you can use channel 2 to look at the
data lines, ROM chip selects, and other signals during the read cycle. It
becomes a simple matter to see if the first instruction gets fetched
correctly. A lot has to be perfect for this to happen. Very often a
power-up problem comes from a bad data line, chip select or buffer
problem, any of which is trivial to find with the scope triggered properly.
Delayed sweep was available way back in the vacuum tube scopes of
Tektronix's 540-series; it's a shame to see it forsaken today.
Thought for the Week
--------------------
I keep hearing that nobody cool uses goto's in C.
To which I say:
C'mon, give me a break;
else,
I'll continue;
------------------------------------------------------------ Embedded Muse 44 Copyright 2000 TGG February 4, 2000 ------------------------------------------------------------ You may redistribute this newsletter for noncommercial purposes. For commercial use contact info(aatt_sign)ganssle.com. EDITOR: Jack Ganssle, jack(aatt_sign)ganssle.com CONTENTS: - Editor's Notes - File Format Central - Upcoming Embedded Seminars - Thought for the Week - About The Embedded Muse Editor's Notes -------------- I get a lot of response to these Embedded Muses, both positive and at times negative, though always well reasoned and thoughtful. One pattern has become clear, though. Musings on technical subjects seems to be well received, while those about managing the embedded design process sometimes provoke discontent. One wag gently calls those Muses that discuss non-technical issues "content-free". Yet I can't leave process and management issues alone; these are at the very root of the success of any development effort. So, I've decided to partition the management musings into a separate newsletter (Managing Embedded Systems). I'd never automatically subscribe anyone, so if interested, sign up by either dropping me an email (mailto:jack(aatt_sign)ganssle.com) or by entering your email address and hitting the signup button at http://www.ganssle.com. Feel free to pass this on to your boss as well, especially if you feel he/she needs more insight into embedded issues... ;) I also want to assure Muse readers that, despite constant queries, I never make this list available to any other person or organization. Finally, last issue I asked readers to submit info about the good and bad experiences they've had with their toolchains. A handful of readers (thanks all!) sent in comments, all of which are now online at http://www.ganssle.com/tool-rvw/tools.html.But here's a bit of gentle nagging: please do contribute comments on your tools. The more data we acquire, the more useful this service will be to the rest of the industry. A frequent complaint we hear is that developers are so busy doing projects they don't take the time to write or to interact with their peers. The result is a loss to us all, as we all wind up making the same mistakes. Here's a chance to break this habit, to the betterment of everyone. File Format Central ------------------- What's the format of OMF? How about .BMP files or IEEE-695? Did you forget the layout of Intel Hex files? Don't despair, help is here. One very cool website that logs hundreds of file formats, as well as other useful goodies like mouse comm standards, x86 instruction sets, etc, is http://www.wotsit.org. Highly recommended. In researching an upcoming article about USB for Embedded Systems Programming I found that Jan Axelson has a new book about the subject. She's the wiz who brought us "Serial Port Complete" and "Parallel Port Complete". Her new book is an essential resource for USB developers. Her site (http://www.lvr.com) contains links to most every USB data source. Also highly recommended. Thought for the Week -------------------- From Steve Litt at troubleshooters.com: Millennium Pie (with apologies to Don McLean) A long, long time ago... I can still remember how Computers used to make me smile. And I knew if I had my chance, That I could make electrons dance, And maybe I'd be happy for a while. But January made me shiver, it chilled me deep down in my liver, Bad news I'd collected... I couldn't get connected. I can't remember back that day When I first knew the Y2K But something touched me anyway, The day computers died. So, ...Bye, bye to the next digit of Pi Ran my PC on some DC but the voltage was dry And good ol' boys were sending e-mail replies Saying this will be the day I retire this will be the day I retire Can you write in C plus plus ? And do you have faith in your local bus If the driver tells you so? Do you believe in Compaq's goals Can software save your mortal soul And can you teach me how to type real slow ? Well I thought that you were prepared 'Cause your memo said you weren't impaired Your stationery's swell But you can go to hell I was a lonely teenage Unix hack With an incantation and a modem jack but I knew the cat had left the sack The day computers died I started singin'... Bye, bye to the next digit of Pi Ran my PC on some DC but the voltage was dry And good ol' boys were sending e-mail replies Saying this will be the day I retire this will be the day I retire Now for 10 years we've ignored the threat And we haven't solved the problem yet But that's not how it used to be When the luddites read for the king and queen with a light they filled with kerosene And some manuals they stole from you and me And while Bill Gates was looking pleased Time stole his monopolies The courtroom was adjourned No verdict was returned While Apple tried a color scheme The engineers returned to steam And we had purges of their dreams The day computers died We were singin' Bye, bye to the next digit of Pi Ran my PC on some DC but the voltage was dry And good ol' boys were sending e-mail replies Saying this will be the day I retire this will be the day I retire Intel inside in an iron smelter The food leftover from my fallout shelter Twinkies old and aging fast I'd rather eat the grass Q and A tried for a system crash With the tester on the sidelines in a cast Now the timeshare net was running Doom While mainframes played a marching tune We all tried to log in Oh, but we never could begin 'Cause Cobol tried to take the field, And Hollerith refused to yield. Do you recall what was revealed, The day computers died? We started singing Bye, bye to the next digit of Pi Ran my PC on some DC but the voltage was dry And good ol' boys were sending e-mail replies Saying this will be the day I retire this will be the day I retire There we were all in a state A generation- really late With no time left to start again So come on mouse be nimble, mouse be quick Don't let my spreadsheet data stick Cause data is the devil's only friend. As I watched him on my screen My hands and face were drenched in steam No angel born in hell Could run that stupid shell And as the ball climbed high into the night To call the sacrificial night I saw Dick Clark laughing with delight the day computers died. I met a girl with a cell phone And I asked her for a dial tone But she just smiled and turned away I went down to the software store Where I'd seen computers years before But the man there said the games there wouldn't play And in the streets the children screamed The lovers cried and the poets dreamed their interface was spoken The Internet was broken And the three things I connect to most The Website, Lan and the Network host Every single one was toast The day computers died They were singin' Bye, bye to the next digit of Pi Ran my PC on some DC but the voltage was dry And good ol' boys were sending e-mail replies Saying this will be the day I retire this will be the day I retire.
The Embedded Muse ------------------------------------------------------------ Embedded Muse 43 Copyright 2000 TGG January 14, 2000 ------------------------------------------------------------ You may redistribute this newsletter for noncommercial purposes. For commercial use contact info(aatt_sign)ganssle.com. EDITOR: Jack Ganssle, jack(aatt_sign)ganssle.com CONTENTS: - Editor's Notes - Metastability - Conclusion - Thought for the Week - About The Embedded Muse Editor's Notes -------------- In response to many requests I've (finally!) archived all of The Embedded Muses, from issue 1 to this one, at http://www.ganssle.com in .PDF format. I've also rescanned the old Signetics Write Only Memory datasheet (it's now much easier to read); find it at http://www.ganssle.com/misc/wom.html. If you'd like me to present the full-day "Better Firmware Faster" seminar at your company, please drop me a note sooner rather than later, as there's only a limited number of these slots. Finally, are you sick of the toolchain you're using? Or perhaps you think the product is the best thing ever produced. Why not share your experiences with the embedded community? A big problem we all face is selecting reasonable tools. The vendor hype may be right on target... or it might be hype. Wouldn't it be nice if there were a repository of real world user experiences with various tools, that prospective buyers could check out before investing time and money into a product? I've decided to try and collect real user experiences with various toolchains. If you're using a particular compiler, editor, debugger, or other tool let me know what you think of the product and the company. Give me the product version number and approximate date, and a synopsis of the good and the bad points of your experience. I'll post the results (if any!) to my web site. Let me know if I can include your email address so others can correspond directly with you to get more info. Use the following questions as a guideline for replies: - Vendor name: - Tool name: - Tool type (compiler, emulator, etc): - Tool version number: - Approximate date you used this product: - Target CPU: - Host platform (PC, UNIX, etc): - Your email address (optional): - May I include your email address in the listing, so potential users can contact you directly?: - Overall rating of the product on a scale of 1 to 9, 1 being unusable, 5 adequate, and 9 being about perfect: - Overall rating of the vendor's tech support on a scale of 1 to 9, 1 being they never answered you calls/emails, 5 adequate support, and 9 being astonishing support at every step of the way: - Did you manage to complete your project using this tool? - List the product's best features: - List the product's worst features: - General comments. Please do feel free to elaborate: I reserve the right to edit all submissions for clarity and politeness. Metastability - Conclusion -------------------------- The volume of email I've gotten about metastability has been gratifying; it shows a lot of us worry about this issue. One theme that a number of correspondents mentioned was the virtue of gray scale sensors - encoders, for example - since during any code change only a single bit transitions. A metastable state means that you can get only the correct current value of the sensor or the previous value, which is likely a reasonable second choice. Stephen Deasy wrote: The real problem is, I believe, most apparent when you consider an example such as a state machine with multiple flip-flops, say four. If you don't properly observe set up times, the problem is not so much which states the flip-flops take or how long they take to switch, it is that some (flip-flops) will change state based on the new (asynchronous) data and some won't at the time of the clock transition for various reasons (different length propagation delay paths, etcetera. The result is that the system or device will be in the wrong actual state for conditions and may not recover. I have actually seen it work and work reliably to bring asynchronous data into a synchronous state machine. However, in order for this to be true the following conditions were true: 1) One and only one asynchronous input. 2) The flip-flop assignments were made, and the state machine design was done such that one and only one flip-flop can change at any one time (Gray coded sequence). 3) Any time the asynchronous input transitions it will remain in that state long enough for at least two synchronous clocks to occur. My theory is that this works since, because of the Gray code, one and only one flip-flop is allowed to change for any state of the system. At the time of the clock, the asynchronous data will either cause a transition at that one flip-flop or it won't. Thus it will either remain in the current state or transition to the new one. If it didn't make it for this clock, it will definitely make it for the next. Thus the transition can delay one clock period, but it cannot end up in a wrong or out-of-sequence state. Hans J Weedon wrote with more detail about the problem: Digital logic is constructed from real-world devices, and like all amplifying devices they have limitations in gain and bandwidth. The devices may be bipolar transistors, FETs or even tunnel diodes. All the amplifying devices have a device limitation called gain-bandwidth product. This gain-bandwidth product represents the frequency at which the device has a gain of one. As far as the amplifying device is concerned, it has no knowledge of which circuit it is connected into, and only does what the laws of the physics of the device is telling it to do. It is only us the design engineers using the device that observe mysterious circuit behavior. To the device itself there are no mysteries, it does exactly what it is supposed to do. As a thought experiment, let us hook two devices up to be a linear amplifier with negative feedback and a gain of one. When this amplifier is amplifying a step input, the output settles according to an exponential function towards the final output. An exponential function never settles completely, but at some time the difference between the final settling and the actual vale is small enough to be immaterial. If we wait long enough, the error will be small enough to be ignored. Every amplifying device has some random, thermally generated noise associated with it. As a practical engineer we can state that when the final value of a settled signal is less than the noise at the output of the device, any further settling is wasted time because the difference between final value and the settled value has "drowned" in noise. This seems to have nothing to do with metastability in flip-flops, but stay with me, this is just a way of helping us understand the basics of metastability. Back to the settling amplifier. When we write the equation for the voltage waveform at the output of the amplifier, the dominant pole of the amplifier will describe an exponential function on the form e^(-t/tau) where e is the base of the natural logarithm and t is time and tau an expression of the gain-bandwidth product of the amplifier. It may not be obvious to all, but the minus in front of t is there because the amplifier has negative feedback. If we hook the amplifier up with positive feedback, the amplifier will settle according to the expression e^(t/tau). In other words the amplifier will "fly to the rails" no matter how small an input. An amplifier with positive feedback is a flip-flop. The flip-flop acts, for small signals, exactly like an amplifier, except that time appears to be running in reverse. The amplifier cascades from a settled state to the final state at the "rails". With a little bit of "hand-waving" and some intuition, we can calculate how long a metastable state can be expected to last. It is a little bit easier to estimate the parameters of a CMOS flip-flop than for a bipolar flip-flop, especially when Shottky saturation protection is used. For CMOS the delay time from CLK to Q is some number of nanoseconds, at least in former days, the device manufacturers would give us a device schematic of a flip-flop. From that schematic we can tell how many devices there are between the input and the output. Take for instant a CD4013B dual Data master slave flip-flop, it has a delay of 65ns CLK to Q and about 5 devices in the loop. 65/5 =13 , and we can "guesstimate" 13ns delay per stage. A rough guess of the gain-bandwidth product of this device as a linear amplifier would be 1/(2*pi*13ns)=12MHz. The Noise density of CMOS amplifiers is from experience no better than 10nV/rootHz. giving us a noise voltage at the input of about 10E-9*SQR(12E6) = 35 micro volts thermal noise. A CMOS device has a switching threshold of 50% of the rail voltage. The specs I used above were from 10V operating conditions, and the transition from metastability will be from 5V to zero or 5V to 10V, a 5V transition. Ln(5/35E-6)=11.8 That means that the metastability of the CD4013B can be expected to last for 11.8*13ns = 153ns. This estimation assumes that the probability of an error is 50% at the calculated time. for lower probability a larger factor must be used. As a general rule of thumb, the metastability of a flip-flop will last about 2-5 times the propagation delay from CLK to Q out. Texas Instruments has good and complete papers on the issues of metastability in flip-flops, SDYA006 is one of many. This paper lists the measured performance of several families of IC technologies. Among other things it shows that 74S series of flip-flops are among the worst offenders in the area of metastability. The ultimate in resistance to metastability, in my experience, is the fast tunnel diode. When done properly, a tunnel diode will have less than 1ns of metastability. That is one of the reasons that the triggers used for High Energy physics electronics around particle accelerators were done with tunnel diode discriminators. How do you design circuits that has minimum sensitivity to metastability? In my opinion you use 74F type circuits with a clock rate less than 80MHz and you use a two stage synchronization. The asynchronous data is clocked into one flip-flop and is reclocked into a second flip-flop. If the first flop doesn't get it right, the second flop will. Above 80MHz: ECL logic will work up to about 250MHz beyond that ECLIPS logic will work to about 500MHz beyond that, I do not know. Metastability was a big problem in very high speed sloped and successive approximation A/D converters. Thought for the Week -------------------- Mutant Marsupials Take Up Arms Against Australian Air Force The reuse of some object-oriented code has caused tactical headaches for Australia's armed forces. As virtual reality simulators assume larger roles in helicopter combat training, programmers have gone to great lengths to increase the realism of their scenarios, including detailed landscapes and - in the case of the Northern Territory's Operation Phoenix- herds of kangaroos (since disturbed animals might well give away a helicopter's position). The head of the Defense Science & Technology Organization's Land Operations/Simulation division reportedly instructed developers to model the local marsupials' movements and reactions to helicopters. Being efficient programmers, they just re-appropriated some code originally used to model infantry detachment reactions under the same stimuli, changed the mapped icon from a soldier to a kangaroo, and increased the figures' speed of movement. Eager to demonstrate their flying skills for some visiting American pilots, the hotshot Aussies "buzzed" the virtual kangaroos in low flight during a simulation. The kangaroos scattered, as predicted, and the visiting Americans nodded appreciatively... then did a double-take as the kangaroos reappeared from behind a hill and launched a barrage of Stinger missiles at the hapless helicopter. (Apparently the programmers had forgotten to remove that part of the infantry coding.) The lesson? Objects are defined with certain attributes, and any new object defined in terms of an old one inherits all the attributes. The embarrassed programmers had learned to be careful when reusing object-oriented code, and the Yanks left with a newfound respect for Australian wildlife. Simulator supervisors report that pilots from that point onward have strictly avoided kangaroos, just as they were meant to. - Reportedly from June 15, 1999 Defense Science and Technology Organization Lecture Series, Melbourne, Australia, and staff reports
The Embedded Muse ------------------------------------------------------------ Embedded Muse 42 Copyright 2000 TGG January 7,2000 ------------------------------------------------------------ You may redistribute this newsletter for noncommercial purposes. For commercial use contact info(aatt_sign)ganssle.com. EDITOR: Jack Ganssle, jack(aatt_sign)ganssle.com CONTENTS: - Metastability - The Answers - Thought for the Week - About The Embedded Muse Metastability - The Answers --------------------------- Many of you kindly wrote in with your comments about my metastability question: "can a metastable flip flop settle in a random state?" My email inbox has been swamped with your replies! Special thanks to Larry Marks for faxing a number of articles about the subject. There are two general themes in the articles and your replies: 1) Yes, the output of a flop operating in the metastable region is random. It will take perhaps a long time to settle into a state; that state may have no relation to the data at the flop's D input. 2) The question I asked might not have a lot of meaning, since who knows what the correct state is? If the flop's input changes just a nanosecond or so before a clock transition, philosophically the signal is transitioning anyway. Eldridge Mount wrote: In "Digital Design: Principles and Practices", second edition by John F. Wakerly, a rather helpful analogy is given to describe metastability on pages 451 & 452: "Metastable behavior of a bistable element can be compared to the behavior of a ball dropped onto a hill. If we drop the ball from overhead, it will probably immediately roll down to one side of the hill or the other. But if it lands right at the top, it may precariously sit there for a while before random forces (wind, rodents, earthquakes) start it rolling down the hill..." ...and here's the kicker, a direct answer to your question: "...Like the ball at the top of the hill, the bistable may stay in the metastable state for an unpredictable length of time before non-deterministically settling into one stable state or the other." Jon Plotky mentioned a design he had worked on: After the board had been shipping for a while, all of a sudden we started having failures with a certain batch of cards. We found that the problem cards all had Signetics 74S74 devices, while the working cards used other manufacturers. Even more interesting, the problem with the Signetics parts was not that it's metastable state lasted longer. The problem turned out to be that when the input setup time was violated it would sometimes cause improper operation of the OTHER flip-flop in the package! The output of the second flop would change when the first flop went metastable. I never received an explanation of this from Signetics. Jerold Green gave his 5 rules of async design: 1) Asynchronous signals should be clocked with an additional flip flop running at the same frequency as the logic that it drives. The direct input should not be used at all. There should only be one such flip flop, since multiple flip flops monitoring the same signal may clock different data. 2) Two flip flops in series should be used if it is absolutely necessary to use this signal to drive asynchronous logic, such as an asynchronous reset or clock. 3) If you are 'pushing' the device speed, review the metastability data on the device regarding maximum metastable delay vs. the clock rate. 4) Know what signals are asynchronous to your system. If you generate multiple clocks to drive different functions, the interfaces between them can easily be asynchronous. 5) Verify setup delays. Synchronous system speeds are limited to the maximum logic delay plus setup time. A PLD may specify a high speed toggle rate for a shift register (no logic), but multi-level logic will limit the attainable speed. Bas van Rossem summed it all up: "The bottom line is; never trust asynchronously received data." Many thanks to the many, many others who wrote in. Clearly this is a subject of concern to many of us! So here's a scenario that's been bothering me lately. Suppose you're latching parallel data that comes asynchronously. Perhaps a 12 bit word from an encoder. If you latch this data before reading it (to eliminate the problem of having the data change during the read cycle), then you create a possible metastable condition. That is, the latching signal may come just as data starts to change. Now we've got up to 12 flip flops entering a metastable state. Since each output may be random, the CPU may see data that literally has no meaning. If it acts on this meaningless data, bad things are sure to result. So I propose adding a sixth rule to Jerold Green's list: When a program reads asynchronous data, always read twice and compare the results to correct for potential metastable states. Thought for the Week -------------------- Words for the New Millennium: Blamestorming: Sitting around in a group discussing why a deadline was missed or a project failed, and who was responsible. Chainsaw Consultant: An outside expert brought in to reduce the employee headcount, leaving the top brass with clean hands. Cube Farm: An office filled with cubicles. Ego Surfing: Scanning the Net, databases, print media and so on, looking for references to one's own name. 404: Someone who's clueless. "Don't bother asking him; he's 404." From the WWW error message "404 Not Found," meaning the requested document couldn't be located. Idea Hamsters: People who always seem to have their idea generators running. Keyboard Plaque: The disgusting buildup of dirt and crud found on computer keyboards.
Back to the top.