Please raise your hand, if you have ever felt like putting a fork in your left eye due to debugging issues. Below is an attempt to ensure that feeling never comes back, and pave way for you to become a Debug Ninja!
So Why Is Mastering Debugging Important Today?
In earlier times, debugging was something one did, but no one gave it much attention. Data was manageable, and developers didn’t pay much attention on making the process too efficient. Over the past few years, however, communication protocols have become more complex. This is true for interface protocols such as PCIE and USB as well as for bus protocols such as AXI, PLB etc. Today, RTL design and verification engineers have to sift through gigabytes of data in the form of log files and waveform dumps in order to analyze and debug a design.
By some estimates, 70% of IP design time, now-a-days, is spent on verification. A large chunk of that time (~30-40%) is spent in debugging test case failures.
The Step by Step Process to Make Debugging Easy:
So here is the good news. By following the below laid out steps, you can make your debugging process a lot more efficient.
- Symptom identification: Finding out what went wrong (e.g., a data integrity/protocol violation)
- Diagnosis: Analysis of what went wrong for symptom to occur
- Scenario Isolation OR hypothesis formulation: Identifying the exact sequence of events that is unique to issue under consideration
- Scenario conformation OR hypothesis testing: Cross checking that the identified scenario or hypothesis formulated actually resulted in the issue
Let us go through these steps mor carefully, one by one:
Step 1: Symptom Identification
The first step is for the engineer to figure out what went wrong, by looking at high level symptoms. For example, did a data integrity error occur? Or was it a protocol violation? In order to be most efficient, it is important to identify correctly, the broad area of the issue first, and then systematically probe further, once the broad issue is correctly identfied. Not doing this, would lead to a possible willd goose chase.
Step 2: Diagnosis
Next, the engineer needs to perform diagnostics by analyzing what went wrong for the symptom to occur. For example, did the data integrity error occur due to a data packet being dropped?
Step 3: Scenario Isolation or Hypothesis formulation
Once the engineer has correctly diagnozed the problem, he needs to isolate the scenario under which the problem occurs. If its not clear under what all scenarios the problem occurs, he needs to formulate a hypothesis about what kind of scenarios are problematic. For example, is it possible that a data packet is dropped whenever there are 2 back-to-back data packets with the same size?
Step 4: Scenario Confirmation
The engineer now needs to confirm that the scenario he has identified is indeed the problematic case. This can be done by analyzing all the errors seen and checking that the scenario hypothesis holds true. Once the scenario is confirmed, the engineer can rest assured that his fix will correctly address the bug or issue seen.
Following these steps, not only makes the debugging process more efficient, it also leads to standardisation of the process across a team. As a manager, this is important, as he or she can help the team more meaningfully, if they know what stage of debugging process has been reached. It also helps in improving debugging quality.
So is This All I Need to Know?
No. Even if you have mastered the above steps, and follow them religously, there is the problem of sifting through gigabytes of data that we mentioned earlier. A developer needs to extract relevant debug information from huge wave form dumps or log files. This is a problem. The current representations of data are not friendly for the drill down steps described above. In order to go through these steps quickly, the design engineer would need to view the data top-down. This means, the data should be available at multiple abstractions and one should be able to search, filter and mark information quickly -at the click of a button. At Arrow Devices, we saw this as an problem worth solving, and so we went ahead and developed our debugger tool – The Protocol Debug Analyzer (PDA). Check out the details of this tool, on our website link here
The best way to figure out if you need a debugging tool, is to ask the following questions:
1. Does the tool help me drill down to issues top down? In other words, does it give me the option to look at data at various abstractions?
2. Does it allow me to filter and mark events, that I want to drill down to?
3. Can it be used with my current verification environment?
4. Do I need to customise it to the particular protocol I am working on?
5. Does it help standardise the debugging process across the team and share the debug knowledge?
6. Finally, does it help me reduce the time I spend on debugging? Or is it an overhead?