Flex Test: When You Care Enough to Test The VERY BEST

Matthew J. BradleyBradley TechnologiesDecember 28, 2016

1. IntroductionAfter developing tests (particularly in the RF industry) for over 20 years, I have come to some conclusions that are probably not too surprising:

1) Software is usually not re-useable from project to project. If it is re-useable, it usually is not easy.2) Software often changes dramatically throughout the product cycle. This is because different

equipment and often different objectives are involved in each step.3) Data is usually not going to be in the format that you want.4) Changing instruments requires nothing short of an act of Congress. I have seen people struggle

with 40 year old spectrum analyzers because no one wants to touch the software that runs the test system.

5) Everyone says they want traceability. Almost no one will wait for it before going into production.

Finally, everyone says they want to solve these problems. But no one actually wants to pay for the solution or take the time to solve it and so it remains an issue.

As a result, I decided to develop the test framework that would overcome these problems. I have implemented it with a few customers and they have been extremely pleased. I am expanding the types of tests that it performs as well as the instruments supported. I hope you will consider it to meet your engineering and production needs.

Please note that development was performed primarily in LabVIEW.

2. User InterfaceFigure 1 shows the engineering interface, ready to perform a harmonics test. Instrument parameters, test conditions, and limits are all stored in a configuration file. As a result, the exact same test can be called from another test executive (such as TestStand). Furthermore, by using multiple configuration files, different conditions or limits can be tested with the same software.

Figure 1: Flex Test showing the engineering interface for the Harmonics Test. Note that although the plot legend indicates that the one half harmonic (and limit) will be tested, the actual harmonics tested will be determined by the configuration file.

Tabs for additional tests (such as RF calibration, Gain Compression, S Parameters, and more) are also shown. This enables the operator to switch between tests as needed for hardware validation and verification, repair, or other tasks as needed.

It may be desirable, however, to have the ability to queue up several tests and run them consecutively. This can be done through the Multi Test tab, as shown in figure 2.

Figure 2: Not only does this allow the operator to setup a list of tests to run, note that it also gives you the option to set a warm up time before executing each test.

In addition, some tests simply take a long time to execute. Generally, this requires operators to frequently check on systems to determine if the tests have completed, failed, or stopped executing. Failure to check on these systems can lead to lower test system utilization. But, with Flex Test, this does not have to be a problem. Text messages can be sent to groups of users to notify them when tests pass, fail, or abort. In my experience, operators appreciate this feature most of all.

Naturally, the tests themselves re-use a great deal of code. Furthermore, they are highly structured. This decreases development time dramatically. Typically, a new test can be built and ready to go in two days. This is even true for custom tests that are specific to a single product. Existing tests (such as the harmonics test shown below) require even less time to verify.

3. Test ConfigurationConfiguring the tests is straightforward. For example, configuration screens for the harmonics test are shown in figures 3 and 4. Other tests have similar appearances. This is separated from the test interface to give our clients more control over who can change configuration files.

Figure 3: Although the format of configuration files can be customized as needed, we have tried to make them as general as possible. For example, multiple frequency bands can be specified as shown and the Harmonics measurement cluster (in the lower left) can have piecewise linear limits.

Figure 4: This is the same as figure 3, but in this case we are showing that we can test for more than one harmonic at a time. In figure 3, we showed the half harmonic whereas in this case we are showing the first harmonic. There is no limit to the number that can be tested.

4. Output Data FilesWhen I have shown this in the past, the next question that is normally asked concerns the output. There are actually a number of files that are created, including:

Copies of all RF calibrations The configuration file A record of all instruments (including model numbers and serial numbers) The raw data (in Excel compatible format) A summary file (for some tests, also in Excel compatible format) A PDF report (optional). This is typically distributed to customers.

All of these files are placed in the same folder which is specific to the UUT model number, serial number, and the test time. The configuration file and calibration file are stored so that tests can be analyzed or repeated even if the configuration changes or the system is re-calibrated.

A page from a sample report is shown in figure 5. Reports are frequently customized to individual client and customer needs. We take special care to ensure that the reports are precisely what you want and need.

Figure 5: One sample page of a test report. Reports are created from the data files, typically using VBA, and are published to PDF format. Frequently, they include a mix of both plots and text data. We have found that our clients’ customers have been very pleased with both the level of detail and the clarity of presentation.

5. Instrument DriversI have also taken special care in the architecture of the instrument drivers that are used by Flex Test. As stated earlier, the inability to easily change instruments to meet new requirements can present major obstacles to the success of software. Quite frankly, I have found that many of the available drivers are not adequate. Although there are some exceptions, many do not include the functionality that I need or have significant errors. For example, I know of one oscilloscope driver that sets both the time per division as well as the time for the entire sweep within the same vi. Which takes precedence when they conflict? More importantly, why would you have one vi for both?

As a result, I have abstracted the instrument architecture. As an example, let us consider a spectrum analyzer. From the perspective of the harmonics test, we do not set the center frequency for a specific model. Rather, we simply set it for a generic spectrum analyzer. The code determines which spectrum analyzer is being used and sends the appropriate commands. Without dwelling on too many details, rest assured that it is fast and can support multiple instruments of the same type in the same test (such as power supplies).

There is always (or at least should be!) concern, however, when replacing instruments in terms of capabilities. I once came across a test system where they had replaced a spectrum analyzer with a different model but did not realize that the new model could not operate at the same resolution bandwidth settings as the old. This caused enormous problems for them. As a result, our instrument drivers have a “Verify” mode. When turned on, this will check each setting as it is made and confirm that the instrument is configured properly. It will also check to see if there are any error conditions present on the instrument. This can be turned on or off quite easily. I usually leave it on, but have turned it off if it impacts test time.

Finally, software developers often find that the instruments they need are unavailable. RF instrumentation can be quite expensive and the equipment may be needed elsewhere. Therefore, our instrument drivers have a “Live” mode option. When turned on, commands and queries will be sent to the instruments. When turned off, they are not – but work can continue for the rest of the test. This can be set for individual instruments or for all instruments in a system.

6. ConclusionNot surprisingly, there are more details than I could discuss in this description. Some would be technical in nature whereas others would be financial.

I would appreciate hearing from you. In particular, I am interested in:

Which instruments do you need supported? What tests would you like to see? What other features would you want to include?

Please feel free to send them to me at mbradley@brad-tech.com or call me at (707) 837-2893.