Checking Key Stages

If the feature counts cannot help to pinpoint the location of a problem, then the next step is inspecting data at key stages of a translation.

Generally, issues in an output dataset occur due to one of these:

• The data was incorrectly read
• The data was incorrectly transformed
• The data was incorrectly written
• The data is being incorrectly interpreted by another application

For example. You open an FME-generated dataset in application X and find that "Parc national des Pyrénées" is rendered as "Parc national des Pyr�n�es".

Obviously an encoding problem has occurred, but where? The fact that data appears incorrect in another application is no real indication of where that problem was introduced.

In this scenario the author should check the data at key stages to determine when the data last looked correct.

Inspect the Source Data Before Reading

It's easy to assume source data is correct without checking it. So, if possible, the source data should be inspected in its native application.

Obviously if the data is incorrect at source (in our example it is "Parc national des Pyr�n�es" in its native application) then there is little chance the translation output will be correct.

Inspect the Source Data After Reading

If the source data is correct in its native application, then inspect it using FME. Either open the data directly in the FME Data Inspector, or - if the workspace was run with caching turned on - open it from within FME Workbench.

If the data is incorrect at this point then the process of reading the data with FME is at fault.

Inspect the Data Before Writing

This is the point most likely to highlight the problem, and so you may wish to start here.

If the workspace was run with caching turned on (Run > Run with Feature Caching) then the data is available for inspection already. Otherwise turn on this option (or Writers > Redirect to FME Data Inspector) and re-run the workspace.

Inspect the data to see if it was correct at the point just before it is written to the output. In the above example, if the Data Inspector shows � characters at this point:

...then the problem appears to be in the data transformation.

Inspect the Output Dataset

If the data is correct before writing, then it might be that it is being written incorrectly.

Open the output dataset in the FME Data Inspector. This will show the data as FME wrote it (and, of course, read it back). If the data is incorrect here, then the problem will have likely occurred during writing of the data.

In the above example, if the Data Inspector shows � characters at this point, then the data has been mangled when it was being written.

Inspect the Data After Transformation

It's best to start by inspecting the data at the point it is about to be written. This is the point most likely to highlight the problem.

If the workspace was run with caching turned on (Run > Run with Feature Caching) then the data is available for inspection already. Otherwise turn on this option (or Writers > Redirect to FME Data Inspector) and re-run the workspace.

Inspect the data to see if it was correct at the point FME just before it is written to the output. In the above example, if the Data Inspector shows � characters at this point:

...then the data was bad before FME even tried to write it. This means the problem is caused in either reading or transforming the data, and you should check those stages.

If the data is correct before writing, then the next step is to inspect the output dataset in FME...

Inspect the Data in a Text Editor

Another check to make is to open the data in a text editor:

For obvious reasons it will not be possible to do this for every dataset (binary files or databases for example) but for text-based files it can provide definitive proof of whether the data is correct at this point.

Inspect the Output Dataset in Another Application

If FME (and a text editor) can display the output data, then it might be that the intended application is not interpreting the data correctly.

So, open the output dataset in the application in which it is intended to be used. If the data is only incorrect here then the problem is more likely to be with how the application interprets the data:

That would be particularly true if the format was non-native to that application (for example, reading a Geodatabase outside of an Esri product).

All of these techniques narrow down where an error might have occurred, but don't always specify the cause. For example, incorrect output could mean that FME has a limitation in that writer or that the workspace author has set an incorrect parameter. Or maybe one application uses a different default encoding to another.

In short, these techniques identify where to investigate first, but won't provide an absolute answer by themselves.