Exercise 4.1 | Multiple Readers, Writers, and Feature Types |
Data | 3-1-1 case location details (XLS hosted on FTP) Local planning area polygons (KML hosted on FTP) |
Overall Goal | Add additional readers and writers Add a dataset fanout Create a report using the HTMLReportGenerator |
Demonstrates | Creating more complex FME workflows |
Start Workspace | C:\FMEData2018\Workspaces\IntroToDesktop\Ex4.1-Begin.fmw |
End Workspace | C:\FMEData2018\Workspaces\IntroToDesktop\Ex4.1-Complete.fmw |
City planners are undertaking strategic planning and want to know which local planning areas will need additional resources. They have requested the following:
- They like the summary table made earlier, but instead of a single CSV, they want a CSV for each planning area to send to local planners
- Reports of the 3-1-1 requests by local planning areas with tables and charts
- A map they can use to view the local planning areas with summary tables of the requests
You have decided to create an HTML report with tables and charts and a KML (Keyhole Markup Language, a spatial data format used by Google Earth) map to share with them. This task requires adding additional readers and writers to our workspace. In this exercise, we will add a KML reader feature type, a KML writer feature type, and an HTML writer feature type. We will also use a transformer to generate the HTML report.
1) Start Workbench
Start Workbench (if necessary) and open the workspace from Exercise 3.3. Alternatively, you can open C:\FMEData2018\Workspaces\IntroToDesktop\Ex4.1-Begin.fmw.
2) Add a KML Reader
First, let's add a KML reader for the local planning areas. Click Readers > Add Reader, or press Ctrl + Alt + R. Enter the following:
Reader Format | Google KML |
Reader Dataset | https://data.vancouver.ca/download/kml/cov_localareas.kml Note: You can also access the data locally at C:\FMEData2018\Data\Boundaries\LocalAreas.kml |
Your dialog should look like this:
Click OK. When prompted, select only the "local_areas_region" feature type; this is the layer containing polygons of the local areas:
Click OK again to add the reader.
3) Add a KML Writer
Next, let's add a KML writer for the local areas. Click Writers > Add Writer, or press Ctrl + Alt + W. Enter the following:
Writer Format | Google KML |
Writer Dataset | C:\FMEData2018\Output\Training\localareas.kml |
Feature Type Definition | Automatic |
In the next exercise we'll be connecting data from our summary table to this writer feature type, so we chose Automatic for our feature type definition. Your dialog should look like this:
Click OK. The feature type parameters dialog will open. Type in local_areas_region as the name for our writer feature type.
Clicking OK adds the KML writer feature type to the canvas. Click and drag to connect your KML reader feature type local_areas_region to it. This part of your workspace should look like this now:
4) Inspect KML
Once your KML reader and writer feature types are connected, click the KML writer feature type. Then click Run to This. Once the translation is complete, click the Inspect button. You should see the local area polygons:
You can also use the Open Containing Folder button and open localareas.kml with Google Earth if you have it installed:
5) Add an HTMLReportGenerator
To create an HTML report, we will use an HTMLReportGenerator. We want to add it after the StatisticsCalculator, so it uses our summarized data, but we don't want to write out to CSV. So, we'll branch our data stream. Add an HTMLReportGenerator and connect it to the Summary port of the StatisticsCalculator:
Double-click the HTMLReportGenerator to open its parameters. First, set the Group By to Local Area
. Doing so will tell the transformer to create a section of the report for each local area separately.
Next, click where it says Chart (Bar) under Page Contents. This table is used to add elements to the report. From the drop-down, select Header:
Click somewhere under Content Settings to update the parameters for a Header block. For Text, choose the Local Area
attribute. For Header Level, pick H1. These settings give us a header with the local area name in each section of the report. Your dialog should look like this:
Click the cell under Header in Page Contents and select Table from the drop-down:
Click somewhere under Content Settings to update the parameters for a Table block. Under Column settings, we'll create two columns, one for Department and one for Cases. Notice that if you type "Department" or "Cases" into the Column Name, FME will automatically choose the attribute value instead of a constant. To fix this, right-click the cell and choose Open Text Editor, then type in the name of the column. This method will store a constant value instead of supplying an attribute. Your dialog should look like this:
These settings will make an HTML table listing the number of cases by department.
Finally, let's add a Chart (Bar) to the Page Contents. Fill out the parameters like this:
Click OK. You can Run to This on the HTMLReportGenerator and inspect the cache, but all you will be able to see is the raw HTML in Data Inspector. We need to write this data out to an HTML file to view it in a browser.
6) Add an HTML Writer
Here is another way to add a reader or writer: click on a blank space on the canvas and type HTML. You can use the mouse or ↑ and ↓ to browse the Quick Add menu. You should see HTML listed as an option under Writers:
Double-click it or press Enter to add an HTML writer. Use the following parameters:
Writer Format | HTML |
Writer Dataset | C:\FMEData2018\Output\Training\report.html |
Your dialog should look like this:
Once the HTML writer appears, connect it to your HTMLReportGenerator's Output port:
7) View Your HTML Report
Click on the HTML feature type and select Run to This. Once it has run, click on the Open Containing Folder button to open C:\FMEData2018\Output\Training. You should see reports.html
. Open it with your preferred web browser to inspect the heading, tables, and charts:
8) Duplicate and Simplify HTMLReportGenerator
In the next exercise, we will be adding our summary data to the local area polygons. KML files can display HTML content in a "bubble" that users see when they click an object, so we need to get an HTML table of the summary data. In preparation for this step, we will duplicate the HTMLReportGenerator. Right-click it and select Duplicate, or select it and press Ctrl + D.
Then connect the Summary output port of the StatisticsCalculator to the input port of HTMLReportGenerator_2. Your workspace should look like this:
We only need the table for our map, so open the parameters of the HTMLReportGenerator_2 by double-clicking it. Click on Header and then click the Remove Row -
button to remove it:
Then do the same for the Chart (Bar) section. Only the Table section should remain:
Click OK. We will come back to this transformer in the next exercise.
9) Conduct a Dataset Fanout on the CSV Writer Feature Type
As the last step, we will fulfill the planners' request to have a separate CSV file for each local area. We could add an AttributeFilter and 23 distinct CSV feature types, but that would be very inefficient. Instead, we can make use of an FME feature known as fanouts. There are two ways to set up a fanout.
A feature type fanout creates multiple layers in the written file based on attribute values:
A dataset fanout creates multiple files based on attribute values:
For more information, see the FME Help on fanouts.
In this exercise we'll do a dataset fanout. Double-click on the 311-requests-summary writer feature type to open its parameters. Telling FME to write separate files is as simple as setting an attribute to the file name. Click the drop-down arrow next to CSV File Name and select Attribute Value > Local Area
:
Click OK to make the change and then select the CSV writer feature type and use Run to This to write out the separate files. Click the Open Containing Folder button. Now in C:\FMEData2018\Output\Training you should see 23 separate .csv
files:
Now we can supply these files to the local planners that need them.
CONGRATULATIONS |
By completing this exercise, you have learned how to:
|