Extracting and visualizing the location based attributes of your data usually requires complex software with experts trained in Geographic Information Systems (GIS). Within your organization how do you answer the “where” and “when” questions? How can you quickly and easily get this information into the hands of decision makers and allow them to interact with the data? The July 2009 release of SpatialKey helps answer the “where” questions by simplifying the normally complex operation of geographically filtering data. Here are some of the many questions that this release helps you answer:
- How can I see the geographic distribution of sales by account manager across all sales territories in the united states?
- How can I visually compare real estate sales data from two different geographic locations filtered by square footage and number of bedrooms?
- How can I compare sales between this year and last year for a custom user defined geographic area?
- How can I see all auto thefts that occurred within 100 meters of all bus and train stations?
- How can I visualize the distribution of home burglaries by day of week and time of day and quickly filter their locations by time of day?
- How can I view the number of customers within x miles of my store filtered using demographic data by income and age?
- How can I see competitors within 5 miles of my store then geographically overlay my sales data to see how competitor stores are affecting my sales?
The July 2009 release of SpatialKey introduces the ability to create, manage and filter with Polygons within any reports that use a map as well as a new automated approach for data import through our Data Import API. Since the initial beta release of SpatialKey, we have listened closely to our users feedback and focused on taking steps to enhance and improve the product. We will continue to improve and provide regular releases with additional bug fixes and new features. Your feedback is important to this process so please let us know what features would make the biggest difference for you.
In addition to the new features, we also continue to strive towards a stable, error free environment. Within this release you will find improved memory usage, performance and overall stability improvements. These fixes are ongoing and many are in direct action from bugs submitted from our beta testing community. Thank you for helping us to make SpatialKey the best product it can be! Keep the bug reports coming.
Now onto the new features:
- Data Import API overview
- Dataset manager column label editing
- Reordering layers on the map
Creating polygon layers
Poly filtering has been a much sought after feature from our customers. Filtering with polygons includes both custom drawn polygons and filtering by pre-existing shape files (.shp). Polygon filtering is provided through adding a custom layer to a map within a report as shown in Figure 1.1. In this step you can select from an existing Shape layer or create a new one. Within a shape layer you can author multiple polygons and selectively enable/disable them.
|Figure 1.1 – Creating a new shape layer||Figure 1.2 – Naming the shape layer|
After creating and naming a new layer SpatialKey allows you to draw on the map to create custom polygons with a powerful set of drawing tools.
Figure 1.3 – Authoring and naming a polygon
With the drawing tools you can outline complex areas using multiple points. By clicking a black point between any two red and white points you can bisect the line (creating a new red and white point for manipulation) and break it into more sections allowing the polygon to follow complex contours of streets and other geographical boundaries. In addition, any red and white point can be clicked and dragged to change the location of that point. In the Figure 1.4 below a housing tract is outlined with a polygon.
Figure 1.4 – Outlining a housing tract with a polygon
Filtering and buffering with Polygons
SpatialKey shines in its ability to make filtering with polygons very simple. Polygons can be used as a filter to filter for data within the polygon, outside of it, or buffered within a certain distance of it. In Figure 1.5 from the example polygon drawn in Figure 1.4 we are filtering real estate data from several hundred thousand rows down to just a few sales by filtering within the polygon as well as a Timeline Pod. In this example we can see the average home prices in this neighborhood is 119K during 2008. In figure 1.6 the option to exclude items within the poly is selected so that only sales outside of this neighborhood are shown. You can see that the housing prices outside of this neighborhood are much higher in comparison.
Figure 1.5 – Filtering home sales within a polygon
Figure 1.6 – Filtering home sales outside of a polygon
We also have the ability to filter by a buffer zone around the polygon. For example if we wanted to take into account sales within a half mile of this neighborhood that were sold in 2008 to see how it affected the average home price it is very simple to do. Using the buffer option within the Shape Layer Pod simply select a distance and unit of measurement. In this case we used .5 miles. Taking into account the houses within .5 miles of this neighborhood the average price is driven up. In addition to the timeline filter and polygon filtering we could use any of the other filters to narrow this down further. For example, we could limit to similar square footage and number of bedrooms. The combination of filtering is limitless and you can answer nearly any question you throw at SpatialKey by tweaking the many different filtering options.
Figure 1.7 – Filtering real estate sales inside of a polygon and buffering within .5 miles
Using Shape Files
SpatialKey also supports rendering and filtering with shape files (.shp) that were authored in a separate GIS system. Shape files contain one or more polygons and associated data representing custom map features. We also introduced the ability to import Shape Files in this release for an overview see the section on importing shape files below.
The example below illustrates the powerful combination of data and shape files. We can ask complex questions like “How many crimes occured within 100 meters of a bus or train station in the month of October 2006?”. You can see the shape file polygons shown in red and crimes in green. These polygons can be filtered and buffered in exactly the same ways that polygons drawn with the drawing tools can be.
Figure 1.8 – Filtering crime data by a Bus and Train shape layer
Importing Shape Files
Shape files were introduced by ESRI (Environmental Systems Research Institute) as a way to package up not only data (like what is contained in a CSV file imported into SpatialKey) but also geographic information (beyond a single point of latitude and longitude). This geographic information is visualized within SpatialKey as polygons or points and can be used for filtering or buffering. A detailed explanation of the various files that make up a Shape File can be found on the Shapefile Wikipedia page.
In order to successfully import Shape File data into SpatialKey the following criteria must be met:
- Only Shape Files containing Points and Polygons (not LineStrings) may be imported into SpatialKey at this time. LineString will be added in a future release.
- All files should be zipped up into a single .zip file.
- The total size of the Zip file should not exceed 5 megabytes in size.
- One of each of the following file types must be present within the Zip file (required files):
- .shp – the shape format file (contains the geographic (feature) data)
- .shx – the shape index file
- .dbf – the attribute (extended data) file
- If the .prj (spatial projection file) is not present SpatialKey will assume that the geographic data is already in latitude/longitude format (SRID 4326). If the .prj file is present then the data will be properly adapted from the provided projection.
- Other Shape File types may be present as well (may or may not be used by SpatialKey depending on type and purpose) within the Zip file.
The process of importing a shape file is simple in SpatialKey, it is as easy as uploading the zipped file and SpatialKey takes care of the rest. The shape file import process can be found in the “Data sets” tab by clicking the “Upload a Dataset” button on the top right as shown in Figure 1.9. The process for importing a shape file is outlined below:
|Figure 1.9 – Upload a dataset or shape file||Figure 1.10 – Select either a CSV or shape file for upload|
|Figure 1.11 – Select the shape file (.zip) for upload||Figure 1.12 – Sharing the shape file to other users in your organization|
Data Import API overview
We have always made the manual upload of data a simple process through the upload wizard included in the SpatialKey client. While this made the process of uploading data simple it was still a tedious process to maintain those datasets. What if you want to keep those datesets updated daily or even multiple times per day? The Data Import API (DIAPI) comes to the rescue making is simple to automate the creation and management of SpatialKey datasets. The DIAPI consists of simple HTTP calls that developers can utilize in a variety of programming languages pulling data from your backend systems and pushing it into the SpatialKey system for visualization, reporting and sharing.
The DIAPI has two simple steps
- Generate an XML file describing the dataset and a CSV containing the actual data.
- Use the HTTP services to authenticate and upload these assets.
For more information, read this article.
Dataset manager column label editing
Many times it is only after importing your data and creating a report that you realize the name of a column in your data doesn’t make sense. SpatialKey provides a way to relabel the columns in your data with names that are clearer for users of the data. These changes can be made in the DataSet manager by selecting a dataset clicking the manager button then clicking the fields tab. Select any of the columns within the dataset as shown in figure 2.1 and change the label as shown in figure 2.2.
Figure 2.1 – Editing column labels for a dataset
Figure 2.2 – Editing the dataset label
Reordering layers on the map
As you start to add multiple layers of data and shape layers to the map you may want to reorder your layers on the map. For example you may want to bring graduated circles above the heat map of another layer or have the shape files on top so that you can easily see them. You can do this by selecting a layer, clicking and holding and dragging and dropping it in front of or after another layer. The layer to the far right is the top most layer in the stack on the map.
Figure 3.1 – Reordering layers on the map