Data Format
Data exists in many formats so it is important to understand what data formats SpatialKey supports.
- CSV files (e.g. file containing list of homes for sale, census records, schedule of insurance locations, etc.)
- Vector data (e.g. hail footprints, geopolitical boundaries, etc.)
- Raster data (e.g. flood zones, earthquake rating territories, etc.) – not supported by SpatialKey
- Excel files (e.g. file containing list of homes for same, census records, etc.) – not supported by SpatialKey
- Etc…
SpatialKey currently supports CSV files and Vector data. Any data not in these two formats need to be converted to these formats prior to loading into SpatialKey.
Point datasets (CSV files)
Point datasets should be provided in CSV format. If your data is in excel format, you can easily convert it to a CSV file – see tips on this process in the “Formatting your Dataset” section of Importing Your First Dataset.
If you want to see examples of properly formatted CSV files, check out our Sample CSV Data files.
If you are using a more advanced import method, like the Data Import API, you will need to provide an accompanying XML file. An XML file is simply a file that describes the information configurations that will be used. The CSV and XML files should provided together in a ZIP file.
How to generate an XML file? To generate an XML file, start by importing your dataset into SpatialKey – after your data is imported into SpatialKey, a complete XML file is just a few clicks away. Find your dataset in the Datasets list, click the gear icon to view data settings, and select “Data Import API”.
You will have the option to “generate API config” on the following screen.
After the XML is generated, save the XML file to the same location as your CSV file and then zip the files together. Note: when saving the XML file, save it directly from the browser page that it appears in – do not copy and paste into another file.
Shapefiles (Vector data)
Shapefiles are composed of a few required files
- .shp – the shape format file (contains the geographic (feature) data)
- .shx – the shape index file
- .dbf – the attribute (extended data) file
These 3 files should be zipped together in a ZIP file before loading into SpatialKey. Other files can be optionally included in the ZIP file as needed, e.g. a projection file. For more detail, check out this article on Importing Shapefiles.
Projection File – While most shapefile projections work in SpatialKey, there are some that do not work (one example that doesn’t work is Mercator Auxilary Sphere). If you have any doubts about your projection, you can re-project to WGS84 which is guaranteed to work.
File size
Now that you understand the supported data formats, let’s talk about how big the files can be. File size is very important when considering what to load into SpatialKey. It is important to also consider if the provided point dataset or shapefile meet your organizations limits as well as whether it will perform optimally in SpatialKey. Here are some additional things to look for for point datasets and shapefiles.
Point datasets (CSV files)
How many records and columns does your point dataset contain? If a point dataset contains more records or columns than your SpatialKey limit allows, you will not be able to import it into SpatialKey. If you don’t know your organization’s limits, you can quickly find them in the Data Upload Wizard.
Shapefile (Vector data)
The actual “size” of a shapefile is a little harder to assess than that of a point dataset. As a general rule, shapfiles that are geographically distributed and the shapes themselves are not complex (not many vertices), will work great in SpatialKey. Here are a couple of examples to help you get an idea of what will work well in SpatialKey and what will not.
| ZIP size | Number of shapes |
Avg number of vertices |
Geographical distribution |
Work well in SpatialKey? |
|---|---|---|---|---|
| 2MB | 5 | 10 | Distributed | YES |
| 5MB | 5 | 100,000 | Concentrated | NO |
| 100MB | 10,000 | 100 | Distributed | YES |
| 100MB | 10,000 | 10 | Concentrated | YES |
| 50MB | 1,000 | 100 | Concentrated | NO |
In the 2nd example above, SpatialKey would have trouble rendering the shapefile on a map because the shapes are highly concentrated and the shapefile is complex (large number of vertices). In the 4th example, although the shapes are concentrated, the complexity is low, therefore SpatialKey could render this file well. Note that this table provides some guidance but each shape is unique and should be evaluated based on its components.
What is the size of the ZIP file itself? It’s important to ensure that this size is within your organizations limits. If you don’t know your organization’s limits, you can quickly find them in the Data Upload Wizard under item (1) – see image above.
Note that a small ZIP file doesn’t always mean that the shapefile is small and will work well within SpatialKey – the number of vertices and geographic distribution are the most significant contributing factors.
How many shapes and vertices are included in the shapefile?
While the number of shapes is important to understand, the number of vertices that each shape contains has a larger impact on whether the data will work will in SpatialKey. Sometimes taking an effort to reduce some of the vertices (smooth out the shapes) will help a shapefile work better in SpatialKey.
How many attributes are provided for the shapes?
Large DBF files work great in SpatialKey so quantifying the number of records and columns here isn’t necessary but note that the size of the DBF file contributes to the overall size of the ZIP file.
What is the geographic distribution of the shapes in the shapefile?
This is another key contributing factor of whether a shapefile will work well in SpatialKey. Shapefiles with shapes that are geographically distributed will work great in SpatialKey. If the shapes in a given shapefile are geographically concentrated, SpatialKey will have trouble rendering the shapes at detailed zoom levels. This is a little tricky to assess based on size alone because some 100MB shapefiles that have good geographical distribution work great in SpatialKey but some 10MB shapefiles can be very geographically concentrated and have many vertices per shape and work poorly in SpatialKey.