Uploading Geodatabases to CartoDB Using Python and Arcpy

The “geodatabase” is a proprietary data format created by ESRI that works a container for spatial and attribute data. It is a very common format used by GIS professionals around the world but usually difficult to use that data outside of ESRI’s stack. Today you will learn how to use Arcpy and Python to automate the process of exporting data from Arcgis Desktop to CartoDB.

As you probably already know, there are tree different types of geodatabases:

  • The “Personal Geodatabase”( based on Microsoft Access),
  • The “File Geodatabase” (based on a file system folder)
  • The “Enterprise Geodatabase” (runs on top of a RDBMS)

Usually GIS users have lots of datasets and features stored in those geodatabases.



Wouldn’t it be nice to visualize and analyze all that data on the web? You can use AGS or AGOL to do so, but you could also have into account that there are other easy to use and cost–effective tools like CartoDB.

If you want to automate the process of exporting from Arcgis to CartoDB you can use Python / Arcpy and the CartoDB Toolbox for Arcgis Desktop.

You will also need “request” package for python:

1
2
python -m  pip install requests
python -m  pip install requests[security]

The following python script describes the process of automation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
# -*- coding: utf-8 -*-
# Requires CartoDB Toolbox for Arcgis http://gkudos.com/blog/2015/05/15/cartodb-toolbox/
# http://stackoverflow.com/questions/29134512/insecureplatformwarning-a-true-sslcontext-object-is-not-available-this-prevent
# python -m  pip install requests[security]

# Import arcpy module
import arcpy
import logging, os
from arcpy import env

os.system('CLS')
# Logging Configuration
logFormat = '%(asctime)-15s %(name)-12s %(levelname)-8s %(message)s'
logfile = "export.log"
logging.basicConfig(level=logging.DEBUG, format=logFormat, filename=logfile,  filemode='w', encoding = "UTF-8" )
logging.debug("***********************************")

#import cartodb toolbox to python environment
arcpy.ImportToolbox("C:/Python27/ArcGIS10.3/Lib/site-packages/gkudos/esri/toolboxes/CartoDBToolbox.pyt")

# Local variables:
personal_geodatabase_path = "C:/data/geodatabases/my_personal_geodatabase.mdb"
env.workspace = personal_geodatabase_path
# List Datasets
featureclasses = arcpy.ListFeatureClasses("*")
for dataset in arcpy.ListDatasets("*", "Feature"):
    logging.debug("*************************************************************************************")
    logging.debug("# dataset: "+dataset)
    dataset_path = os.path.join(personal_geodatabase_path , dataset)
    # List Features
    env.workspace = dataset_path
    for featureName in arcpy.ListFeatureClasses():
        try:
            logging.debug( "uploading to cartodb: "+featureName )
            # Process: Upload data to CartoDB using Cartodb Toolbox
            results = arcpy.CartoDBImportToolbox_CartoDB("cartodb_user", "cartodb_api_key", featureName)
            logging.debug( "results from toolbox:")
            logging.debug( results )
            # Done. Happy Mapping
        except Exception as e:
            logging.error(e)

To execute the script you will have to type something like this

1
python GeodatabaseToCartoDB.py

It will generate a log file discribing the execution:



Done! Now your data is on CartoDB:



Happy Mapping.



Need Help? Contact us for more information



Comentarios