esri / ago-tools

A Python package to assist with administering ArcGIS Online Organizations.

ago-tools

A Python package to assist with administering ArcGIS Online Organizations.

Project Status

This project is no longer being actively developed. In the short term, the intent is to tidy up any outstanding bugs and small fixes. Longer term, I'd like to rewrite all of the examples from this project to work against the new ArcGIS API for Python which will be more robust and ultimately much better maintained than this library. It is actively being developed and you can join the conversation in the beta today.

I will continue to accept new PRs in the meantime if anybody is feeling particularly ambitious or wants to help out with the cleanup.

@ecaldwell July 7, 2016

Features

• Create a spreadsheet of all users in the org
• Update map service urls in webmaps
• Search for new users and add them to a list of groups
• Move (migrate) all items between accounts (single or batch)
• Search a portal
• Update sharing of content in groups
• Remove items from an organization
• Automate registration of items via spreadsheet
• Calculate presence of attachments for features

Installation

1. Download or clone the project.
2. Run python setup.py install from the command line --OR-- add the agoTools folder to your script directory.

Samples

Admin Class

• Create a spreadsheet of all users in the organization
• Add new users to a set of groups
• Migrate items and group ownership/membership between user accounts:
  • Move all items from one account to another, reassign ownership of all groups, and/or add user to another user's groups
  • Migrate account(s)
• Generate a CSV listing the items in the organization
• Register items listed in an input CSV to the organization
• Remove (delete) all items from a designated folder under My Content
• Remove (unshare) all items from a designated group in the organization
• Update any missing thumbnails for items under My Content with a default
• Assign roles to Username and Role from input CSV
• Delete items listed in a CSV from an organization
• Find items containing reference of specific path
• Insert bookmarks into webmap from feature extents
• Calculate status of attachments for features in layer

Utilities Class

• Update map service URLs in web maps
• Update the URL for registered services or web applications
• Search Examples (search cheat sheet):
  • Search for the top 10 most viewed public items in the organization
  • Search for all content owned by a specific user (admin view)

Requirements

• Python
• Notepad or your favorite Python editor

Resources

• Python for ArcGIS Resource Center
• ArcGIS Blog
• twitter@esri

Issues

Find a bug or want to request a new feature? Please let us know by submitting an issue.

Contributing

Esri welcomes contributions from anyone and everyone. Please see our guidelines for contributing.

Please use the following style guides when contributing to this particular repo:

• 4 spaces for indentation
• 'singleQuotes' instead of "doubleQuotes"
• publicFunction() vs __internalFunction__()`
• # Place comments on their own line preceding the code they explain.

Licensing

Copyright 2013 Esri

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

A copy of the license is available in the repository's license.txt file.

[](Esri Tags: ArcGIS-Online Python Tools Library) [](Esri Language: Python)

ago-tools's Issues

Remove defining of roles in getUsers

Currently, if the roles parameter is not supplied when the getUsers function is called, then the default list of roles if applied. Due to the impending custom roles in ArcGIS Online, this set of defaults should be removed.

The new behavior should be: if roles is not passed, then all users are added rather than testing them against a set of defaults.

Thanks @tschwartzman!

Replace user loop with single call to add multiple users in addUsersToGroups

Currently are looping through list of users and adding individually; however, it's likely the undocumented addUsers operation supports passing multiple users at once.

Only original user who setup the ArcGIS Online account can run scripts

Hi there,

I have come across a problem whereby only the original user who setup the ArcGIS Online organisation account can run the registerItems.py script (I have only tested this with this specific script). Other users added following on from this do not seem be allowed run the script, even if they are assigned an 'Administrator' role. I have tested this with two separate organisation accounts and come across the same problem. The following is the error I get (it's the same error that you get when the username/password/portal parameter is wrong):

However, I am definitely using the correct credentials and the users have Administrator roles. Has anyone encountered this problem or is this a requirement?

Kind regards,

Michael

Too many arguments in migrateAccount (reassignAllUser1ItemsToUser2())

Hello,
I'm trying to migrate to enterprise logins and was testing the migrateAccounts sample. When I try using either a single account or a csv file I get the error:

TypeError: reassignAllUser1ItemsToUser2() takes exactly 3 arguments (4 given)

Our enterprise logins are structured like '@city.kingston.on.ca_' (not sure if this is consistent across all enterprise login usernames or specific to us). I wondered whether the issue was coming from the '@' symbol, so I tried to use re.escape to properly format the username:

userTo = re.escape('')
myAgol.migrateAccount('', userTo)

but I'm getting the same error. I've tried lots of different combinations of escape characters and quotes but haven't landed on the right one yet. Any ideas?

Thanks,
Chris

add releases

This will make it easier for people to track major changes and download the repo for usage.

AGOLCat.py - some attributes not populated in output csv

Hi I tried using AGOLCat.py and found that attributes (i.e. name, and size) are populated for some but not all items listed in the ouput. Can someone advise as to why this might be? Thanks - great tool with great potential for assisting AGOL admin... Meg

Enhancement: Create listNewUsers function

Separate addNewUsersToGroups into listNewUsers and addUsersToGroups

Input: daysToCheck
Returns: list of new users in the last 'daysToCheck' days

Enhancement: Create listUsersByRole function

Create a list of all users in a particular role or roles

input: role or roles
returns: list of users matching the provided role(s)

UnicodeEncodeError when running AGOLCat.py sample from command line

Issue when running AGOLCat.py from the command line as shown in the sample with -size True for ArcGIS Online organization.

I believe \u2014 maps to the em dash.

Add getItems function

Add a getItems function (similar to getUsers). One use case is to be able to easily create a spreadsheet of all items with views and other information that can be turned into a PivotTable and used to summarize activity in an organization.

Pull users by Groups

Would there be a way to add an enhancement to the get users script or a new script to specify a group and pull the users just from that group?

Thanks!

remove or retool __getItemFolder__ function

At some point the ownerFolder property showed up in an item's description. The __getItemFolder__ in utilities.py function can probably be removed altogether, but at the least it should be retooled to make a simple request to the item's description endpoint rather than looping through all of a user's content to find a matching ID.

Add Authorized Issuer for Aeration

Delegation order came through for Denise Elston to issue aeration permits. We now need her added to be an Authorized Issuer"

Open Data site doesn't see data after script ran

I ran this script against a feature layer in our AGOL account that is being fed into our Open Data site. Once I ran the script, the layer completly disappeared from our open data site. I don't know why this is. Any thoughts/ideas?

Enhancement: Create stand-alone addUsersToGroup function

Separate addNewUsersToGroups into getNewUsers and addUsersToGroups

New Function: establishCuratedGroup

Add a new function that adds users to a group as org_admin and also sets the group to accept contributions from group admins only.

Inputs: existing groupID and a list of users
Uses: undocumented addUser operation
Returns: nothing (or report?)

Refactor package to remove authentication method for new classes

It's cumbersome to initialize a new Utilities class (requiring username and password) when you've already initialized an Admin class (or vice versa). Rethink how these modules are organized and how authentication should be handled.

Two ideas:

• allow for passing a token parameter to be used for authenticating the class
• remove the User class initialization from the Admin and Utilities classes and instead add a token parameter to all functions

Problem using migrateAccount()

I'm trying to test the migrateAccount() function prior to moving our Org to enterprise logins. Having a problem getting it to run.

Given the script:

from agoTools.admin import Admin
agolAdmin = Admin('pdx.admin')
agolAdmin.migrateAccount('CSMike_demo_cofp', 'mike.quetel')

I'm getting the following runtime error:

C:\Tools>python migrate.py
Password:
Copying all items from CSMike_demo_cofp to mike.quetel...
Traceback (most recent call last):
File "migrate.py", line 3, in <module>
agolAdmin.migrateAccount('CSMike_demo_cofp', 'mike.quetel')
File "C:\Python27\ArcGIS10.4\lib\site-packages\agoTools\admin.py", line 353, in migrateAccount
self.reassignAllUser1ItemsToUser2(self, userFrom, userTo)
TypeError: reassignAllUser1ItemsToUser2() takes exactly 3 arguments (4 given)
C:\Tools>

Would appreciate any assistance in the correct usage as it must be something simple I'm missing. Thanks in advance!

ago-tools / samples / registerItems.py Add note about required fields

RegisterItems.py and corresponding function in admin.py:

I don't know if this is because of the way I am running it, but the registerItems function in the admin script errors out if I don't have all of the optional fields in my csv file, including one field not mentioned, "id".
Once I add all these fields (I don't have to actually populate them), the script runs as expected. Could you add this information to the readme notes?

500 - Internal server error

Attempting to run app in browser yields 500 - Internal Server Error
There is a problem with the resource you are looking for, and it cannot be displayed.
http://bosdemo2.esri.com/Apps/AGOLCat/

Present examples differently.

Suggest moving usage examples/scripts from the README to individual .py files in an "Examples" folder. There are quite a few examples now and I think we've outgrown having them in the README.

createUserListCSV doesn't handle unicode and delimiters in FullName

The full name of a user might have unicode characters. These are not handled correctly at the moment.

dataWriter.writerow([user['fullName'], user['email'], user['username'], user['role'], time.strftime("%Y-%m-%d",time.gmtime(user['created']/1000))])
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 4: ordinal not in range(128)

The full name of a user might contain a delimiter character (such as a comma) so a csv.QUOTE_NONNUMERIC may be more appropriate.

new function: batch register map services from CSV

It would be helpful if there was a method to batch register items from some sort of CSV formatted input. Given, for example, columns for NAME, URL, TAGS, have these registered with the portal.

Unable to log in as Admin using enterprise account

When attempting to log in with the Admin command when using an enterprise account, I get the following error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/******/agoTools/admin.py", line 16, in __init__
    self.user = User(username, portal, password)
  File "/Users/******/agoTools/__init__.py", line 19, in __init__
    self.token = self.__getToken__(self.portalUrl, self.username, self.password)
  File "/Users/******/agoTools/__init__.py", line 30, in __getToken__
    response = urllib.urlopen(url + '/sharing/rest/generateToken?', parameters).read()
KeyError: 'token'

There are no issues logging in with a non-enterprise account. This was done on OSX, have tried from Windows as well.

Sample to update sourceURL for registered service ?

I need to update the sourceURL for a lot of registered layers. Is that currently supported by the api?

The sample updateRegisteredUrlForServiceOrApp.py updates the url that is assigned to the layer by ArcGIS at time of registration, not the layer's sourceURL (a ArcGIs Server service). Any help is much appreciated. Thanks!

registerItems.py fails to add Type: WMS services

The script does not function for services of Type WMS to an ArcGIS Online or Portal for ArcGIS environment. This script executes perfectly for "Map Service" type, but not "WMS" type

CSV Line 1:
title,url,tags,type,id,thumbnail,description,snippet,extent,spatialReference,accessInformation,licenseInfo,culture
CSV Line 2:
WMS_Service,https://sampleserver6.arcgisonline.com/arcgis/services/SampleWorldCities/MapServer/WMSServer?request=GetCapabilities&service=WMS,testtag,WMS,,,WMS,,,,,,

When executed directly, the script returns "0 items added". Watching network traffic, the script never executes the "addItem" call to the Portal sharing API.

Updating Item : File Geodatabase

Does this library support updating a file geodatabase item by uploading a new zip file of a file geodatabase

Last Login

Does anyone know the code to add Last Login to the createUserListCSV.py by chance? 'Last Login' did not work.

Thank you

registerItems.py Language Support

Hi there,

I came across a problem whereby we had a folder that contained the character 'ú'. This then threw an error as below:

It doesn't matter if no folder is selected, this error is still thrown. By the looks of it, ASCII characters are only supported for folder names. For language support it might be worth expanding this to include other characters.

Kind regards,

Michael